Self-build Arduino HA sensors controlled with VoxCommando

[marcusvdt]

If I'm right, each of the values you see in this data that has been recorded by the library you're using in your arduino is the time spent before a bit changed from low to high (or vice versa).
  This means the status of the digital pin which is reading from the IR receiver have changed its status as many times as the quantity of numbers registered,  and each number individually means the time that pin stayed on the same state. As we have large values, two bytes are needed to register each bit time. One single bit (could use the most significant one) could be used to store the state (zero or one).
This is a technique that one can use to repeat data when the protocol is not known whatever is the protocol used, if you repeat the exact same bits in the same timing, that will result in the exact same data in the end. When the protocol is known,  you can read bytes instead because either the hardware or the software knows in advance how the data will come (how in the lowest level, in the bit level)..
So reading from the first,  the input pin stayed in a specific state for 68us (I wonder it's recording in microseconds), then changed to the opposite state for 4500us and so on.
This is kind of a poor's man oscilloscope recording. I've done that in the past for reverse engine a digital protocol.

[jitterjames]

As Dave has pointed out, we need to use the Raw method so that it can work with virtually any device. Raw codes can be quite long, and the maximum size for the MySensors API that we are using is 25 bytes (or characters since that is how messages are passed).

The iTach has a simple but effective method of compressing raw codes. Here is a sample compressed code for controlling my Samsung TV:

Code: [Select]

sendir,1:3,1,37993,1,1,171,171,21,64BB21,21CCC21,22BBBDDCDCBBDBDBDDDDBDBDBB21,4863
Keep in mind that this is just a sample code, and there could be other codes out there that are longer.

I don't fully understand how this code works, but obviously the letters stand for some number. Instead of writing 250,450,450,250 we can write BDDB etc.  Other information is included such as what frequency is used (usually about 38K so we could probably drop this) and what the actual letters stand for. So what is the shortest code you could possibly come up with to represent this? Is that less than 25 characters? 

Probably there is a way to do it. Using a whole byte to represent a letter is very wasteful. We could actually use a byte to represent 4 letter codes. A single byte is 8 bits.

00 = A
01 = B
10 = C
11 = D

BBCA would be represented as 01 01 10 00.  This could be stored in a single character (decimal value is 88 or ascii letter 'X'). This is assuming that the protocol actually allows any byte value from 0-255.  It might only work with ascii characters... I'm not sure.

We would also need to send information about what each letter stands for. Also, there will probably be more than 4 different pulse widths, so anomalous lengths would probably have to be sent directly somehow.

All to say, it might be possible to get it so that MOST codes could fit into a 25 byte message, but it won't be easy, and I think it's highly unlikely that it would be possible for ALL device codes.

This leaves two options.
1 - Use an SD card to store it on the device or do something similar.
2 - Send the message in parts.
or...
3 - Use short protocol codes when they can be detected and fall back to raw codes only when necessary. This would be nice, but does not actually remove the need to find a solution to handle raw codes.
4 - Don't use MySensors (just use an Arduino directly connected to via serial USB to your PC - easy fast cheap solution, but not wireless!)

Again, as Dave has pointed out, for option 2 we have run into problems with the fact that the MySensors protocol running on the NRF chips is not 100% reliable.  The error checking is not perfect.  I occasionally get weird messages from sensors that do not exist (the ID is transmitted incorrectly) or a temperature reading of -240 degrees etc.  If we send multiple longer messages we increase the likelihood of errors.  So we would need to implement our own error checking and possibly ask for the codes to be resent if we detect an error.  This is also possible, but again, not a trivial task!

[nime5ter]

So reading from the first,  the input pin stayed in a specific state for 68us (I wonder it's recording in microseconds), then changed to the opposite state for 4500us and so on.

I believe that the first number in Dave's example -- 68 -- actually indicates the full length of the IR code. That is, it indicates that 68 numbers will follow.

[marcusvdt]

nime5ter, yes it is more likely to be the lenght. If you look at it, it recorded 67 numbers which refer to 67 bit times. BUT one important thing to notice is that the smallest amount of time you see in the data is 550 and I guess any larger value that you see there represents a period of time that the input pin of the arduino stayed for more time in the same logic state. So, when you see 1700 or 1650, it means 3 times the single bit time.
This is not necessarily pure data. Protocols as manchester for example use 2 halfs of a bit to represent a single bit value.

Anyway, I think it is crazy to reverse engineer a protocol nowadays. Most things are already done, we just need to adapt it. That's why I'm interested in looking at this, wondering if I can help with something.

[marcusvdt]

James, I believe the correct way is to add support to the standard Samsung protocol or use a library that supports Samsung.

If the library does not support the protocol natively, then we can always reverse engineer it like I just did with Dave's capture.
It took me more time to make Excel to plot the graphic like I wanted than to actually make the reverse engineering, lol!


This graphic attached is a representation of how the data captured by Dave would look in an oscilloscope. Plus it shows what I was trying to explain.
From this analysis, now I know the code for the button that Dave has captured is actually 0xE0E0D02F (4 bytes long) which fits perfectly in the My Sensors max payload limit of 25 bytes.

The problem with Dave's sketch is one of:
1-The library used does not support the protocol as I showed in the graphic.
2-The library used must be configured correctly so it supports the right protocol.

I'm not sure because I did not have the libraries yet, nor I have researched about them. I only know that there are two well known: IRlib and IRremote.

Either way, I must say it looks really easy to write the sender/receiver routines by myself for this specific protocol. What I mean is that I could write a sketch to support this protocol and consequently we would save in in the PC only the code 0xE0E0D02F plus some extra information that can tell to the arduino what is the actual protocol to use when sending that data through the IR led. The data could easily go into a data table in VC.
This is the easy part.

The difficult part IMHO is to get someone else's code (specifically these IR libraries) and understand how the it does to automatically detect and switch between each protocol when receiving (learning) IR pulses (which in fact are digital 1s and 0s). I suppose that except in this Samsung TV that you guys are having trouble with, the arduino does already switch automatically when learning a new code, right?
If so, then it means the library has some intelligence to detect and switch to a receiving protocol so it can translate the packet and keep the actual code while removing the extra bits (like start bits, stop bits, etc.)


I'll try to start developing a sketch for my own IR blaster on monday. Unfortunately my TVs are either LG, Sony or AOC, so I cannot test the Samsung case myself for now, but I'll need to support it in the near future.
My plan is to use some of the publicly available IR libraries and I really hope I don't have to fix anything on them.
If you guys have more details about what libraries you have used and how you were getting errors with the samsung tv, I can maybe help and get my own code compatible with those tvs.
For example, if you try to learn an IR code using the standard IR receive routine from the lib, what is the captured code for the same button?

[marcusvdt]

Dave, Is the button that you sent the raw data the Volume Down?

[Dave]

Hi Marcus,
we are using the IRremote library.
You are right, it is the code for volume down and the hex code you showed is exactly what I get with the library. But sending this code did not work. If I remember correctly, Kalle got this code to work, when he manually changed the sending routine to use the NECx protocol.

I think even if you manage to change the library, so it can identify the right protocol for Samsung, there will always be some devices where it does not work. E.g. Kalle has a reciever by Unitymedia. He wasn't able to learn codes for this device with this library and even not with his Global iTach, but it does work well with our receiver using raw codes.

There is another benefit of the raw codes: you can easily convert them to the commonly used Pronto Codes. My plan is to name the files with the IR codes after the function and put them in device specific folders on the sd card. Then I want to use Pronto codes instead of the "normal" raw codes. So you could just download the Pronto codes for your device, put it on the sd card and use it with VC without learning every single button yourself.


Edit: Ok, I just saw that using the IRlib library, I can detect the correct protocol (NECx) for my Samsung remote. But there will still be some exotic remotes where it will not work. So I personally would stick with the raw codes. But it shouldn't be too hard change the sketch to send the short codes and the protocol to VC, since this is what I do with the RF codes already.

[marcusvdt]

Dave,  I was not considering the out of standard remotes, which indeed are equally important.
I agree that raw codes should be used for more compatibility. They are more reliable most times and supposedly will work with any remote.

Be aware however, that the IR library that you are using may have a maximum length limitation for saving raw codes. For example, Air Conditioner remotes usually send the full config every time you press a button (kind of a sync between what you see on the remote LCD and the A/C unit). These may be very large packets of data if we are looking only to the bytes. But then it is worse, as you are tracking bits instead of bytes, then the size grows a lot. Add to it that most over the air protocols will have a single bit composed by two bits (just like I showed in the graphic) and hence there are more raw data to save.
So, if instead of 68 bit times you have for example 680? Would it still be able to save it?

Regarding the use of a SD card (which is the reason I started this discussion), I still would like to keep it away from my circuit just for simplicity. But I agree that it has its advantages. Just like you guys, I'll prioritize compatibility.

One last comment since you mentioned the limited size of memory in the arduinos. For saving only raw codes, maybe you can fire away the IRremote lib. I'll explain.
I did not get into details, but I suppose the lib has some significant amount of code (and presumably use of ram) to deal with each protocol, etc. But for saving raw codes, all you need is to monitor the state of the input pin of the arduino (which is connected to the output pin of the IR receiver chip) and use the timer to count the time between each change of state. The timer is a hardware feature on these microcontrollers and it counts time infinitely.
You'll attach an interrupt for that pin and keep looping until the current pin state is different from the previous. Once a change has been detected, you will have an interrupt that calls a function.
On the function that has been called, you will reset the timer because the pin has changed its state and you need to start counting the next one. And you will also save the time elapsed since the last state change in an array.
Once an idle state has been detected, ie a lot of time has passed without any changes to the pin state, you are finished and can save the entire array to the card.
What I mean, is that maybe its better to write you own functions to save the raw data because you don't need all the extra stuff that the library potentially have.

Thanks for the discussion and please keep us updated! 

[Dave]

Hi Marcus,
yes I'm planning to get rid of the IR library. I already have some code working without the library, which (hopefully) should be able to read those long AC codes, and I will try to integrate it in my sketch soon. Unfortunately I don't have one of those ACs (35°C today, I really wish I had one...), so I can not test it.

[marcusvdt]

Unfortunately I have a long way before I get the IR Blaster done here. Otherwise I could run the test for you guys.

[Kalle]

Dave, the codes are already available in the Logitec Database 


and they are really longer as a normal TV remote code, but I was able to capture this code with our IR-Blaster. This is no guarantee if it will work - I can check it at work with a Panasonic AC tomorrow.

[Dave]

The pcb boards arrived! 

I already assembled one and everything works fine!

[jitterjames]

SAWEEEEET! 

[Haddood]

You guys are doing great stuff ... Wish I had more time to catch up ....

[Dave]

And this is how it looks fully assembled and ready to go 

The size is ~ 65 x 45 x 32 mm