Writing a efficient & reliable protocol for slow speed links

[ejeffrey]

As for the protocol, I would suggest using CAN.  It is more or less designed for exactly these sort of constraints -- low speed, small frame size, lots of devices requiring arbitration, and a potentially noise environment requiring error checking.  The shortest CAN data frame is 44 bits which includes framing, device ID, length and CRC.  Of course you can also use X10 directly.

Only create your own serial protocol as a last resort!

[DrGeoff]

Darkman's answer was one that I had used on similar systems, using rs422 or repeated rs232 between nodes (back in the early 1980's). A simple protocol that included device ID and device channel, as well as IO type and command was all that was needed, along with start and end frame characters.

IIRC, I used '#' as the start delimiter, two bytes following for device address ('00' to 'ff' as ascii two chars), a byte for command ('0'-'f') and a byte for channel ('0'-'f') followed by a state byte (depending on the command). All up the longest commands used 10 ascii chars (that's 80 bits) and the IO devices each had 8 digital ins, 8 digital outs and 4 analogue ins. Operation was poll-response, so the master station would poll each device in turn, or send out specific commands to particular devices and channels. The end delimeter was <CR>.

Testing could easily be done using a terminal, or hyperterm, as suggested. Typing something like '#011<CR>' would have the device return the 8 digital inputs, using a start delimiter of '$' (something like '$0113e<cr>', where the state of the 8 DI's was 0x3e). I ran this in a house for 15 years and did not need any error detection or correction. Some error rejection is already present by allowing only the small range of characters '0'-'f' and '#', <CR> in the message, otherwise the message is dropped.

[Mechatrommer]

another technique is to use "variable length encoding" where most frequent command will encodes in lesser bit. search for morse code and huffman encoding.

[Psi]

As for the protocol, I would suggest using CAN.  It is more or less designed for exactly these sort of constraints -- low speed, small frame size, lots of devices requiring arbitration, and a potentially noise environment requiring error checking.  The shortest CAN data frame is 44 bits which includes framing, device ID, length and CRC.  Of course you can also use X10 directly.

Only create your own serial protocol as a last resort!

CAN is probably a bit too complex and 44bits is a bit long but i'll have another look.
X10 is to electrically complex and expensive to implement on a large scale for the price i want.

[Psi]

Then you should also put the oscilloscope in persistent mode and see all the spikes that occur on the power line over a long period of time, since this is what our devices will get... If you are (un)lucky, you will see noise generated by triacs used in light dimmers, universal motors, switching power supply used in small and portable appliances and fluorescent lamps, just to name the more common sources.

I've tried that, you do get some noise when things like electric drills are switched on but its only for a cycle or two. 99.9% of the time its pretty smooth.
At least it is here, maybe i just have very good power.

[Simon]

I've not read the whole topic but, why try and reinvent the wheel ? I can't remember the name and the system is pretty much out of use but there already is a protocol to do what you want that was designed for home automation like you say. I'm sure you could get the transceivers still or just make your own

[Psi]

I've not read the whole topic but, why try and reinvent the wheel ? I can't remember the name and the system is pretty much out of use but there already is a protocol to do what you want that was designed for home automation like you say. I'm sure you could get the transceivers still or just make your own

There's a few, the UPB protocol, X10,  others.
They all look to be more complicated than i really need and i suspect i'd spend more time coding them than to write my own.
Also, I'd want to test that i got an existing codec right and that would mean needing buy devices to test it with.
There is also questions about copyright if i open source the device.

[Mechatrommer]

why try and reinvent the wheel ?

to show and tell people with pride and say "hey! i built this wheel! i dont buy it, its one of a kind on this planet!"

[scrat]

Reading the topic (maybe not with enough attention), I don't realize some things. As Simon asks, why don't you use already made solutions? PowerLine Communications is an entire research branch, and there are quite a few chips designed for that. I know ST, for example, makes them. I don't know about the costs, but I suppose any integrated solution (which is reliable and helps you about compliance with regulations) will be far cheaper than any diy solution, at least for the development cost.

Protocol is still an issue. But at least, if you have some (flexible) kbit/s instead of just a fixed 200bit/s, surely you're going to be more comfortable with robust protocols, which must add some redundance.

CAN is a good idea, but requires some characteristics for the physical layer (open-collector, for example), which I'm not sure are applicable in this case. I2C is clock based...

[Psi]

Reading the topic (maybe not with enough attention), I don't realize some things. As Simon asks, why don't you use already made solutions? PowerLine Communications is an entire research branch, and there are quite a few chips designed for that. I know ST, for example, makes them. I don't know about the costs, but I suppose any integrated solution (which is reliable and helps you about compliance with regulations) will be far cheaper than any diy solution, at least for the development cost.

The available modules on the market for communication over the house powerline are expensive, $50-$200 per unit, not the sort of thing i can afford to put in every pointpoint / lightswitch in the house. And i doubt they would have all the options i would like. eg, Current monitoring, GPIO, triac on/off, Analog inputs, IR remote tx/rx... etc.

Protocol is still an issue. But at least, if you have some (flexible) kbit/s instead of just a fixed 200bit/s, surely you're going to be more comfortable with robust protocols, which must add some redundance.

CAN is a good idea, but requires some characteristics for the physical layer (open-collector, for example), which I'm not sure are applicable in this case. I2C is clock based...

200bits/s is just the official UPB datarate (200bit/s for 50hz / 240bit/s for 60hz. I suspect i can get more, there is plenty more room on the waveform for more than 4 positions per half cycle.

[Psi]

It is foolish to think that you will get any reliable communication over powerline just by being able to send and sense some pulses around your house. All the existing protocols may seem complex compared to such approach (and you may consider them expensive accordingly), but this has not been done lightly: it is the price to pay to get this reliability, and unless you come with a ground-breaking solution, this has already been solved by these powerline standards.

What i'm doing isn't some crazy idea, it's a method of sending data over the powerwiring that is already used in products on the market, it's called UPB (Universal powerline bus)
It's not as well known but it's superior in most ways to X10.
X10's 70-80% reliable while UPB is 99%. Much faster too, which is funny since everyone keeps saying 200bits/s is too slow when X10 is 20bits/s !!
There is plenty of extra room for pulses on UPB so i think 400bits/s is perfectly doable, but since official UPB is 200 i know for sure that is possible. So until i test it at 400 i'm assuming 200

http://en.wikipedia.org/wiki/Universal_powerline_bus

What i'm doing is my own electrical and protocol implementation of UPB, so i can then get pcbs made up for and build 50 or so modules for my house cheaper than the mega $$$ they charge for ones on the market.

At least, you should document yourself on what is existing and try to understand the problems they faced and how they solved them before rolling your own design.

Going back to your protocol question and not even speaking of compliance with regulations, you can't expect to have a working solution without adding enough redundancy / error correction / modulation on such an unreliable medium, unless this is just an experiment you want to carry for fun around your house, but don't expect too much, then .

I don't plan to market these, they're just for me to have some fun with doing home automation without having to fork out lots of money on existing home automation modules.

[johnwa]

Hi,

I have just been through all of this, designing a protocol for my own home automation system.

The physical layer of my system uses RS-485, running over CAT 5 cable in conjunction with a 24V power bus. It is a master-slave system, with the master running on an old laptop via an RS-232 converter.

The baud rate is 9600bps at the moment, which should be slow enough to allow short spurs on the network, rather than a pure bus topology.

I decided to use an ASCII line-based protocol, to simplify debugging. I believe this is the way to go if you have got enough bandwidth. My commands have a target address, optional checksum, command word, and command parameters. For example, '03+FF TOGGLE 2', to toggle channel 2 of node 3.

As other people have explained, using the master-slave system means that you don't have to deal with collisions, although then you have to poll each node. I have found this approach to be a bit slow if the design maximum of 256 nodes are polled in sequence. You might want to look at using a dominant-recessive encoding scheme, like in I2C.

I would agree that you would need some protection from interference if you are just signalling with spikes on the mains. This could be provided with a modulation scheme and/or checksums.

Hopefully this will give you some ideas.

John