RS485 Stress test

LeoDesigner

@AWI
Thanks for your efforts and persistence :)
Yes, before the packet will be sent, the MyTransport485 will check if there any bytes arrived recently from the serial port. The collision itself can occur during period of the first byte of the packet. (I wish to have time to draw a time diagram for this.)

I think I forgot to mention one more change to the original library:

//The number of SOH to start a message
//some device like Raspberry was missing the first SOH
//Increase or decrease the number to your needs
#define ICSC_SOH_START_COUNT 3

Could you please try to modify this constant ?
It may affect amount of lost packets.
You may also try to increase speed up to 250000 baud.

I have one more idea about collision avoidance:
(In this case we need to make a small modification to the original RS485 module connection, /RE input should be connected to the ground to make receiver always active - then we would be able to receive own messages sent)

We can try to implement the following algorithm:
(Send a packet procedure):
-- check if there any bytes received recently - continue if we have a silence on the bus (done at this moment)
-- send a first SOH (start of header byte)
-- check if it is received by our serial and it's equal SOH.
(so we know at this moment we have no collision on the bus for sure)
-- send next SOH bytes and all packet bytes.

It's like we are sending bytes to the line and reading them at the same time to make sure we had no collision at the same time.
This way we can even resend our packet if collision was detected during packet transmission.

Also we can implement one master and slaves nodes poling mode. But this will not be easy to make in the mySensors protocol architecture.

Let me know what you think about, you probably should have some other ideas too.

AWI

@LeoDesigner We are thinking in the same direction. We could do a little tuning to minimize the collisions/ bus errors but I would like to be absolutely sure that a message is sent and could have been received. If we take some measures like you suggested, I think we could guarantee quality even at lower baud rates and using altsoftserial. I "pseudo coded" my thoughts. Pleas feel free to comment / adjust.

// RS485 collision detection
// algorithm to avoid/ detect bus collision
// considerations, assumptions & conditions:
// message size (incl overhead) 40 bytes
// - timing needs to be in line with bit rate 
//		9600 	~> 1 ms/Byte 	-> 40 ms/ message
// 		19200	~> 500 us/Byte 	-> 20 ms/ message
// 		115200	~> 10 us/Byte	-> 0.4 ms/ message
// no master/ slave - all equal rights
//
do{ // until message sent -or- timeout/ max retries
	// Send phase 1 - avoid collision
	If ( bus activity ){
		// 1. either wait for message to complete (we know the protocol) - and/or -
		//		wait until no more activity == no bytes during x bytes period
		// 2. be gentle and don't always want to be first in line 
		delay( random number of bytes );
		// now we are ready to start
		}
	// Send phase 2 - send and detect collision
	send start_byte_out ;
	read start_byte_in ;
	// Assuming all nodes are aware of and oblige the bus protocol, collision detection in start byte would suffice
	if (start_byte_in != start_byte_out){
			// possible collision so restart phase 1
		}
	for (the rest of the bytes){
		send byte_out ;
		read byte_in ;
		// be absolutely sure that all bytes are sent 
		if (byte_in != byte_out){
			// possible collision so restart phase 1
		}
	}
} while ( < max_retries  || ! messageSent || < timeout)

LeoDesigner

@AWI
The pseudocode looks correct, with one minor addition.
We may have corrupted incoming bytes received (or in our case just dropped) by the USART. So some kind of incoming byte waiting timeout should be implemented.

Please take a look here, to the bus arbitration process (J1708 NETWORK ACCESS part)

0_1478452964585_AN01230A.pdf

I hope we are not reinventing the wheel :)

One more interesting project to check:
https://github.com/MichaelJonker/HardwareSerialRS485/wiki/software-capabilities

kimot

If you all think, that reading message back on "real" (= not 10cm cables on table) RS485 bus by sender guarantee delivering to recipient, you are wrong. RS485 bus do not work like this and message in long distance from sending node with other parallel sending node my looks different on both places. You must use drivers for CAN bus instead of. I wrote about this in different thread.
Then some modified SoftwareSerial library with checking every sent bit and immediately stopping sending when collision detected does best work. But unfortunately nobody wrote this piece of software still ....
I need robust wired network connected to Domoticz via serial gateway too, so I choose CAN protocol on CANbus. CAN controller is not so expensive today. I am using extended 29bit ID, first 7 bits are receivers ID, so I can set mask and filter registers so, that I receive only messages for my node or broadcast.
Others bits in ID and in payload I use for senders ID, sensor ID,message type , data type, data and etc.
I am planning send only short messages from individual sensors, so data length of complete CAN message is enough for me.
But MySensors library is growing and growing and very complex for me, to implement this into it. So I try make my own library for connecting nodes to gateway and use MySensors serial protocol to the controller only.
I am planing Arduino and pure AVR libraries.
I am still coding, when I have got free time, so stay tuned.....

LeoDesigner

@kimot
I have the same concerns regarding long RS485 lines. Also SoftwareSerial cannot send and receive at the same time (not sure about AltSoftSerial).
We would like to make the solution as cheap as we can.
How about sending the message and get ACK back from node/gateway ? (Resend if ACK not received) It is like immediate confirmation of received packet from the other side ? I think it's could be reliable solution. CAN bus is a good solution, but one of the reasons why many people choosing mysensors library is a simplicity and cheapness.
Another way is too look to the modbus - but it's not multimaster bus solution.

AWI

@LeoDesigner The MySensors protocol allows for an ack from the destination although not all controllers support it. I would be more than satisfied if we could make sure that the message left the source in good shape and check with ack if its critical.
Judging from the links you sent there is a good chance we are reinventing the RS485 collision detection. The hardwareserialRS485 looks like a complete solutions, but a bit out of my league...

napo7

I'm also looking for a wired network...
I think, as @kimot said, that we cannot rely on "listen what I just said", since this just ensure that the message we did wanted to say was correctly put on the line, but it doesn't ensure us that it was correctly received !
Perhaps we should use collision AVOIDANCE (be sure that the line is free before speaking), but ALSO use ACKing system.

LeoDesigner

I know about mysensors ACK protocol feature, I was talking about small ACK packet with checksum as a confirmation of the received packet.
So basically the easiest way to improve current situation is to add ACK/CONFIRMATION packets to the existing library. Unfortunately I am really busy right now with main projects.

hausinger

@kimot

I'm very interested in your development.
What kind of can bus module are you using? I found 2 different modules for very less money. The very small module is called SN65HVD230 (controlled by serial rx, tx) and cost approximately 1,75$ from China. The other module with bigger size is called MCP2515 (controlled by spi) and cost approximately 1,26 $ from China. Both are very cheap.
What is you currently status in this case?
I'm planning to use it for my new house in some month.

Thank you

pjr

@kimot @hausinger could you open new topic about can bus transport implementation for MYS? I think there would be a lot of interest about it.

kimot

@pjr @hausinger

Ok.
New topic is here:

https://forum.mysensors.org/topic/5327/can-bus-transport-implementation-for-mys

LeoDesigner

One more idea regarding cheap collision detection on the bus.
We can almost eliminate collisions on the bus (should be almost no lost packets at all).
Before packet transmission we need to check the bus state during one first byte time using digitalRead(rxPin). In this case collision may occur very rarely, because time between bus checking and actual first byte sending is very short.
Also the code modification will be very small in current library.

napo7

Hi,

@LeoDesigner , that's the begining of a way avoiding collisions, but it's not sufficient :
The problem is that if two devices performs that check at exactly the same time (even if it's not so-probable, it can happen), they will start speaking at the same time !

A second step would require HANDLING (and not avoiding) collision : what can (what MUST) we do when two devices speak at the same time ?

That's where CAN drivers become interresting : we can use them on a "simili-RS485" bus : if two devices speak at the same time, the device speaking louder (understand, placing higher bits first...) will be overwriting what second device is saying. So, the first device (the one that is speaking "louder") will not see any problem, it will continue speaking, and the second one (which will see that his bits are not heard) will stop speaking, and try again a few milliseconds later !

pjr

So we need collision AVOIDANCE and DETECTION for (almost?)perfect solution?

AVOIDANCE:

line checking ( digitalRead(rxPin) ) what @LeoDesigner suggested

DETECTION:

"listening what I just said"?
checksum at receiving end?
perhaps similar "hw ACK reply" as the radio is using?

With most important nodes I could additionally use controllers ACK functionality.

msmacs

Hi all,
this is my first post and I would like to congratulate you for this wonderful site :-)
I am also interested in a wired network and I know that rs485 bus has some limitations.. but rs485 modules are also really cheap (and I have already bought a lot of them :laughing: ).

Just an idea to handling the bus.. why don't you use a different size for the first message that Leo propose to check the collision? The size should follow a priority: maximum priority for Controller/Gateway, less priority for Repeater, lower priority for Sensor nodes.
For example
3 start bytes - Controller/Gateway
2 start bytes - Repeater
1 start byte - Sensor Node
In this way if a sensor and the Gateway start sending the byte in the same time, the gateway wins.

This could help also with sensors that need high priority, for example alarms. In this case we only need to change the priority of the sensor that send the message (maybe with a new option on the send() command .. like the ack option).

Max

LeoDesigner

@pjr said:

So we need collision AVOIDANCE and DETECTION for (almost?)perfect solution?

AVOIDANCE:

line checking ( digitalRead(rxPin) ) what @LeoDesigner suggested

DETECTION:

"listening what I just said"?

checksum at receiving end?

perhaps similar "hw ACK reply" as the radio is using?

With most important nodes I could additionally use controllers ACK functionality.

I hope we will finally will come to the right most perfect solution.
Let me say first: I am not saying that we should use only raw RS485. CAN bus is very good idea too. It's good to have alternatives. We can get some ideas from CAN protocol also. I like challenges, and right now it's like a getting something good out from dirt cheap staff.

So, how about this procedure:
Before packet will be sent:

Collision AVOIDANCE: listen rxPin for time of one byte + few bits symbol interval
- if bus is free: wait random time ( few bit's interval, 5-7) and check the bus again
- if bus is free: start transmission of the first byte (start of the packet marker).

Collision DETECTION could be reliable done only via CHECKSUM ACK (small confirmation packet once we are received the input packet).

I think it's should be really rare case when two nodes will start transmission at the same time in case if we will make this random wait interval. Actually IMHO, Ethernet protocol using something like this (randomness before start).

pjr

Is the AVOIDANCE part already there?
I think the "HW" CHECKSUM ACK could be nice addition.

LeoDesigner

@pjr
Yes the AVOIDANCE part is implemented, however it's not perfect yet.
We still have collisions during the first start packet byte.
I really hope to find some time to make necessary patches.

kimot

I am looking to current MyTransportRS485.cpp and it seems, that only one SOH is sent at start of message.

// Start of header by writing multiple SOH
for(byte w=0; w<1; w++) {
_dev.write(SOH);

original:

// Start of header by writing multiple SOH
for(byte w=0;w<ICSC_SOH_START_COUNT;w++) _dev->write(SOH);

When node will lose this start of message, whole message will be lost.
Multiple SOHs can helps bus synchronization for software serial.

LeoDesigner

@kimot
I am absolutely agree about the number of SOH bytes.
Currently in my github code I am using:

//The number of SOH to start a message
//some device like Raspberry was missing the first SOH
//Increase or decrease the number to your needs

#define ICSC_SOH_START_COUNT 3

We need to ask someone to make minor changes to the MyTransportRS485.cpp or submit PR to the MyTransportRS485.cpp.

RS485 Stress test

27

11.7k

11.2k

113.1k