Use Node-Red to Allow Multiple Connections to Ethernet Gateway?

[petewill]

Has anyone used node-red to allow multiple connections to an Ethernet gateway? I am using a w5100 and recently discovered that it doesn't like multiple connections. I am new to node-red and I'm struggling to make this work so I thought I'd check if there was anyone out there who has already accomplished this... Basically what I'd like to do is have node-red connect to the Ethernet gateway then act as a gateway for other controllers to connect to.

I seems to be getting some sort of loop so I'm clearly missing something... If I connect "From MySensors" to port 5003 like this I can get my Vera to read messages.

But, when I try to listen to messages "From Vera" like this I'm no longer able to read messages because it seems to be connecting to itself and creating a loop.

Any idea what I'm missing?

[FotoFieber]

ToVera: is vera on localhost?

Maybe you can describe the setup (ip adresses of node-red, ethernet gateway and vera, ports) and post the flow.

[petewill]

@FotoFieber Thanks for the reply!

I'll do my best to describe it.

Node-red is running on my Pi at 192.168.1.60
Vera is at 192.168.1.70
Ethernet gateway is at 192.162.1.80:5003

Currently my Vera is configured to connect to my Ethernet gatway at 192.162.1.80:5003. I'd like to test other controllers (Domoticz and OpenHAB) which is why I'd like to configure node-red. I have successfully configured node-red to point to 192.162.1.80:5003 and I have Domoticz reading/writing data via a socat created virtual serial connection (credit to @tbowmo for the flow and instructions) . The problem is when the two controllers are trying to read data from my Ethernet gateway it will crash the gateway after a day or two. So, I'd like to have node-red connect to 192.162.1.80:5003 then connect my Vera to my Pi running node-red at 192.168.1.60:5003. I'm not sure if this is even possible though.

I have been able to get Vera to read OR write data to the gateway via node-red but not both.

Here is my (broken) flow:

[{"id":"de5d55d0.21a2a8","type":"serial-port","serialport":"/dev/ttyS80","serialbaud":"115200","databits":"8","parity":"none","stopbits":"1","newline":"\\n","bin":"false","out":"char","addchar":true},{"id":"3352750d.ccad8a","type":"tcp in","z":"4fad2dd7.b052d4","name":"From MySensors","server":"client","host":"192.168.1.80","port":"5003","datamode":"stream","datatype":"utf8","newline":"\\n","topic":"","base64":false,"x":98,"y":330,"wires":[["bb0caa93.44f358","88eba200.77146"]]},{"id":"bb0caa93.44f358","type":"mysdecenc","z":"4fad2dd7.b052d4","name":"","x":298,"y":353,"wires":[["3d08601a.c2f7a"]]},{"id":"3d08601a.c2f7a","type":"function","z":"4fad2dd7.b052d4","name":"Filter ID Requests","func":"if (msg.messageType != 3 &  msg.subType != 3) {\n\tif (msg.messageType == 1 & msg.subType == 0){\n\tmsg.payload = (msg.payload - 32)/1.8\n\t}\n    return msg;\n}","outputs":1,"noerr":0,"x":330,"y":395,"wires":[["2f974230.d068be"]]},{"id":"2f974230.d068be","type":"mysdecenc","z":"4fad2dd7.b052d4","name":"","x":469,"y":316,"wires":[["414476ef.bebb88","b747ff8b.48b8"]]},{"id":"414476ef.bebb88","type":"serial out","z":"4fad2dd7.b052d4","name":"To Domoticz","serial":"de5d55d0.21a2a8","x":630,"y":364,"wires":[]},{"id":"b747ff8b.48b8","type":"debug","z":"4fad2dd7.b052d4","name":"","active":true,"console":"false","complete":"false","x":646.8888888888889,"y":435.3333333333333,"wires":[]},{"id":"fd25d7e7.02da28","type":"serial in","z":"4fad2dd7.b052d4","name":"From Domoticz","serial":"de5d55d0.21a2a8","x":97.14289474487304,"y":713.4285888671875,"wires":[["b09c4bf8.4f63b8"]]},{"id":"e79547e2.186ab8","type":"tcp out","z":"4fad2dd7.b052d4","host":"192.168.1.80","port":"5003","beserver":"client","base64":false,"end":false,"name":"To MySensors","x":741.1428833007812,"y":672.4285888671875,"wires":[]},{"id":"3f257a34.c0da86","type":"debug","z":"4fad2dd7.b052d4","name":"","active":false,"console":"false","complete":"false","x":637.142894744873,"y":816.4285888671875,"wires":[]},{"id":"b09c4bf8.4f63b8","type":"mysdecenc","z":"4fad2dd7.b052d4","name":"","x":274.14289474487304,"y":686.4285888671875,"wires":[["2368952f.dc976a"]]},{"id":"32fd0967.cd02f6","type":"mysdecenc","z":"4fad2dd7.b052d4","name":"","x":483.14289474487304,"y":699.4285888671875,"wires":[["3f257a34.c0da86","e79547e2.186ab8"]]},{"id":"2368952f.dc976a","type":"function","z":"4fad2dd7.b052d4","name":"Filter Var","func":"if (msg.messageType != 2 &\n    msg.subType != 24 &\n    msg.subType != 25 &\n    msg.subType != 26) {\n    return msg;\n}","outputs":1,"noerr":0,"x":392.14289474487304,"y":642.4285888671875,"wires":[["32fd0967.cd02f6"]]},{"id":"88eba200.77146","type":"tcp out","z":"4fad2dd7.b052d4","host":"localhost","port":"5003","beserver":"client","base64":false,"end":false,"name":"To Vera","x":634,"y":222,"wires":[]},{"id":"391d496c.c6e2b6","type":"tcp in","z":"4fad2dd7.b052d4","name":"From Vera","server":"server","host":"localhost","port":"5003","datamode":"stream","datatype":"utf8","newline":"\\n","topic":"","base64":false,"x":97,"y":560,"wires":[["e79547e2.186ab8"]]}]

[Qu3Uk]

Wondering if you got anywhere with this? I was planning on moving to the MQTT Gateway and was wondering if Node-Red would provide TCP connectivity for various controllers as MQTT isn't widely supported etc.

but this sounds like it could work as well?

[petewill]

@Qu3Uk said:

Wondering if you got anywhere with this?

No, not yet. I need to play with it some more. If you're using an serial gateway on a Raspberry Pi it should be working using @tbowmo's setup. I still need to play with @Anticimex's setup to see if I can get Ethernet working in the method described above. I think the key is socat but I just have been working on other projects. Never enough time!

[tbowmo]

I'm loving nodered more and more now. It's so easy to route messages around, and transforming them to suit different things.

I'm still using domoticz as controller, but with nodered I have fixed a "bug" where domoticz have inverted the state of V_LOCK. And I'm able to send sensor data to influxdb as well, and advanced graphs created with grafana.

I have a non mysensor arduino reporting usage from my electricity meter. Nodered is transforming the data to mysensors protocol and injects the data into the mysensors stream towards domoticz and influxdb.

I can also emulate sensors, and new types, in order to check if a given controller is capable of understanding the sensor messages (used it to test some new features in domoticz)

So nodered is a kind of Swiss army knife. It can do almost everything :)

[Qu3Uk]

So I'm pretty sure you can't do what you're trying to do with TCP as NodeRed can't listen on a port multiple times. It seems to see it as two connections as opposed to one connection and accessing that connections input/output.

I guess this might be a security thing because you could use NodeRed as a man in the middle and do all sorts.. which is what we're trying do it. Sit between the gateway and the controllers and do awesome stuff.

I'm sure "To Vera" in your code @petewill needs to listen on the same port "From Vera" is however if you do this you'll only be able to connect to one of the nodes. Once connected the other node will throw a port in use error.

Also I'm pretty sure it's using two connections to the MYS gateway which should be fine as mine seems to handle two connections but if your using nodeRed and have something else directly connected to the GW you may have problems.

I was looking to have NR pretend to be an ethernet gateway and convert MQTT messages back and forth down a TCP connection. Seems it's not the tool for the job.

[petewill]

@Qu3Uk Thanks for the reply. Did you try SOCAT at all? I was hoping that would help. I haven't had a chance to do anything with this recently though.

[Qu3Uk]

@petewill

No i didn't but I don't think I can. I run all my "things" nodered/controllers etc on separate Virtual Machines and communication is all over IP rather than serial/usb.

Sitting on this for a bit see if I come up with anything.

[tbowmo]

@Qu3Uk, @petewill

It should be possible to setup a socat socket converting the tcp stream into a virtual serialport, that could be used within NodeRed as a standard serial port in / out unit. I haven't tried that part yet though, as I'm not using a ethernet GW on my production system. I could set one up though, as I have a development version of the upcomming mysensors GW device (which also is ethernet enabled)

I'll see if I can get the time to look into it, the comming days..

[Qu3Uk]

@tbowmo Have to admit I didn't actually look in to socat (read: Google it).

Seems very possible. I'll have a play tonight.

[petewill]

@tbowmo said:

I'll see if I can get the time to look into it, the comming days..

That would be awesome! No hurry :)

[Qu3Uk]

So I had a go at using SOCAT and although it's working it doesn't seem to like multiple connections on the TCP port. Although it will accept the connection it seems to output data from Serial>Ethernet is a round robin kind of way.

Reason I had two connections to it was to ensure the pipe was always read as the other post it was mentioned if the pipe isn't cleared there would be memory problems and I noticed not all the data was getting to all the TCP connections..

Anyway here is the code if you want to check it out.

Socat

sudo socat PTY,link=/dev/ttyS81,mode=666,group=dialout,raw TCP-LISTEN:40501,fork,reuseaddr & 
sudo socat PTY,link=/dev/ttyS82,mode=666,group=dialout,raw TCP-LISTEN:40502,fork,reuseaddr &

NodeRead

[{"id":"e9998a90.266cc8","type":"serial-port","z":"a89027d9.b1ef08","serialport":"/dev/ttyS82","serialbaud":"9600","databits":"8","parity":"none","stopbits":"1","newline":"\\n","bin":"false","out":"char","addchar":true},{"id":"ee7d4fbe.58c5f","type":"serial-port","z":"a89027d9.b1ef08","serialport":"/dev/ttyS81","serialbaud":"9600","databits":"8","parity":"none","stopbits":"1","newline":"\\n","bin":"false","out":"char","addchar":true},{"id":"50839f8a.fbcaa","type":"debug","z":"a89027d9.b1ef08","name":"Full-Debug","active":true,"console":"false","complete":"true","x":595,"y":575,"wires":[]},{"id":"d5b18044.18e1f","type":"tcp in","z":"a89027d9.b1ef08","name":"MYS-GW-OUT","server":"client","host":"192.168.1.150","port":"5003","datamode":"stream","datatype":"utf8","newline":"\\n","topic":"MYS-GW-Output","base64":false,"x":103,"y":963,"wires":[["db05059b.37e4e8","4f3b7e79.739e6"]]},{"id":"c462686d.bb5a78","type":"catch","z":"a89027d9.b1ef08","name":"","scope":null,"x":387,"y":574,"wires":[["50839f8a.fbcaa"]]},{"id":"9fdda3da.86eb7","type":"debug","z":"a89027d9.b1ef08","name":"PayloadDebug","active":true,"console":"false","complete":"payload","x":584,"y":666,"wires":[]},{"id":"db05059b.37e4e8","type":"serial out","z":"a89027d9.b1ef08","name":"ControllerOne-Out","serial":"ee7d4fbe.58c5f","x":619,"y":875,"wires":[]},{"id":"75f6db75.bba3c4","type":"serial in","z":"a89027d9.b1ef08","name":"ControllerOne-IN","serial":"ee7d4fbe.58c5f","x":103,"y":894,"wires":[["6bbbe087.e897"]]},{"id":"6bbbe087.e897","type":"tcp out","z":"a89027d9.b1ef08","host":"192.168.1.150","port":"5003","beserver":"client","base64":false,"end":false,"name":"MYS-GW-IN","x":605,"y":949,"wires":[]},{"id":"4f3b7e79.739e6","type":"serial out","z":"a89027d9.b1ef08","name":"ControllerTwo-Out","serial":"e9998a90.266cc8","x":625,"y":1044,"wires":[]},{"id":"73b94814.883c88","type":"serial in","z":"a89027d9.b1ef08","name":"ControllerTwo-IN","serial":"e9998a90.266cc8","x":104,"y":1045,"wires":[["6bbbe087.e897"]]},{"id":"8121ed9b.ba597","type":"tcp in","z":"a89027d9.b1ef08","name":"ControllerOne-TCP-Out","server":"client","host":"127.0.0.1","port":"40501","datamode":"stream","datatype":"utf8","newline":"\\n","topic":"","base64":false,"x":151,"y":731,"wires":[["9fdda3da.86eb7"]]}]