MQTT Client Gateway / Node Controller / OTA

[maghac]

Yes, I was hoping that since the only interface to the gateway is actually through MQTT, the controller wouldn't "own" it and MyController could also connect to it.

I didn't actually get as far as MyController since I never managed to burn the bootloader required for OTA updates and haven't had time to play with it more :)

[Samuel235]

@maghac - i'm going to play with the bootloader later today, if i can get that on some nanos (just to test) then i will load up MyController and connect to the MQTT. If i can get this working i can't see any issues with this plan, as like we have both said, the controller isn't owning the gateway. So, I personally don't see any issues AS LONG AS we do not set any rules or anything to trigger on anything within the MyController environment. The moment we do that it will then conflict with OpenHAB (or any other controller for that matter).

However, I may take a look around to see if we have any decent way of implementing an interface for tablets/scene controllers with MyController, if so this could even be an easier controller. Not sure if it has much capabilities like OpenHAB does with other protocols though. I run a Windows server at home so I'm okay with throwing that onto there if needs be. But i would like to stick with OpenHAB as I have learnt how it works (some what anyway).

Do you know of any info or guides on MyController?

I keep getting MyController and MYSController mixed up too, its a shame that MYSController doesn't support MQTT, that would have been perfect for our needs.

[gohan]

So basically if we want to make ota updates we need to temporarily replace the mqtt gateway with another ethernet gateway, wait for the sensors to reconnect, run updates and restore back the mqtt, correct?

[Samuel235]

@gohan - If MyController is able to push OTA updates over MQTT (which i'm about to test, give me an hour or two) then no. My theory is that because MQTT doesn't connect the controller directly to the gateway then we shouldn't need to shut anything down, only start MyController up and push the updates.

[vikasjee]

@Samuel235 What are your results?

[Samuel235]

@vikasjee - I've upgraded my gateway to MySensorsV2.1 and having some pretty bad results with just powering up. I'm waiting for people with V2.1 MQTT gateways to reply to threads. Do you run with MQTT gateways on V2.1 by any chance?

[maghac]

I am running a v2.1 MQTT Gateway on a NodeMCU (so it connects to the network with wifi) and it works fine.

It's a bit picky with the power supply but I think that's more a hardware than a software issue.

[Samuel235]

@maghac - I'm not sure how that setup works, but are you able to see any sort of serial monitor to debug or anything?

[maghac]

Yes, when I program it with Arduino IDE I use the serial monitor like normal with other boards. Never had any problems with it.

If you go back to an older version, does it work then?

[Samuel235]

@maghac I downgraded my boardDefs last night to 1.6.11 like advised on this forum. I'm about to install arduino IDE 1.6.11 and replace my latest install of Arduino and attempt to use this. What arduino IDE environment do you use, out of curiosity?

[Samuel235]

Exact same behaviour is happening. This is what is being printed to the serial monitor:

IP: 192.168.0.22
0;255;3;0;9;Attempting MQTT connection...
0;255;3;0;9;MQTT connected
0;255;3;0;9;Sending message on topic: Gateway1-out/0/255/0/0/18
0;255;3;0;9;Message arrived on topic: Gateway1-in/4/1/1/0/2
0;255;3;0;9;!TSF:MSG:SEND,0-0-4-4,s=1,c=1,t=2,pt=0,l=1,sg=0,ft=0,st=NACK:0
IP: 192.168.0.22
0;255;3;0;9;Attempting MQTT connection...
0;255;3;0;9;MQTT connected
0;255;3;0;9;Sending message on topic: Gateway1-out/0/255/0/0/18
0;255;3;0;9;Message arrived on topic: Gateway1-in/4/1/1/0/2
0;255;3;0;9;!TSF:MSG:SEND,0-0-4-4,s=1,c=1,t=2,pt=0,l=1,sg=0,ft=0,st=NACK:0
IP: 192.168.0.22
0;255;3;0;9;Attempting MQTT connection...
0;255;3;0;9;MQTT connected
0;255;3;0;9;Sending message on topic: Gateway1-out/0/255/0/0/18
0;255;3;0;9;Message arrived on topic: Gateway1-in/4/1/1/0/2
0;255;3;0;9;!TSF:MSG:SEND,0-0-4-4,s=1,c=1,t=2,pt=0,l=1,sg=0,ft=0,st=NACK:0

I'm not even certain that anything is wrong here. It just keeps reconnecting to the MQTT server, but it is actually working but i'm concerned about why its doing such inside of the serial monitor. I read somewhere that it does a 'touching base' case to keep it alive or something, but i also recall someone saying that was every 15 seconds or so, this is every 8 seconds (about). Do you happen to know what its sending to a MQTT topic 0/255/0/0/18 for, is this standard behaviour or just mine?

[maghac]

Not sure if mine did that, I can check later.

I'm using the latest Arduino IDE (1.8.1 or something like that)

[Samuel235]

@maghac - The same as the one I've been using apart from the last test then. Thanks for confirming that.

If you wouldn't mind checking a serial output on your later, that would be really appreciated.

[maghac]

Type 18 is the heartbeat message type, I think this is the last element in your MQTT topic so it looks fine I think. Can you monitor the MQTT broker and see what messages are passing through it? (e.g if you can see something being sent by the sensors)

[Samuel235]

@maghac - I know the sensors are sending messages, the gateway is responding as it should to them, i was just concerned about the gateway keep connecting itself to the MQTT server, i was a little worried in case there is a feature in Mosquitto broker that declines the connection if it detects any sort of brute force or anything like that.

[Samuel235]

@maghac

On my MQTT server i'm sure enough receiving a message from the gateway every time it says its reconnected in the serial monitor, sending its version number to the server. By the looks of the serial monitor, its actually reconnecting and not just sending a new message to the MQTT topic. I'm confused to why this is doing this, I had just assumed that the heartbeat message is just a re-sending on a message on that topic. Would like to know if this is the normal behaviour of the gateway to keep constantly reconnecting.

[maghac]

Mine doesn't reconnect. Here's the serial monitor on startup:

0;255;3;0;9;MCO:BGN:INIT GW,CP=RNNGE--,VER=2.1.1
0;255;3;0;9;TSF:LRT:OK
0;255;3;0;9;TSM:INIT
0;255;3;0;9;TSF:WUR:MS=0
scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 14
cnt 

connected with hacker, channel 9
dhcp client start...
0;255;3;0;9;TSM:INIT:TSP OK
0;255;3;0;9;TSM:INIT:GW MODE
0;255;3;0;9;TSM:READY:ID=0,PAR=0,DIS=0
0;255;3;0;9;MCO:REG:NOT NEEDED
f r0, scandone
....ip:192.168.1.215,mask:255.255.255.0,gw:192.168.1.1
.IP: 192.168.1.215
0;255;3;0;9;MCO:BGN:STP
OTA init
Ready
IP address: 192.168.1.215
0;255;3;0;9;MCO:BGN:INIT OK,TSP=1
IP: 192.168.1.215
0;255;3;0;9;Attempting MQTT connection...
0;255;3;0;9;MQTT connected
0;255;3;0;9;Sending message on topic: mysensors-out/0/255/0/0/18
0;255;3;0;9;Message arrived on topic: mysensors-in/200/255/3/0/6
0;255;3;0;9;!TSF:RTE:200 UNKNOWN
0;255;3;0;9;!TSF:MSG:SEND,0-0-200-200,s=255,c=3,t=6,pt=0,l=1,sg=0,ft=0,st=NACK:M
0;255;3;0;9;Message arrived on topic: mysensors-in/254/255/3/0/6
0;255;3;0;9;!TSF:MSG:SEND,0-0-254-254,s=255,c=3,t=6,pt=0,l=1,sg=0,ft=0,st=NACK:M
0;255;3;0;9;Message arrived on topic: mysensors-in/222/255/3/0/6
0;255;3;0;9;!TSF:MSG:SEND,0-0-222-222,s=255,c=3,t=6,pt=0,l=1,sg=0,ft=0,st=NACK:M
0;255;3;0;9;Message arrived on topic: mysensors-in/222/255/3/0/1
0;255;3;0;9;!TSF:MSG:SEND,0-0-222-222,s=255,c=3,t=1,pt=0,l=10,sg=0,ft=0,st=NACK:1485810579
0;255;3;0;9;Message arrived on topic: mysensors-in/123/255/3/0/6
0;255;3;0;9;!TSF:MSG:SEND,0-0-123-123,s=255,c=3,t=6,pt=0,l=1,sg=0,ft=0,st=NACK:M
0;255;3;0;9;Message arrived on topic: mysensors-in/255/255/3/0/4
0;255;3;0;9;TSF:MSG:SEND,0-0-255-255,s=255,c=3,t=4,pt=0,l=3,sg=0,ft=0,st=OK:234
0;255;3;0;9;Message arrived on topic: mysensors-in/227/255/3/0/6
0;255;3;0;9;!TSF:MSG:SEND,0-0-227-227,s=255,c=3,t=6,pt=0,l=1,sg=0,ft=0,st=NACK:M
0;255;3;0;9;Message arrived on topic: mysensors-in/227/1/1/0/24
0;255;3;0;9;!TSF:MSG:SEND,0-0-227-227,s=1,c=1,t=24,pt=0,l=3,sg=0,ft=0,st=NACK:102
0;255;3;0;9;Message arrived on topic: mysensors-in/233/255/3/0/6
0;255;3;0;9;!TSF:MSG:SEND,0-0-233-233,s=255,c=3,t=6,pt=0,l=1,sg=0,ft=0,st=NACK:M
0;255;3;0;9;Message arrived on topic: mysensors-in/234/255/3/0/6
0;255;3;0;9;!TSF:MSG:SEND,0-0-234-234,s=255,c=3,t=6,pt=0,l=1,sg=0,ft=0,st=NACK:M
0;255;3;0;9;TSF:MSG:READ,227-227-0,s=1,c=1,t=17,pt=5,l=4,sg=0:7160
0;255;3;0;9;Sending message on topic: mysensors-out/227/1/1/0/17
0;255;3;0;9;TSF:MSG:READ,227-227-0,s=1,c=1,t=24,pt=5,l=4,sg=0:530
0;255;3;0;9;Sending message on topic: mysensors-out/227/1/1/0/24
0;255;3;0;9;TSF:MSG:READ,227-227-0,s=1,c=1,t=18,pt=7,l=5,sg=0:0.5300
0;255;3;0;9;Sending message on topic: mysensors-out/227/1/1/0/18
pm open,type:2 0
0;255;3;0;9;TSF:MSG:READ,227-227-0,s=1,c=1,t=17,pt=5,l=4,sg=0:7175
0;255;3;0;9;Sending message on topic: mysensors-out/227/1/1/0/17
0;255;3;0;9;TSF:MSG:READ,227-227-0,s=1,c=1,t=24,pt=5,l=4,sg=0:570
0;255;3;0;9;Sending message on topic: mysensors-out/227/1/1/0/24
0;255;3;0;9;TSF:MSG:READ,227-227-0,s=1,c=1,t=18,pt=7,l=5,sg=0:0.5700

After this, it's just continously getting data from sensor 227 (my power meter). My mosquitto broker is getting this:

mysensors-out/0/255/0/0/18 2.1.1
mysensors-out/227/1/1/0/17 7160
mysensors-out/227/1/1/0/24 530
mysensors-out/227/1/1/0/18 0.5300
mysensors-out/227/1/1/0/17 7175
mysensors-out/227/1/1/0/24 570
mysensors-out/227/1/1/0/18 0.5700
mysensors-out/227/1/1/0/17 7195

etc etc - it's just the one message on startup.

[Samuel235]

@maghac - Right okay. Atleast i now have someone else's to compare too. So, something is pushing it off the network OR it can't hold its connection.

My start up sequence:

0;255;3;0;9;MCO:BGN:INIT GW,CP=RNNGA--,VER=2.1.1
0;255;3;0;9;TSM:INIT
0;255;3;0;9;TSF:WUR:MS=0
0;255;3;0;9;TSM:INIT:TSP OK
0;255;3;0;9;TSM:INIT:GW MODE
0;255;3;0;9;TSM:READY:ID=0,PAR=0,DIS=0
0;255;3;0;9;MCO:REG:NOT NEEDED
IP: 192.168.0.22
0;255;3;0;9;MCO:BGN:STP
0;255;3;0;9;MCO:BGN:INIT OK,TSP=1
IP: 192.168.0.22
0;255;3;0;9;Attempting MQTT connection...
IP: 192.168.0.22
0;255;3;0;9;Attempting MQTT connection...
IP: 192.168.0.22
0;255;3;0;9;Attempting MQTT connection...
IP: 192.168.0.22
0;255;3;0;9;Attempting MQTT connection...
IP: 192.168.0.22
0;255;3;0;9;Attempting MQTT connection...
IP: 192.168.0.22
0;255;3;0;9;Attempting MQTT connection...
IP: 192.168.0.22
0;255;3;0;9;Attempting MQTT connection...
IP: 192.168.0.22
0;255;3;0;9;Attempting MQTT connection...
IP: 192.168.0.22
0;255;3;0;9;Attempting MQTT connection...
IP: 192.168.0.22
0;255;3;0;9;Attempting MQTT connection...
IP: 192.168.0.22
0;255;3;0;9;Attempting MQTT connection...
0;255;3;0;9;MQTT connected
0;255;3;0;9;Sending message on topic: Gateway1-out/0/255/0/0/18
0;255;3;0;9;Message arrived on topic: Gateway1-in/4/1/1/0/2
0;255;3;0;9;!TSF:MSG:SEND,0-0-4-4,s=1,c=1,t=2,pt=0,l=1,sg=0,ft=0,st=NACK:0

The start is different to mine, that is probably because you're running that on a ESP8266, right?

I have connected to the MQTT server with MQTTFX, which is a MQTT monitoring tool. Not too familiar with it right now but i can't see the server dropping any connections or anything, it almost seems like the gateway is stuck in a loop.

[gohan]

Did you try to swap the mega for an arduino uno? Or swap the w5100 shield? Just to see if there is any difference, so we can exclude any hw issues

[Samuel235]

@gohan - The only thing that be causing this is software as the 1.6 branch works perfectly on this setup with the same sketch but back when it was 1.6 development.

So, if i'm thinking correctly, even if there is a issue between software and hardware, it should be fixable through software, hope this makes sense to anyone other than myself. I don't have an UNO, the only other thing i have is a nano. I'm currently using a W5100 shield, not module, but i do have some modules here, but they were a pain to get working before and hence why i made the switch to the shield.

EDIT: I can confirm that the shield is working properly as i have ran a http test sketch and i'm able to communicate from my web browser to the arduino. So it must be a software issue.