Scene too fast for gateway?

[ferpando]

@hek
I got the timer working with this help:

http://forum.micasaverde.com/index.php?topic=10258.0

I'll implement it and see what happens

[ferpando]

@hek

Well, I think it sort of works for now.. still too slow and a lot of polishing might be needed, but the concept I think is valid.

Here I turn on 2 devices on an unplugged node, so the commands are put into the queue and the timer is started. Because the node i sunplugged no response is received.

08 03/20/15 22:13:19.706 JobHandler_LuaUPnP::HandleActionRequest device: 219 service: urn:upnp-org:serviceId:SwitchPower1 action: SetTarget <0x2f7fc680>
08 03/20/15 22:13:19.707 JobHandler_LuaUPnP::HandleActionRequest argument DeviceNum=219 <0x2f7fc680>
08 03/20/15 22:13:19.707 JobHandler_LuaUPnP::HandleActionRequest argument serviceId=urn:upnp-org:serviceId:SwitchPower1 <0x2f7fc680>
08 03/20/15 22:13:19.707 JobHandler_LuaUPnP::HandleActionRequest argument action=SetTarget <0x2f7fc680>
08 03/20/15 22:13:19.707 JobHandler_LuaUPnP::HandleActionRequest argument newTargetValue=1 <0x2f7fc680>
08 03/20/15 22:13:19.708 JobHandler_LuaUPnP::HandleActionRequest argument rand=0.980960675558267 <0x2f7fc680>
50 03/20/15 22:13:19.710 luup_log:130: ****<-*->**** Arduino: ****<-*->**** >F> switchPower <0x2ac6e000>
50 03/20/15 22:13:19.710 luup_log:130: ****<-*->**** Arduino: ****<-*->**** >F> sendCommand <0x2ac6e000>
50 03/20/15 22:13:19.711 luup_log:130: ****<-*->**** Arduino: *** **** **** **** *** *** Sending: 6;0;1;1;2;1 <0x2ac6e000>
04 03/20/15 22:13:19.714 <Job ID="5" Name="" Device="219" Created="2015-03-20 22:13:19" Started="2015-03-20 22:13:19" Completed="2015-03-20 22:13:19

08 03/20/15 22:13:22.567 JobHandler_LuaUPnP::HandleActionRequest device: 221 service: urn:upnp-org:serviceId:SwitchPower1 action: SetTarget <0x2f5fc680>
08 03/20/15 22:13:22.567 JobHandler_LuaUPnP::HandleActionRequest argument DeviceNum=221 <0x2f5fc680>
08 03/20/15 22:13:22.568 JobHandler_LuaUPnP::HandleActionRequest argument serviceId=urn:upnp-org:serviceId:SwitchPower1 <0x2f5fc680>
08 03/20/15 22:13:22.568 JobHandler_LuaUPnP::HandleActionRequest argument action=SetTarget <0x2f5fc680>
08 03/20/15 22:13:22.568 JobHandler_LuaUPnP::HandleActionRequest argument newTargetValue=1 <0x2f5fc680>
08 03/20/15 22:13:22.568 JobHandler_LuaUPnP::HandleActionRequest argument rand=0.3527142092837271 <0x2f5fc680>
50 03/20/15 22:13:22.570 luup_log:130: ****<-*->**** Arduino: ****<-*->**** >F> switchPower <0x2ac6e000>
50 03/20/15 22:13:22.570 luup_log:130: ****<-*->**** Arduino: ****<-*->**** >F> sendCommand <0x2ac6e000>
50 03/20/15 22:13:22.571 luup_log:130: ****<-*->**** Arduino: *** **** **** **** *** *** Sending: 6;1;1;1;2;1 <0x2ac6e000>
04 03/20/15 22:13:22.573 <Job ID="6" Name="" Device="221" Created="2015-03-20 22:13:22" Started="2015-03-20 22:13:22" Completed="2015-03-20 22:13:22"

then it starts to try to resend the messages. i did set a limit of retries and then the message is deleted for good.

50 03/20/15 22:13:24.101 luup_log:130: ****<-*->**** Arduino: -=OOOO-chk-OOOO=- queue length: 2 <0x2cba1680>
50 03/20/15 22:13:24.102 luup_log:130: ****<-*->**** Arduino: -=OOOO-chk-OOOO=- queue : {6;0;1;1;2;1, 6;1;1;1;2;1, } <0x2cba1680>
50 03/20/15 22:13:24.102 luup_log:130: ****<-*->**** Arduino: queue last elem: 6;1;1;1;2;1 retried: 0 <0x2cba1680>
50 03/20/15 22:13:24.102 luup_log:130: ****<-*->**** Arduino: ****<-*->**** -=OOOO-snd-OOOO=- resending: 6;1;1;1;2;1 <0x2cba1680>
50 03/20/15 22:13:24.104 luup_log:130: ****<-*->**** Arduino: -=OOOO-ooo-OOOO=- starting timer <0x2cba1680>

........

50 03/20/15 22:13:49.102 luup_log:130: ****<-*->**** Arduino: -=OOOO-chk-OOOO=- queue length: 2 <0x2cba1680>
50 03/20/15 22:13:49.102 luup_log:130: ****<-*->**** Arduino: -=OOOO-chk-OOOO=- queue : {6;0;1;1;2;1, 6;1;1;1;2;1, } <0x2cba1680>
50 03/20/15 22:13:49.102 luup_log:130: ****<-*->**** Arduino: queue last elem: 6;1;1;1;2;1 retried: 5 <0x2cba1680>
50 03/20/15 22:13:49.103 luup_log:130: ****<-*->**** Arduino: ****<-*->**** -=OOOO-snd-OOOO=- resending: 6;1;1;1;2;1 <0x2cba1680>
50 03/20/15 22:13:49.105 luup_log:130: ****<-*->**** Arduino: -=OOOO-ooo-OOOO=- starting timer <0x2cba1680>

then when I connect the node back online, the messages are delivered

50 03/20/15 22:13:54.179 luup_log:130: ****<-*->**** Arduino: ****<-*->**** >F> processIncoming msg: 6;1;1;1;2;1 <0x2e3fc680>
50 03/20/15 22:13:54.180 luup_log:130: ****<-*->**** Arduino: ****<-*->**** >T> queue : {6;0;1;1;2;1, 6;1;1;1;2;1, } <0x2e3fc680>
50 03/20/15 22:13:54.180 luup_log:130: ****<-*->**** Arduino: ****<-*->**** >T> queue retries: {0, 7, } <0x2e3fc680>
50 03/20/15 22:13:54.181 luup_log:130: ****<-*->**** Arduino: msg found in pos: 2. Removing from list... <0x2e3fc680>
50 03/20/15 22:13:54.182 luup_log:130: ****<-*->**** Arduino: ****<-->**** Set variable: 6;1;1;1;2;1 <0x2e3fc680>
50 03/20/15 22:13:54.182 luup_log:130: ****<-*->**** Arduino: ****<-*->**** >F> setVariable <0x2e3fc680>
50 03/20/15 22:13:54.182 luup_log:130: ****<-*->**** Arduino: Setting variable 'Status' to value '1' <0x2e3fc680>

........

50 03/20/15 22:13:59.101 luup_log:130: ****<-*->**** Arduino: -=OOOO-chk-OOOO=- queue length: 1 <0x2cba1680>
50 03/20/15 22:13:59.101 luup_log:130: ****<-*->**** Arduino: -=OOOO-chk-OOOO=- queue : {6;0;1;1;2;1, } <0x2cba1680>
50 03/20/15 22:13:59.102 luup_log:130: ****<-*->**** Arduino: queue last elem: 6;0;1;1;2;1 retried: 0 <0x2cba1680>
50 03/20/15 22:13:59.102 luup_log:130: ****<-*->**** Arduino: ****<-*->**** -=OOOO-snd-OOOO=- resending: 6;0;1;1;2;1 <0x2cba1680>
50 03/20/15 22:13:59.104 luup_log:130: ****<-*->**** Arduino: -=OOOO-snd-OOOO=- set cheking off queue just emptied <0x2cba1680>
50 03/20/15 22:13:59.209 luup_log:130: ****<-*->**** Arduino: ****<-*->**** >F> processIncoming msg: 6;0;1;1;2;1 <0x2e3fc680>
50 03/20/15 22:13:59.210 luup_log:130: ****<-*->**** Arduino: ****<-*->**** >T> queue : {6;0;1;1;2;1, } <0x2e3fc680>
50 03/20/15 22:13:59.210 luup_log:130: ****<-*->**** Arduino: ****<-*->**** >T> queue retries: {1, } <0x2e3fc680>
50 03/20/15 22:13:59.211 luup_log:130: ****<-*->**** Arduino: msg found in pos: 1. Removing from list... <0x2e3fc680>
50 03/20/15 22:13:59.211 luup_log:130: ****<-*->**** Arduino: ****<-*->**** >T> queue updated: {} <0x2e3fc680>
50 03/20/15 22:13:59.212 luup_log:130: ****<-*->**** Arduino: ****<-->**** Set variable: 6;0;1;1;2;1 <0x2e3fc680>
50 03/20/15 22:13:59.212 luup_log:130: ****<-*->**** Arduino: ****<-*->**** >F> setVariable <0x2e3fc680>
50 03/20/15 22:13:59.212 luup_log:130: ****<-*->**** Arduino: Setting variable 'Status' to value '1' <0x2e3fc680>

I don't really know if this would be a useful way to solve the problem, or maybe I should look somewhere else

[hek]

Looks good, Might be more people finding this useful. Could you prepare a pull request?

Q: Should the queuing/#retries be options in the plugin settings?

[ferpando]

@hek
I use 2 variables that might be user configurable. One is the number of retries and the other is the time between retries.
I suppose there will always be someone willing to change something.

I've been thinking about the approach for a while and I'd like to have your input on this.

Now, as I coded, commands are sent by sendCommandWithMessageType() and then added to the queue. Then the timer starts and tries to re-send lost messages.

What I was thinking is if sendCommandWithMessageType() should send the messages directly to the queue function, and let that manage the sending and ack's.

It would be more organized don't you think?

[hek]

@ferpando

Hmm. do you have your fork on github so I can see the code?

But i like the idea of just queuing up messages. And something else picking up messages from this and send it (could be timer or some explicit call to get a instant pop/send).

[ferpando]

@hek
Here's the link

https://github.com/Fernando-Checa/Vera/blob/master/L_Arduino.lua

Now it has a delay of 5 seconds just for testing, but that should be reduced
Also changed some log calls just for clarity, but that will be discarded later on

[ferpando]

@hek
I have to say dispite this queue upgrade I'm still getting some messages lost. Not really lost, because they are ack'd but never executed by the remote node.

I started to investigate other posibilities, and since I'm using shift registers and optocouplers on some nodes (the problematic ones), maybe those components are just to slow to respond and if I change the register too fast, the optocupler does not have the time to really trigger anything.

Although I think this queue would be a good thing to add to the system, I might need to add something similar to the receiving part to slow down the procesing of messages, because shometimes might be neded.

I didn't try to use relays directly attached to the arduino without the optocouplers, so in that instance might no be really necesary. I read somewhere this particular shift register can be as fast as 500.000 changes per second.

Anyway, I'll think of something for the nodes and let you know.

[hek]

@ferpando said:

I have to say dispite this queue upgrade I'm still getting some messages lost. Not really lost, because they are ack'd but never executed by the remote node.

Ok, if the message reaches the node it must be something else then.

I do like the queue stuff.
But you should probably test it for a while and add the retry delay configuration and max retries easy accessible in the vera gui.

[ferpando]

@hek
Messages now are correctly ack'd because of the queue. Before they were lost.
But that is not all to it, because now the lost messages are re-sent, ack'd back but not executed by the node.

I'll do some more testing and post back

[ferpando]

@hek
I've been trying to get some buttons in the vera plugin but I've had not much luck.
I addded some code to arduino.xml and arduino.json but nothing appears.
I uploaded the files to the repo. Could you take a look when you have the time?

S_Arduino.xml
D_Arduino1.json
L_Arduino.lua

Yesterday I also made a new system for the nodes to have also a queue.
Looks something like this:

#include <MySensor.h>
#include <SPI.h>
#include <MyBuffer.h>

.......

long previousMillis = 0;       // will store last time buffer was processed
long interval = 1000;           // interval at which to check buffer
MyBuffer buffer;               // define a new buffer
MySensor gw;

..........
void setup() {  .... sensor setup code....  }

void loop() 
{ 
  gw.process();
        
  unsigned long currentMillis = millis();
  if(currentMillis - previousMillis > interval) {
        previousMillis = currentMillis;  
        processBuffer();    
      }
} 

void incomingMessage(const MyMessage &message) {
    buffer.push(message.sensor,message.type);
}

//gets message from buffer if exists
void processBuffer(){
if(!buffer.isEmpty()){
      String msg=buffer.pop();
      
      int mIndex = msg.indexOf(';');
      int secondmIndex = msg.indexOf(';,', mIndex+1);

      String firstValue = msg.substring(0, mIndex);
      String secondValue = msg.substring(mIndex+1, secondmIndex);
      
      int sensor = firstValue.toInt();
      int type = secondValue.toInt();
   
      processMsg(sensor, type);
  }
}

//process message from queue
void processMsg(int sensor, int type){
     //do some stuff
}

It's pretty basic but it works.
The library files are here:

MyBuffer.h
MyBuffer.cpp