1.4 Beta

[hek]

The 1.4 version of the MySenors Arduino library is new open for beta testing.

Arduino library and examples
https://github.com/mysensors/Arduino/tree/development

Vera plugin (1.4)
https://github.com/mysensors/Vera/tree/development

Here are some of the hi-lights.

• Improved communication reliability (now uses hardware acks and resend functionality).
• Simplified sketches (only one include needed).
• Most common sleep scenarios build in.
• Helper for permanently storing values in the Arduinos EEPROM.
• Acknowledgments can now be requested from gateway and other sensors in network.
• Smaller footprint.
• The message structure has been adopted to work better on RPi platform.
• Binary payloads supported and used for integers between sensors.
• Configuration message (only metric setting in it today).
• Allow static parent (no dynamic lookups)
• Callbacks for incoming messages and time. No synchronous waiting methods any more -> no missed messages.

All examples in the development branch above has been converted to use the new functionality of the library.

The new API:

/**
* Constructor
*
* Creates a new instance of Sensor class.
*
* @param _cepin The pin attached to RF24 Chip Enable on the RF module (defualt 9)
* @param _cspin The pin attached to RF24 Chip Select (default 10)
*/
MySensor(uint8_t _cepin=9, uint8_t _cspin=10);

/**
* Begin operation of the MySensors library
*
* Call this in setup(), before calling any other sensor net library methods.
* @param incomingMessageCallback Callback function for incoming messages from other nodes or controller and request responses. Default is NULL.
* @param nodeId The unique id (1-254) for this sensor. Default is AUTO(255) which means sensor tries to fetch an id from controller.
* @param repeaterMode Activate repeater mode. This node will forward messages to other nodes in the radio network. Make sure to call process() regularly. Default in false
* @param parentNodeId Use this to force node to always communicate with a certain parent node. Default is AUTO which means node automatically tries to find a parent.
* @param paLevel Radio PA Level for this sensor. Default RF24_PA_MAX
* @param channel Radio channel. Default is channel 76
* @param dataRate Radio transmission speed. Default RF24_1MBPS
*/

void begin(void (* msgCallback)(const MyMessage &)=NULL, uint8_t nodeId=AUTO, boolean repeaterMode=false, uint8_t parentNodeId=AUTO, rf24_pa_dbm_e paLevel=RF24_PA_LEVEL, uint8_t channel=RF24_CHANNEL, rf24_datarate_e dataRate=RF24_DATARATE);

/**
 * Return the nodes nodeId.
 */
uint8_t getNodeId();

/**
* Each node must present all attached sensors before any values can be handled correctly by the controller.
* It is usually good to present all attached sensors after power-up in setup().
*
* @param sensorId Select a unique sensor id for this sensor. Choose a number between 0-254.
* @param sensorType The sensor type. See sensor typedef in MyMessage.h.
* @param ack Set this to true if you want destination node to send ack back to this node. Default is not to request any ack.
*/
void present(uint8_t sensorId, uint8_t sensorType, bool ack=false);

/**
 * Sends sketch meta information to the gateway. Not mandatory but a nice thing to do.
 * @param name String containing a short Sketch name or NULL  if not applicable
 * @param version String containing a short Sketch version or NULL if not applicable
 * @param ack Set this to true if you want destination node to send ack back to this node. Default is not to request any ack.
 *
 */
void sendSketchInfo(const char *name, const char *version, bool ack=false);

/**
* Sends a message to gateway or one of the other nodes in the radio network
*
* @param msg Message to send
* @param ack Set this to true if you want destination node to send ack back to this node. Default is not to request any ack.
* @return true Returns true if message reached the first stop on its way to destination.
*/
bool send(MyMessage &msg, bool ack=false);

/**
 * Send this nodes battery level to gateway.
 * @param level Level between 0-100(%)
 * @param ack Set this to true if you want destination node to send ack back to this node. Default is not to request any ack.
 *
 */
void sendBatteryLevel(uint8_t level, bool ack=false);

/**
* Requests a value from gateway or some other sensor in the radio network.
* Make sure to add callback-method in begin-method to handle request responses.
*
* @param childSensorId  The unique child id for the different sensors connected to this Arduino. 0-254.
* @param variableType The variableType to fetch
* @param destination The nodeId of other node in radio network. Default is gateway
*/
void request(uint8_t childSensorId, uint8_t variableType, uint8_t destination=GATEWAY_ADDRESS);

/**
 * Requests time from controller. Answer will be delivered to callback.
 *
 * @param callback for time request. Incoming argument is seconds since 1970.
 */
void requestTime(void (* timeCallback)(unsigned long));


/**
 * Processes incoming messages to this node. If this is a relaying node it will
* Returns true if there is a message addressed for this node just was received.
* Use callback to handle incoming messages.
*/
boolean process();

/**
 * Returns the most recent node configuration received from controller
 */
ControllerConfig getConfig();

/**
 * Save a state (in local EEPROM). Good for actuators to "remember" state between
 * power cycles.
 *
 * You have 256 bytes to play with. Note that there is a limitation on the number
 * of writes the EEPROM can handle (~100 000 cycles).
 *
 * @param pos The position to store value in (0-255)
 * @param Value to store in position
 */
void saveState(uint8_t pos, uint8_t value);

/**
 * Load a state (from local EEPROM).
 *
 * @param pos The position to fetch value from  (0-255)
 * @return Value to store in position
 */
uint8_t loadState(uint8_t pos);

/**
* Returns the last received message
*/
MyMessage& getLastMessage(void);

/**
 * Sleep (PowerDownMode) the Arduino and radio. Wake up on timer.
 * @param ms Number of milliseconds to sleep.
 */
void sleep(int ms);

/**
 * Sleep (PowerDownMode) the Arduino and radio. Wake up on timer or pin change.
 * See: http://arduino.cc/en/Reference/attachInterrupt for details on modes and which pin
 * is assigned to what interrupt. On Nano/Pro Mini: 0=Pin2, 1=Pin3
 * @param interrupt Interrupt that should trigger the wakeup
 * @param mode RISING, FALLING, CHANGE
 * @param ms Number of milliseconds to sleep or 0 to sleep forever
 * @return true if wake up was triggered by pin change and false means timer woke it up.
 */
bool sleep(int interrupt, int mode, int ms=0);

/**
 * getInternalTemp
 *
 * Read temp from internal (ATMEGA328 only) temperature sensor. This reading is very
 * inaccurate so we round the result to full degrees celsius.
 * http://playground.arduino.cc/Main/InternalTemperatureSensor
 *
 * @return Temperature in full degrees Celsius.
 */
int getInternalTemp(void);

###To convert an old 1.3 sketch follow this guide:

Include section

Remove the following includes
#include <Sleep_n0m1.h>
#include <EEPROM.h>
#include <RF24.h>
#include <Sensor.h>
#include <Relay.h>

Add
#include <MySensor.h>

Global variable scope

Change the following lines

Sensor gw;
or
Relay gw;

To

MySensor gw;

Also message containers for outgoing messages. E.g. Light level message for child sensor id 1.

MyMessage msg(1, V_LIGHT_LEVEL);

####Setup()
In setup() replace sendPresentation with present.
Also note that begin() now allows you to add an function-argument to get callbacks for incoming messages (actuators). begin also controls wether this node should act as an repeater node. See above for full argument list.

####Loop()

The sending of values looks a bit different. The old sketches could look like this:

gw.sendVariable(CHILD_ID_LIGHT, V_LIGHT_LEVEL, lux);

In new code you send a value by using the MyMessage contaner defined in global scope. Fill it with the value to send like this (where lux is light level in this case).

 gw.send(msg.set(lux));

Replace any sleeping with the new build in sleep functions. The old code might have a few lines like this:

delay(500);
gw.powerDown();
sleep.pwrDownMode(); //set sleep mode
sleep.sleepDelay(SLEEP_TIME * 1000);

Replace those with:

gw.sleep(<sleep time in milliseconds>);

[jendrush]

@Hek Ok, so last time i was writing about my problem with latest version of this beta, and Relays not working. When i am sending message, example 11;1;1;2;1 with, or wothout newline char nothing happen. Even stranger is that serial gateway giving me message with 4 segments, istead of normal 5. Look at the screenshoot: http://screenshooter.net/1288732/vlliswb . This is Relay + button, but same thing is with Relay alone.

[BSoft]

While server was down, I've used a recent tab from this post. This file represent the mhl backup

Due to server upload limits, removed one photo and zipped the mhl file. It is like it never happened.

Arduino Library 1_4b1_ Call for beta testers_ MySensors Forum.zip

[MisterE]

This post is deleted!

[MisterE]

@BSoft

Crowd Backup (TM)

[Damme]

Hello

I'm using serialgateway and trying to interface with a linux machine.

I setup serial with a simple stty -F /dev/ttyUSB0 cs8 115200 -onlcr -icrnl
cat /dev/ttyUSB0 works fine;
0;0;3;9;Arduino startup complete.
0;0;3;9;read: 255-255-0 s=255,c=3,t=3,pt=0,l=0,cr=ok:
255;255;3;3;

but I am unable to send data from the gateway
echo -n -e "255;255;3;4;1\n" > /dev/ttyUSB0

Nothing happens, not even in perl I can get it to work.
$port->write("255;255;3;4;1\n");

what am I doing wrong?

[hek]

@Damme said:

$port->write("255;255;3;4;1\n");

Try
$port->write("255;255;3;0;4;1\n");

[Damme]

@hek

Thanks, that worked. I thought I understood the message layout.. never seen that 0 before:)

[hek]

@Damme said:

Thanks, that worked. I thought I understood the message layout.. never seen that 0 before:)

It's a change in the serial protocol for 1.4.
1 = request ack back from destination node.
0 = no ack.

[jendrush]

@hek said:

It's a change in the serial protocol for 1.4.

Could you describe it better keeping in mind my problem with Relay? How the message should look like to turn on Relay? Previously message like this worked ok - 11;1;1;2;1.

[Damme]

@jendrush I suppose it should be 11:1:1:0:2:1 then

@hek what about C_STREAM ? Didnt find much about it.

[hek]

@Damme

It will be used for firmware updates.

[jendrush]

@Damme said:

I suppose it should be 11:1:1:0:2:1 then

I fugured it out yesterday, but thanx!

[Damme]

@hek

I just tried to send myself a picture over the network. real ugly code but it worked.. thought if I put a small cam and someone rings on the doorbell or something.. (Right now I just read the data from SPI flash)

[hek]

@Damme

great

[MisterE]

This post is deleted!

[Damme]

I've tried the lib for a while now and found the following:

Then using ack=1 the returned message is exacly the one I sent. No way of knowing if its a request or an ACK.

example:
Client sends
3;255;3;6;0 (Give me configuration 0 (btw, why not leave config as a byte 0-255 instead of hardcoing it? I could have plenty of uses for configuration-values.)
Gateway responds: 3;255;3;6;M and requests ack
client sends
3;255;3;6;M
my software tried to lookup config id 'M' (Not a big deal for letter, but what if its a number?)
maybe ACK should be some sort of incremental number in return in special message type. I dont really know what is best.
It would be great if GW resends the messsage automatically a couple of times (configurable) if ACK = 1.

I've also noticed during DEBUG enabled that the message gets overwritten by old one, i.e.

send: 3-3-0-0 s=255,c=0,t=18,pt=0,l=15,st=fail:1.4b1 (18848a2)
send: 3-3-0-0 s=255,c=3,t=11,pt=0,l=4,st=ok:test1 (18848a2)
send: 3-3-0-0 s=255,c=3,t=12,pt=0,l=3,st=ok:2.0t1 (18848a2)
(message 2: test1, message 3: 2.0)

Edit; One more thing
Do I really need one gw.process(); before each gw.send(..); in the loop? I seam to loose messages if I dont do like that.

void loop()
{
delay(dht.getMinimumSamplingPeriod());

gw.process();
float temperature = dht.getTemperature();
gw.send(msgTemp.set(temperature, 1));

gw.process();
float humidity = dht.getHumidity();
gw.send(msgHum.set(humidity, 1));

// gw.sleep(SLEEP_TIME); //Seems to break recieing message, loosing ~75%

}

[Yveaux]

@Damme Other wireless protocols usually have a counter which is increased with each message. Returning the counter as an ack would be sufficient to correlate the ack and the original message.
Returning the whole message seems like overkill indeed...

The send-methods could e.g. return the Id counter of the message sent, and an app waiting for an ack on that message should check for this value in any ack received (using some timeout)

[Damme]

Additional comment on config;
I think it should be request config, response config messages.

I also think its a bit odd that the actuator reports as
gw.present(2, S_LIGHT);
but requests as
if (message.type==V_LIGHT) {

V_LIGHT != S_LIGHT (2 vs 3)
I think the def should follow as much as possible.

hope any of my thoughts are somewhat useful

[Yveaux]

@Damme there's a lot of Vera legacy in there... This forum had a discussion going on about decoupling from Vera, but I'm afraid that got lost in the crash...

[Damme]

@Damme said:

// gw.sleep(SLEEP_TIME); //Seems to break recieing message, loosing ~75%

me just dumb here, delay(SLEEP_TIME); works. gw.sleep puts radio in sleep I guess.

[hek]

@Damme said:

me just dumb here, delay(SLEEP_TIME); works. gw.sleep puts radio in sleep I guess.

gw.sleep() puts both Arduino and radio to sleep.

[hek]

@Yveaux said:

The send-methods could e.g. return the Id counter of the message sent, and an app waiting for an ack on that message should check for this value in any ack received (using some timeout)

That mean you might have to rememeber the original message. The idea is to have all the information necessary in the callback.
In my example sketches (e.g. RelayActuator) the ack message actually changes the relay status when the ack comes back from gateway. If not full message came in you would have to keep buffers with sent messages and start matching counter-values. Much more complicated.

But a message counter will probably be necessary if/when we start encrypting messages to prohibit replay attacks.

@Damme
I actually thought about having a bit in the message header which says if the message is an ack or not. With that implemented you would just call a msg.isAck() to determine if the incoming message is an ack message. Would that be ok?

You shouldn't need to call gw.process() after each send. It is only necessary in the loop() section if you expect incoming messages or have enabled repeater mode.

I don't understand what you mean with messages getting overwritten with DEBUG enabled. Could you explain it a bit more (with an example?).

[hek]

@Yveaux

To pick up the discussions about existence or not of getTemp().
A better solution would probably be to create a MySensorsATMega328-subclass of the library which contains AtMega specific stuff such as sleep() and potentially getTemp().
This class should also implement the EEPROM specifics.

[Yveaux]

@hek said:

That mean you might have to rememeber the original message.

You have to remember it anyway when you regard the message as lost after some time without an ack and you decide to send the message again. Only for a simple sensor node which sends a sensor value with ack, waits for an ack reply and then continues with the next sensor value this wouldn't be necessary. When a single node has multiple messages with ack 'in flight', you definately need message buffering.
This is independent from the ack message format IMHO.

I also think message buffering and retrying, when implemented, should be part of the MySensors library.
This improves realiability even more than with the NRF24 ack's enabled and avoids putting the retry-burden on the clients of the library.

[Damme]

@hek said:

I don't understand what you mean with messages getting overwritten with DEBUG enabled. Could you explain it a bit more (with an example?).

here is a serial output from one of my nodes
repeater started, id 3
send: 3-3-0-0 s=255,c=0,t=18,pt=0,l=15,st=fail:1.4b1 (18848a2)
send: 3-3-0-0 s=255,c=3,t=6,pt=1,l=1,st=fail:0
send: 3-3-0-0 s=255,c=3,t=11,pt=0,l=4,st=fail:test1 (18848a2)
send: 3-3-0-0 s=255,c=3,t=12,pt=0,l=3,st=fail:2.0t1 (18848a2)
send: 3-3-0-0 s=0,c=0,t=7,pt=0,l=15,st=fail:1.4b1 (18848a2)
send: 3-3-0-0 s=1,c=0,t=6,pt=0,l=15,st=fail:1.4b1 (18848a2)
send: 3-3-0-0 s=2,c=0,t=3,pt=0,l=15,st=fail:1.4b1 (18848a2)
send: 3-3-255-255 s=255,c=3,t=7,pt=0,l=0,st=fail:1.4b1 (18848a2)

and the sketch has: gw.sendSketchInfo("test", "2.0");

yes, I know they all failed, I'm using the microwave oven atm, it kills the channel 76 (usually uses 21 but forgot to change last time, works better for me here..)

[hek]

@Damme said:

send: 3-3-0-0 s=255,c=3,t=12,pt=0,l=3,st=fail:2.0t1 (18848a2)

Ahh.. The strings-payloads don't seem to be terminated correctly.

[Yveaux]

@hek said:

A better solution would probably be to create a MySensorsATMega328-subclass of the library

I wouldn't restrict to the Atmega328. Quite some AVR's have this or simular behaviour and it would be nice to have a single library usable for all (e.g. using #ifdef's if absolutely necessary).
Seems like you're trying to move towards board- and cpu-support packages, like Linux does...

Another way to tackle it is to create a separate library to read the AVR's internal 'sensors' like the temperature sensor and e.g. the VCC voltage (as I did in https://github.com/Yveaux/arduino_vcc) and another one for persistent storage (EEPROM), and one for power management (sleep). This approach seems to follow the Arduino-way of creating a library for every 'function'.

It all depends on whether you think big and target different hardware (MPU) platforms or if you want to stick with the Arduino (or even only the ATmega328). When porting MySensors to the RPi it would be very nice if the MySensors library and examples would directly compile on both platforms.
This requires libraries to have a clear interface which doesn't limit portability to one platform or architecture.

[Damme]

Hello

I was looking in the nrf24l01 datasheet and found "Setup of Automatic Retransmission"
Is this something that mysensors use?

[Yveaux]

@Damme at least in the 1.4beta release. I think it's disabled in 1.3

[hek]

Yep, as @Yveaux says we use it it 1.4.

[lodewyk]

Hi guys,

I just got relays working with but struggling requesting a variable value on Beta 1.4.

My set variable commands looks as follow.

Switching On - > 1;1;1;0;2;1 (work as expected)
Off -> 1;1;1;0;2;0 (work as expected)

Request variable for the same relay -> 1;1;2;0;2; (sets the relay to 0 ???)

Can someone confirm if my request variable command is correct and if not what should it be?

Thanks

[Damme]

@lodewyk I use this:
void incomingMessage(const MyMessage &msg) {
// We only expect one type of message from controller. But we better check anyway.
if (msg.type==V_LIGHT) {
if (strlen(msg.getString())==0) {
gw.send(message.setSensor(msg.sensor).setType(V_LIGHT).set(digitalRead(msg.sensor-1+RELAY_1)?RELAY_ON:RELAY_OFF));
} else {
digitalWrite(msg.sensor-1+RELAY_1, msg.getBool()?RELAY_ON:RELAY_OFF);
gw.saveState(msg.sensor, msg.getBool());
}
}
}

(How do i use [code] ?? I start to hate this forum :P)

[hek]

@lodewyk

The relay example does not support requesting state. You must add some code in incomingMessage() to handle incoming request-messages (and reply to them).

To reply incoming request-command in incomingMessage for the RelayActuator-example do something like this (note I have not compiled/tested this)

if (mGetCommand(msg) == C_REQ) {
     mSetCommand(msg, C_SET);
     msg.setDestination(msg.getSender());
     msg.set(gw.loadState(msg.getSensor());
     gw.send(msg); 
} else (
    // Do the normal stuff here
}

[lodewyk]

@DAMME and @HEK

Thanks , very helpful.

I got so occupied with the commands and never thought about the sensor code.

Thanks

[lodewyk]

This post is deleted!

[lodewyk]

This post is deleted!

[hek]

@Damme said:

(How do i use [code] ?? I start to hate this forum :P)

This forum uses markdown. If you need help, press the little questionmark-icon in the compose window. http://daringfireball.net/projects/markdown/syntax

To decorate your codeblock use 4 spaces or one tab character first on each line.

[Yveaux]

@lodewyk hey man, you've got -2 posts.... How's that possible?

[hek]

Pushed a few new changes/bugfixes. Sorry for the slow progress. Had to wait until we got home and had the chance to do some verification.

• Ack bit in header indicating if message is an ack (see RelayWithButton for an example how to read it out).
• Sleep functions now takes unsigned long. This allows sensor to sleep for than 30 sec
• Corrected string termination

https://github.com/mysensors/Arduino/commits/development

[hek]

A major code drop coming from @ToSa has now been merged into 1.4. It contains:

• The new MySensors Bootloader supporting Over-the-air sketch updates ("client"-side).
• A simple NodeJs Controller to test the OTA stuff ("server"-side).
• Changes to cope with new sketch meta data stored in EEPROM and conversion between serial protocol/binary data and some new STREAM command types added.
• A new internal command has also been added to allow resetting a node remotely.

A more detailed change log can be found here:
https://github.com/ToSa27/Arduino/commits/development

I haven't had time to test myself yet but everything has been verified by @ToSa and is reported working. He is on a business trip right now so advanced bootloader questions have to wait until he's back.

[marceltrapman]

@hek Sounds like a good step forward!
Only one boot loader question: does this mean we can use an external boot loader or do we have to use an external boot loader now?

[hek]

@marceltrapman

The OTA stuff is optional. But if you want to be able to hot deploy sketches over the air, you have to install the MySensors bootloader.

[Damme]

@hek There are still some problem with string termination,

repeater started, id 3
send: 3-3-0-0 s=255,c=0,t=18,pt=0,l=15,st=fail:1.4b1 (18848a2)
send: 3-3-0-0 s=255,c=3,t=6,pt=1,l=1,st=fail:0
send: 3-3-0-0 s=255,c=3,t=11,pt=0,l=9,st=fail:Relaytest848a2)
send: 3-3-0-0 s=255,c=3,t=12,pt=0,l=3,st=fail:1.0aytest848a2)
read: 5-5-0 s=11,c=1,t=1,pt=0,l=4:58.4
send: 5-3-0-0 s=11,c=1,t=1,pt=0,l=4,st=fail:58.4
send: 3-3-0-0 s=10,c=1,t=0,pt=0,l=5,st=fail:26.20

[Damme]

@Damme My bad wrong working directory! sorry

[marceltrapman]

@hek said:

• Sleep functions now takes unsigned long. This allows sensor to sleep for than 30 sec

Thank you

[marceltrapman]

@hek said:

• Corrected string termination

And thanks again!!!

[timminater]

How do I request information from a node from the gateway?

I have a relayactuator node with 2 relays that is transmitting its initial state at the startup of the node. This is also received by the gw.
But after that I cant seem to request the current relay status from the node by sending a message from the gateway.

I tried lots of combinations, nothing worked.

I thought this would be correct:
1;2;2;1;2;0\n
1=node number
2=sensor (relay number 2)
2=request
1=ack message
2=light
0=string??

Can anyone point me in the correct location?
maybe a new API page for 1.4 would be handy

edit: also I can't find anywhere what the debug letters mean
what is s=,c=,t=,pt=,l=,st= ??

[hek]

@timminater

You cannot request state from a node without programming a little . I answered exactly this question a couple of posts back in this thread.

http://forum.mysensors.org/topic/168/1-4-beta/34

[mitekg]

@hek
Could u plz help me
getting in eth GW. Vera and Libs updated.

**0;0;3;9;Arduino startup complete.
0;0;3;9;read: 0-0-0 s=0,c=0,t=0,pt=0,l=0:
0;0;3;9;version mismatch0;0;3;9;read: 0-0-0 s=0,c=0,t=0,pt=0,l=0:
0;0;3;9;version mismatch0;0;3;9;read: 0-0-0 s=0,c=0,t=0,pt=0,l=0:
0;0;3;9;version mismatch0;0;3;9;read: 0-0-0 s=0,c=0,t=0,pt=0,l=0:
**

[hek]

@mitekg

All the zeroes indicates that something is wrong in radio communication. Check wires.

[mitekg]

• The new MySensors Bootloader supporting Over-the-air sketch updates ("client"-side).
  • A simple NodeJs Controller to test the OTA stuff ("server"-side).
    Is any instructions, how to use this?
    thx.

[Damme]

@hek
I think BASE_RADIO_ID should be declared in MyCofig.h

[Zeph]

@hek @ToSa

Could we get a description of how the OTA programming works?

(Terminology: the program flash is divided into a bootloader section and an application section; compile sketches go in the application section)

One approach is to have a separate flash or eeprom chip which one somehow fills (this could even happen at the application level). When booting, check this flash or eeprom and if valid - load into internal program flash, clear the vallidity of the external flash or eeprom (so you won't loop loading it again) and reboot. When you reboot there will be no valid image to copy from external memory, so you jump to the start of internal application flash. This takes external flash as large or larger than application flash (eg: >= 32 KiB for the 328p)

I think you are taking a different route, where the bootloader is somehow given control, after which it receives nRF packets to RAM, periodically writing another page from RAM to the application section of internal flash. When done, reboot.

If booting and there is no valid image in external memory and the CRC of the internal application flash section is valid, jump to it as usual.

I'm not sure if you can also still use serial programming.

This could be all wrong. It implies keeping (at least part of) the nRF library in bootload program flash, to receive packets. I'm putting it out as a hypotheis to help elicit the real description.

How big is the OTA bootloader?

[hek]

@Damme said:

I think BASE_RADIO_ID should be declared in MyCofig.h

Yes, that's a better place for it.

[hek]

@Zeph, @mitekg

Right now the OTA works without any need for external flash. @ToSa will have to answer the details when he's back (like size, building, serial programming).

Yes it uses a VERY stripped down rf24 library.
https://github.com/mysensors/Arduino/blob/development/Bootloader/MyOtaBootloaderRF24.h

Not much of the MySensors library is needed.
https://github.com/mysensors/Arduino/blob/development/Bootloader/MyOtaBootloader.c

[joe.k]

I am in process of upgrading to 1.4b1 - have most things working. I noticed that my status consistently fails during setup() when presenting sensors AND also never receive a return message from controller after a time request. It's always the same two failures.

Everything works after adding a 100ms delay between calls. Not sure if there's a timing problem somewhere.

Hek - as always thanks for ALL your contributions.

Joe K.

SERIAL CONSOLE OUTPUT:
sensor started, id 4
send: 4-4-0-0 s=255,c=0,t=17,pt=0,l=15,st=ok:1.4b1 (18848a2)
send: 4-4-0-0 s=255,c=3,t=6,pt=1,l=1,st=ok:0
send: 4-4-0-0 s=255,c=3,t=11,pt=0,l=11,st=ok:LCD Display
send: 4-4-0-0 s=255,c=3,t=12,pt=0,l=3,st=ok:1.0
send: 4-4-0-0 s=0,c=0,t=7,pt=0,l=15,st=ok:1.4b1 (18848a2)
send: 4-4-0-0 s=1,c=0,t=6,pt=0,l=15,st=fail:1.4b1 (18848a2)
send: 4-4-0-0 s=2,c=0,t=8,pt=0,l=15,st=ok:1.4b1 (18848a2)
send: 4-4-0-0 s=255,c=3,t=1,pt=0,l=15,st=fail:1.4b1 (18848a2)

CODE in setup():
// Send the Sketch Version Information to the Gateway
gw.sendSketchInfo("LCD Display", "1.0");

// Register all sensors to gw (they will be created as child devices)
gw.present(CHILD_ID_DHT_HUM, S_HUM, true);
delay(100); // needed otherwise status may come back as failed

gw.present(CHILD_ID_DHT_TEMP, S_TEMP, true);
delay(100); // needed otherwise status may come back as failed

gw.present(CHILD_ID_BMP085_BARO, S_BARO, true);
delay(100); // needed otherwise status may come back as failed & time request doesn't work

// request time
gw.requestTime(receiveTime);

[Damme]

@hek Does the gateway use any eeprom?

[hek]

@Damme

Yes, it uses eeprom for storing routing information.

[Damme]

@hek ok, how many bytes does it use?

[hek]

@Damme

If you want to store you own data in EEPROM you can use the provided saveState(), loadState(). It makes sure you won't overwrite anything important. If 256 bytes isn't enough you can start writing at position EEPROM_LOCAL_CONFIG_ADDRESS defined in MySensors.h.

[Damme]

@hek I just cant figure this out:
0;0;3;9;read: 255-255-0 s=255,c=3,t=3,pt=0,l=0:
0;0;3;9;send: 0-0-255-255 s=255,c=3,t=4,pt=0,l=2,st=fail:42
the last one isn't sent.What have I missed?
I have also tried 42 as int (no "")

	msg.destination = 255; 		//NodeID
	msg.sender = GATEWAY_ADDRESS;
	msg.sensor = 255;	  				//SensorID
	msg.type = I_ID_RESPONSE;			//MsgType
	mSetCommand(msg,C_INTERNAL); 		//Subtype
	mSetRequestAck(msg,true);			//Request ack
	mSetAck(msg,false);
	msg.set("42");					//Payload
	sendRoute(msg);

[marceltrapman]

@hek Do you know if the ota is something initiated by the controller or do we update the gateway with a new version and will the gateway be responsible?

[hek]

@marceltrapman

The controller will be responsible for initiating and serving the new sketch version. There is basically no changes in gateway except being able to handle binary data.

[hek]

@Damme

mSetRequestAck(msg,true);

You should probably not request ack back on a id-response. But I doubt that is the cause for transmission error. What is happening in the sensor node?

[Damme]

@hek Tried with no ack also. And the node doesn't recieve anything. I'm out of components to build myself a sniffer right now.. Might have to kill something.. I can recieve From the node though.. but the node does not read from gateway..

[hek]

@Damme

You only live 190 km from me. Come and get some

What kind of antenna on your gateway? Have you tried lowering transmit power of gateway?

[Damme]

@hek I fixed the problem , it was the RF24_PA_LEVEL_GW .. needed to be low.

You are correct with the ack on id-response, it gets all fucked up. Stuck in loop requesting ID 10+times/second. And doesnt register the id.. Too bad. would be really good for me to know if the sensor actually got the id_response or not... (and if it's sent as node 255 or node newid doesnt really matter..)

edit:
I changed my aproach to checking if sendRoute(msg) is true. But I still have a problem that the node actually recieves the packagt but just ignored is. sometimes I need to reset the node for it to get accept the new ID.. example;

0;0;3;9;read: 255-255-0 s=255,c=3,t=3,pt=0,l=0:
0;0;3;9;send: 0-0-255-255 s=255,c=3,t=4,pt=1,l=1,st=ok:20
0;0;3;9;read: 255-255-0 s=255,c=3,t=3,pt=0,l=0:
0;0;3;9;send: 0-0-255-255 s=255,c=3,t=4,pt=1,l=1,st=ok:21
0;0;3;9;read: 255-255-0 s=255,c=3,t=3,pt=0,l=0:
0;0;3;9;send: 0-0-255-255 s=255,c=3,t=4,pt=1,l=1,st=ok:22
0;0;3;9;read: 255-255-0 s=255,c=3,t=3,pt=0,l=0:
0;0;3;9;send: 0-0-255-255 s=255,c=3,t=4,pt=1,l=1,st=ok:23

I'll try to look at the debug output on the node tomorrow..

btw: \n missing line 278 mysensor.cpp debug(PSTR("version mismatch"));

[marceltrapman]

@hek said:

The controller will be responsible for initiating and serving the new sketch version. There is basically no changes in gateway except being able to handle binary data.

How do I get this started Henrik?

[hek]

@marceltrapman

Until @ToSa is back, I suggest you have a look at how the NodeJSController serves hex-files.
https://github.com/mysensors/Arduino/blob/development/NodeJsController/NodeJsController.js

[Yveaux]

@hek I wanted to update my MQTT gateway to 1.4beta state but run into a problem with the serial message format.
If I interpret the code right, the messages sent (in serial message format) to the ethernet gateway (over ethernet, so from my linux system running the MQTT gateway) contain 6 fields, of which the 4th is the ack flag.
Messages received from the ethernet gateway contain 5 fields, with no ack indication.
Shouldn't the message format be identical in both directions, as it is for the 1.3 version?

[hek]

@Yveaux

Yes, It might be interesting (for the controller side) to use the 4th field to flag if the received message is an ack or regular message.

...so for outgoing you enable receiver to send ack back. And incoming the flag indicates if this as an ack or regular message.

Make sense?

[Yveaux]

@hek said:

Make sense?

Makes sense

[John]

Short question. I'm browsing the 1.4 code and is there a replacement for waitForMessage? I've looked into the code online but can't seem to find it. And if not, is it possible to attach the radio to an interrupt to call an interrupt routine on incoming message?

[Yveaux]

@John said:

is it possible to attach the radio to an interrupt to call an interrupt routine on incoming message

The last question is a definite yes, see for example https://github.com/Yveaux/NRF24_Sniffer/blob/master/Arduino/NRF24_sniff/NRF24_sniff.ino function handleNrfIrq. Connect the NRF24 IRQ to pin 2 and call attachInterrupt to register the interrup handler.

Be very careful what you do in this interrupt handler, though. Don't try to receive any message for example, as the SPI access to the NRF24 is not interrupt-proof, as is the MySensors library!

[hek]

@John

Yep, as @Yveaux says, be careful what you do inside interrupt handler.

Not sure what you try to archive here.. but in 1.4 you should usually just call process() in your loop(). The callback you registered a in begin(callback) will be called on incoming messages.

See RelayActuator for an example.

[John]

@Yveaux

Be very careful what you do in this interrupt handler, though. Don't try to receive any message for example, as the SPI access to the NRF24 is not interrupt-proof, as is the MySensors library!

Darn, i hoped it could.

@hek

Not sure what you try to archive here.. but in 1.4 you should usually just call process() in your loop(). The callback you registered a in begin(callback) will be called on incoming messages.

I'm averaging sensor results, so i read sensors every second an send them every minute, also this setup has an 16x2 display attached to show the values. But i would like it to be bi-directional with the server so it can display the server send data. Thus if i would use waitForMessage i can let it wait about a second for a message read sensor values, and let it wait for a second again. (because i understood it blocks?)

[Yveaux]

@John You can obtain the same result as the old waitForMessage by registering a callback. Have this callback set a flag and call process() in the loop, as hek suggests.
You could also turn things around and read and average your sensors from a (timer)interrupt handler.
But when you're not completely confident with using interrupts then its better to avoid them and just process everything from your loop(). Interrupts can introduce lots of nasty problems if you don't know exactly what you're doing!

[John]

@Yveaux

You can obtain the same result as the old waitForMessage by registering a callback. Have this callback set a flag and call process() in the loop, as hek suggests.

Well, that was my first try and hoped it was interrupt based because of one wire connected on arduino pin 2 to IRQ on the NRF. The reason i asked it because i saw an example from an other library: http://maniacbug.github.io/RF24/pingpair_irq_8pde-example.html .

You could also turn things around and read and average your sensors from a (timer)interrupt handler.

I will have to investigate if any of the components is using which timer and what will be influenced by it. I'm using an HD44780 alike as display so i'm not sure because never dived deep into the used library.

But when you're not completely confident

about 90%?

@hek
It would be nice if a waitForMessage like functionality would be included in 1.4.

[clippermiami]

@hek Any idea when 1.4 will become the production version?

[hek]

@clippermiami

We're still making small changes in the protocol and serial API which we need to stabilize before release.
I also need time to update the main site to reflect the new version before its "released" .

[hek]

Added few small changes:

• Add dimmable LED actuator example (converted @blacey 1.3 example)
• Rename PING to the more suitable FIND_PARENT
• Float values sent in binary format over the air
• Move base-radio-id to MyConfig.h

https://github.com/mysensors/Arduino/commits/development

[Damme]

@hek another suggestion for addition

RelayWithButtonActuator
incomingMessage should check if there is any payload or not. (could be bad package)
I use empty payload as a request for latest state (Yes I could do this some other way but I think it is logical..)

  if (msg.type==V_LIGHT && strlen(msg.getString())!=0) {

[John]

Because of the use of custom variables and the possibility of receiving and sending and of many sensor type possibilities is it possible to add a "sensor" type of S_OTHER (or alike)? I think this then would be in line of having var types V_VAR1, etc.. of which then would be something else then defined.

This will add the possibility to send other then sensor related data. This is of course possible but could semantic wise be more applicable.

[hek]

@John

Yes

[hek]

A small serial protocol breaking change was just checked in. @John and @marceltrapman you might want to have a look at this.

Serial protocol changed to allowing controller to know if an incoming message is an ack. Outgoing and incoming serial messages now also have the same parameter count.

radioId;childId;messageType;ack;subType;payload\n

Where ack parameter means the following:
outgoing: 0 = unacknowledged message, 1 = request ack from destination node
Incoming: 0 = normal message, 1 = this is an ack message

(previously only outging messages had the ack-flag)

[John]

@hek

Thanks for the heads up. This will give a good way of tracking messages send controller wise. I can take a look at it tomorrow evening if you would like a third party ack this works ok.

[Yveaux]

@hek Thanks for the quick response!

[Damme]

@Yveaux take a look at (was thinking about your sniffer) https://github.com/mysensors/Arduino/commit/c910bfdb9e54a9a41e991734b60c757968bd8210

[John]

@hek I have a couple of questions:

Have not tested it yet, but that now ack's are possible i would like to implement a retry queue next to a normal send queue. The questions then would be:

• How long does it take for the longest possible message to be send?
• Is there a delay needed between each send?
• Is there an ack timeout?

John.

[Yveaux]

@Damme Yeah, so header changed again... Madness
I'll have to update the dissector then...

[hek]

@John

The send time depends on how many hops the message need to make. But on the controller side you could perhaps wait for a couple of second before trying again.

There is no ack timeout but each hop on the way to the destination will only be tried once (using the build in retry functionality of the nrf-chip).

[lininger]

@hek Any tips on upgrading the June 1.4b1 to the August 1.4b1. I just attempted it with disastrous results.

I started off.

1. Uploading the Vera files
2. Re-compiled the new gateway (after changing the my IP Address and such)
3. Re-complied the Repeaters

Just to get going. Both repeaters failed at startup with the standard FAIL in the debug logs. Wiping the Eeprom and trying again did not work. Dropping the repeaters for vera and re-including did not work? Finally went back to the June build and restored the vera from last night and then two other units failed to come back online. Had to replace the antennas on those two units to get them back on-line – BIZARRE.

1. Do all the sensors have to be updated at once?
2. Will existing sensors under the June 1.4b1 build co-exist with the August 1.4b1 Gateway build?
3. Can we just recompile our existing June sketches under the new mySensors library as long as no compile errors show?

Thanks

[hek]

@lininger

Yes, you'll have to recompile/update them all as message format has changed. No need for wiping eeprom or re-include sensors. Also update Vera plugin.

Replacing antenna should not be necessary and I don't understand why that would change anything..

[Yveaux]

@hek I had a look at the latest state of the 1.4 header format and saw the sender/last/destination have moved to the front position.
This means the protocol version is, once again, located at a different position w.r.t. 1.3 and earlier 1.4 version.
For software trying to interpret any MySensors message (e.g. my sniffer or a (future) MySensors version supporting different protocol versions) or software that checks the library version to match with its own version it becomes nearly impossible to detect which library version a message was sent with.
I propose to start any message, now and in the future, with a protocol version (use a full byte or use some remaining bits for protocol independent data) and stick to that. This makes life for sugested applications a lot easier.
Adding a version which has a different offset every time makes no sense...

[hek]

@Yveaux

Sorry.. the sniffer slipped my mind. The purpose was to prepare for future encryption (keeping the changing "last"-field first) and potential split of the routing information and payload. Which would be necessary for supporting other transportation layers (like RadioHead). They might not be interested in any MySensors version-specifics first in the radio message.

Do you have any suggestion?

[Yveaux]

@hek said:

Do you have any suggestion?

It's in my post hek

[Yveaux]

@hek I have another question: when sending from a sensor to a gateway the data can have all kinds of formats (text, byte, float etc) which comes out the gateway through the serial protocol. When sending the other way, I guess the serial data is always sent as text by the gateway to the sensor, right?
Too bad we can't also use different formats when sending through the serial api... Did you think about this?

[hek]

@Yveaux said:

It's in my post hek

Yes, you suggest to always have but that might not be possible if we choose some other transportation layer (which most probably will use the first bytes for it's own routing).

When sending the other way, I guess the serial data is always sent as text by the gateway to the sensor, right?
Too bad we can't also use different formats when sending through the serial api... Did you think about this?

Yes, I've thought about it. We have a couple of options here. Start type:ing variables through the serial protocol (controller must send datatype) or have a hard coded variable->type map on arduino side. Both of these are valid options.
A binary serial protocol is also an option (could be an option when building gateway).. But that makes it harder to debug e.g. Arduino Serial Monitor.

[Yveaux]

@hek said:

but that might not be possible if we choose some other transportation layer

Then the transportation layer data is not part of mysensors protocol. It's that simple!
IMHO This again boils down to nested protocol headers which we talked about before.
The routing info is part of transportation layer (it could even have its own version info) followed by the mysensors header ( starting with a version number) which tells us what's in the message. Then come the actual data, be it a value, presentation info or whatever. I can make a sketch of this structure if that helps in the discussion.

[hek]

@Yveaux

I think we agree. Only problem is that I haven't split up the structure in mysensors-transportation/mysensors-header/mysensors-payload yet but if you look from the sniffer side this would not matter.

Please post you suggested structure (but in a new thread).