Posts made by cvdenzen

@Nigel31 Thank you, I will try to write some code and test it. It can take a few weeks as I have to do it in my spare time (like most of us).

I have been struggling with the MySensors sleep method. My PIR sensor is not using one of the two supported interrupts.
I have started writing some code, but that is just a beginning. Is anybody interested in this subject or should I just write it for my own use?
Any help would be welcome, it is not unlikely that I am missing something important in this mechanism.
The actual (partial) code can be found at
https://sourceforge.net/p/easypirmultisensorsbox/code/ci/cvdenzen/tree/easyPIRmultisensorsBox2_155/easyPIRmultisensorsBox2_155/

I named the new class Sleep3.

Advantages of current (december 2021) sleep implementation:

• it works
• it is simple to use
• there is a good description how to implement it
  Problems with current (december 2021) sleep implementation:
• it only handles two interrupt sources
• it captures these two interrupts, but these interrupts don't belong to the sleep mechanism
• it only works for ATMega
• it is not scalable to many interrupt sources
• the return value is not safe: it is possible to miss interrupts if two interrupts happen simultaneously

Trying to make a better sleep (class Sleep3) that improves some of the shortcomings:

• only an interrupt service routine for the watchdog timer, the other interrupts are not of our (Sleep3) concern
• Implement the Observer pattern
• Use the Observer pattern to register methods that can veto a sleep call (see note 1)
• turn off adc only once (not every iteration of sub-sleep)
• turn off network only once (not every iteration of sub-sleep)
• Two public methods: "sleep()" (forever), and "sleep(unsigned long sleep_time)". They return a struct
  with a few members: int errno (e.g. re-entry not allowed), bool wokeUpByWDT (if false, some other interrupt triggered wake-up),
  unsigned long requestedSleepMS, unsigned long sleepRemainingMS.
  requestedSleepMS-sleepRemainingMS can be used to adjust the millis() counter ?? it will not be accurate (can be 8 seconds off in avr).

Note 1:
If there is a non-processed interrupt, the cpu must not go to sleep. That happens e.g. in this scenario: the cpu is executing the loop() code and is almost at the point to call sleep(). A PIR sensor triggers an interrupt, the ISR is run, it sets a flag that is to be checked in the loop(). At that point the cpu would go to sleep. The flag would only be seen after the sleep call!
Solution: check whether sleeping is allowed, same scenario extended with the check:
the cpu is executing the loop() code and is almost at the point to call sleep(). A PIR sensor triggers an interrupt, the
ISR is run in which a flag is set to be checked in the loop(). New: at that point the cpu disables interrupts, calls the
callback methods of the observers and will be signaled by the PIR observer that it should not go to sleep(), but continue its processing. The loop is re-entered immediately and the PIR data can be sent to the gateway.

The Observer method will be called in a locked (interrupts disabled) state just before the hardware sleep is entered.
If the Observer notices that there is a pending interrupt on behalf of its subject, it should return false, otherwise
it should return true (as in: allright, go to sleep).

// The Sleep3 code that calls the observers will look like this:
boolean sleepIsAllowed=true;
noInterrupts();
for (Observer observer:observers) { if (!observer()) {sleepIsAllowed=false;break}}
if (sleepIsAllowed) {interrups();sleep;} else interrupts();

// An example application looks like this:
//
Sleep3 sleep3; // Sleep3 is the proposed sleep class
before() {
  sleep3.attach(sleepObserver);
  .. enable interrupts etc.
}
ISR (PCINT2_vect) {
  boolean PIRPinValue = digitalRead(PIR_PIN);
  queue.add(PIRPinValue);
}
// This method is entered with interrupts disabled
boolean sleepObserver() {
  if (!queue.isEmpty()) return false; else return true;
}
void loop() {
  noInterrups(); // to lock the queue
  if (!queue.isEmpty()) {
    boolean pir=queue.remove();
    interrupts();
    // do something with pir
  } else {
    interrupts();
  }
  sleep3.sleep(SLEEP_TIME);
}

@mfalkvidd Thank you!

It seems like building MySensors with my edit failed but I don’t understand the error messages.
The first error: https://ci.mysensors.org/job/MySensors/job/MySensors/job/PR-1497/2/warnings2Result/new/file.702256833/source.9006459693321295180/#1

@mfalkvidd Thank you for the link. I followed the instructions, had to download/compile some packages.
It will take some time for me, I have to solve this error:

 git commit -m "Accept wildcards in mqtt subscription prefix"
cppcheck: Failed to load library configuration file 'std.cfg'. File not found
cppcheck: Failed to load library configuration file 'avr'. File not found
nofile:0:0: information: Failed to load the library avr [failedToLoadCfg]

Few minutes later, resolved, had to add CFGDIR to make cppcheck:

sudo make install FILESDIR=/usr/share/cppcheck CFGDIR=/usr/share/cppcheck

I have made some modifications to the code, tested it and it seems to work.
There is a pull request https://github.com/mysensors/MySensors/pull/1497 but I don't know how this will be handled by the MySensors developers.

Can anybody give me a link or some information about the development process? A review of my code will be necessary, I am not a C programmer.

The problem is clear: the subscription topic that I use contains a wildcard: “+/mysensors”.
I send my message with prefix “carl/mysensors”.
The MySensors code assumes the length of the prefix I send is the same as the length of the subscription prefix. But it isn’t because of the wildcard character.
Is there anybody who likes to change the code to make it work? Or should I try it myself?
I see two possible solutions:

1. Parse the message from end to begin and not from begin to end.
2. Try to match the wildcard character(s) with the sent string.

@cvdenzen
I have added a GATEWAY_DEBUG line in core/MyProtocol.cpp at line 122:

	for (str = strtok_r(topic + strlen(MY_MQTT_SUBSCRIBE_TOPIC_PREFIX) + 1, "/", &p);
	        str && index < 5;
	        str = strtok_r(NULL, "/", &p), index++
	    ) {
	    GATEWAY_DEBUG(PSTR("GWT:TPS:Index=%d, mqttstr=%s\n"), str);

My conclusion is that the length of the mqtt subscription topic is the problem. The first call shows that it thinks "rs" is part of the topic beyond the PREFIX. And "rs" gets translated to "0" in the outgoing mqtt message, shifting the real items one position further and dropping the last item.
Later on I will have a look at the cause of this limitation (or maybe someone else is quicker than I am).
The output of mysgw, first try is with topic carl/mysensors, second try (that succeeds) is with mqtt topic c/mysensors:

Aug 15 23:53:19 DEBUG GWT:TPS:TOPIC=mysensors/all/157/255/3/0/21,MSG SENT
Aug 15 23:53:24 DEBUG GWT:IMQ:TOPIC=carl/mysensors/157/0/1/0/3, MSG RECEIVED
Aug 15 23:53:24 DEBUG GWT:TPS:Index=0, mqttstr=rs
Aug 15 23:53:24 DEBUG GWT:TPS:Index=1, mqttstr=157
Aug 15 23:53:24 DEBUG GWT:TPS:Index=2, mqttstr=0
Aug 15 23:53:24 DEBUG GWT:TPS:Index=3, mqttstr=1
Aug 15 23:53:24 DEBUG GWT:TPS:Index=4, mqttstr=0
Aug 15 23:53:24 DEBUG GWT:TPS:TOPIC=mysensors/all/0/157/0/1/0,MSG SENT
Aug 15 23:54:42 DEBUG GWT:IMQ:TOPIC=c/mysensors/157/0/1/0/3, MSG RECEIVED
Aug 15 23:54:42 DEBUG GWT:TPS:Index=0, mqttstr=157
Aug 15 23:54:42 DEBUG GWT:TPS:Index=1, mqttstr=0
Aug 15 23:54:42 DEBUG GWT:TPS:Index=2, mqttstr=1
Aug 15 23:54:42 DEBUG GWT:TPS:Index=3, mqttstr=0
Aug 15 23:54:42 DEBUG GWT:TPS:Index=4, mqttstr=3
Aug 15 23:54:42 DEBUG TSF:MSG:SEND,0-0-157-157,s=0,c=1,t=3,pt=0,l=2,sg=0,ft=0,st=OK:35
Aug 15 23:54:42 DEBUG TSF:MSG:READ,157-157-0,s=0,c=1,t=2,pt=1,l=1,sg=0:1
Aug 15 23:54:42 DEBUG GWT:TPS:TOPIC=mysensors/all/157/0/1/0/2,MSG SENT

@evb It took some time (have also another home renovation project), but now I have set up a test gateway.
The mqtt topic with a slash in its name seems to be the problem. I changed the mqtt topic to mysensorsa and now the sketch receives the message. The mqtt gateway outputs:

Aug 15 22:50:33 DEBUG GWT:IMQ:TOPIC=mysensorsa/157/0/1/0/3, MSG RECEIVED
Aug 15 22:50:33 DEBUG TSF:MSG:SEND,0-0-157-157,s=0,c=1,t=3,pt=0,l=2,sg=0,ft=0,st=OK:35
Aug 15 22:50:34 DEBUG TSF:MSG:READ,157-157-0,s=0,c=1,t=2,pt=1,l=1,sg=0:1
Aug 15 22:50:34 DEBUG GWT:TPS:TOPIC=mysensors/all/157/0/1/0/2,MSG SENT
Aug 15 22:50:34 DEBUG TSF:MSG:READ,157-157-0,s=0,c=1,t=3,pt=2,l=2,sg=0:35
Aug 15 22:50:34 DEBUG GWT:TPS:TOPIC=mysensors/all/157/0/1/0/3,MSG SENT

I will have a look at the MySensors code and if I cannot find the problem, maybe I will change my mqtt subscription topic.

Is the mysgw output looking good? The sent message doesn”t include the message type (3). Has anybody tested this with a topic that contains a slash?

No, the bare sketch (with only the necessary modifications, like NEW_DRIVER, NODE_ID etc.) does not receive the message.
I tried the bare sketch on two different pieces of hardware (an arduino mini pro and an easypirmultisensors2, which also is an 8 MHz avr).
The sketch:

/*
 * The MySensors Arduino library handles the wireless radio link and protocol
 * between your home built sensors/actuators and HA controller of choice.
 * The sensors forms a self healing radio network with optional repeaters. Each
 * repeater and gateway builds a routing tables in EEPROM which keeps track of the
 * network topology allowing messages to be routed to nodes.
 *
 * Created by Henrik Ekblad <henrik.ekblad@mysensors.org>
 * Copyright (C) 2013-2019 Sensnology AB
 * Full contributor list: https://github.com/mysensors/MySensors/graphs/contributors
 *
 * Documentation: http://www.mysensors.org
 * Support Forum: http://forum.mysensors.org
 *
 * This program is free software; you can redistribute it and/or
 * modify it under the terms of the GNU General Public License
 * version 2 as published by the Free Software Foundation.
 *
 *******************************
 *
 * REVISION HISTORY
 * Version 1.0 - February 15, 2014 - Bruce Lacey
 * Version 1.1 - August 13, 2014 - Converted to 1.4 (hek)
 *
 * DESCRIPTION
 * This sketch provides a Dimmable LED Light using PWM and based Henrik Ekblad
 * <henrik.ekblad@gmail.com> Vera Arduino Sensor project.
 * Developed by Bruce Lacey, inspired by Hek's MySensor's example sketches.
 *
 * The circuit uses a MOSFET for Pulse-Wave-Modulation to dim the attached LED or LED strip.
 * The MOSFET Gate pin is connected to Arduino pin 3 (LED_PIN), the MOSFET Drain pin is connected
 * to the LED negative terminal and the MOSFET Source pin is connected to ground.
 *
 * This sketch is extensible to support more than one MOSFET/PWM dimmer per circuit.
 * http://www.mysensors.org/build/dimmer
 */

// Enable debug prints to serial monitor
#define MY_DEBUG
#define MY_BAUD_RATE 38400

// Enable and select radio type attached
//#define MY_RADIO_RF24
//#define MY_RADIO_NRF5_ESB
#define MY_RADIO_RFM69
//#define MY_RADIO_RFM95
// If using mysensors version<2.3.0 and want to use NEW DRIVER (OH SO BAD NAMING):
#define MY_RFM69_NEW_DRIVER
#define MY_RFM69_FREQUENCY   RFM69_868MHZ
#define MY_RFM69_NETWORKID 197// Comment it out for Auto Node ID #, original was 0x90=144
#define MY_NODE_ID 157
#include <MySensors.h>

#define SN "DimmableLED"
#define SV "1.1"

#define LED_PIN 3      // Arduino pin attached to MOSFET Gate pin
#define FADE_DELAY 10  // Delay in ms for each percentage fade up/down (10ms = 1s full-range dim)

static int16_t currentLevel = 0;  // Current dim level...
MyMessage dimmerMsg(0, V_DIMMER);
MyMessage lightMsg(0, V_LIGHT);


/***
 * Dimmable LED initialization method
 */
void setup()
{
	// Pull the gateway's current dim level - restore light level upon node power-up
	request( 0, V_DIMMER );
}

void presentation()
{
	// Register the LED Dimmable Light with the gateway
	present( 0, S_DIMMER );

	sendSketchInfo(SN, SV);
}

/***
 *  Dimmable LED main processing loop
 */
void loop()
{
}



void receive(const MyMessage &message)
{
   Serial.print( "Received a message! (node 157)" );
	if (message.getType() == V_LIGHT || message.getType() == V_DIMMER) {

		//  Retrieve the power or dim level from the incoming request message
		int requestedLevel = atoi( message.data );

		// Adjust incoming level if this is a V_LIGHT variable update [0 == off, 1 == on]
		requestedLevel *= ( message.getType() == V_LIGHT ? 100 : 1 );

		// Clip incoming level to valid range of 0 to 100
		requestedLevel = requestedLevel > 100 ? 100 : requestedLevel;
		requestedLevel = requestedLevel < 0   ? 0   : requestedLevel;

		Serial.print( "Changing level to " );
		Serial.print( requestedLevel );
		Serial.print( ", from " );
		Serial.println( currentLevel );

		fadeToLevel( requestedLevel );

		// Inform the gateway of the current DimmableLED's SwitchPower1 and LoadLevelStatus value...
		send(lightMsg.set(currentLevel > 0));

		// hek comment: Is this really nessesary?
		send( dimmerMsg.set(currentLevel) );


	}
}

/***
 *  This method provides a graceful fade up/down effect
 */
void fadeToLevel( int toLevel )
{

	int delta = ( toLevel - currentLevel ) < 0 ? -1 : 1;

	while ( currentLevel != toLevel ) {
		currentLevel += delta;
		analogWrite( LED_PIN, (int)(currentLevel / 100. * 255) );
		delay( FADE_DELAY );
	}
}

The mqtt invocation:

mqtt pub --host rpi3.home --mqttVersion 3 --topic 'carl/mysensors/157/0/1/0/3' --message 35

The mysgw logging:

Aug  9 22:20:04 rpi3 mysgw: GWT:IMQ:TOPIC=carl/mysensors/157/0/1/0/3, MSG RECEIVED
Aug  9 22:20:04 rpi3 mysgw: GWT:TPS:TOPIC=mysensors/all/0/157/0/1/0,MSG SENT

The logging from the easypirmultisensorsbox2 (nothing shows up when an MQTT message is sent to the MQTT broker):

 
 __  __       ____
|  \/  |_   _/ ___|  ___ _ __  ___  ___  _ __ ___
| |\/| | | | \___ \ / _ \ `_ \/ __|/ _ \| `__/ __|
| |  | | |_| |___| |  __/ | | \__ \  _  | |  \__ \
|_|  |_|\__, |____/ \___|_| |_|___/\___/|_|  |___/
        |___/                      2.3.2

51 MCO:BGN:INIT NODE,CP=RPNNA---,FQ=8,REL=255,VER=2.3.2
83 TSM:INIT
86 TSF:WUR:MS=0
92 TSM:INIT:TSP OK
96 TSM:INIT:STATID=157
102 TSF:SID:OK,ID=157
108 TSM:FPAR
116 ?TSF:MSG:SEND,157-157-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK:
425 TSF:MSG:READ,0-0-157,s=255,c=3,t=8,pt=1,l=1,sg=0:0
440 TSF:MSG:FPAR OK,ID=0,D=1
2138 TSM:FPAR:OK
2142 TSM:ID
2146 TSM:ID:OK
2150 TSM:UPL
2187 TSF:MSG:SEND,157-157-0-0,s=255,c=3,t=24,pt=1,l=1,sg=0,ft=0,st=OK:1
2392 TSF:MSG:READ,0-0-157,s=255,c=3,t=25,pt=1,l=1,sg=0:1
2406 TSF:MSG:PONG RECV,HP=1
2414 TSM:UPL:OK
2418 TSM:READY:ID=157,PAR=0,DIS=1
2449 TSF:MSG:SEND,157-157-0-0,s=255,c=3,t=15,pt=6,l=2,sg=0,ft=0,st=OK:0100
2656 TSF:MSG:READ,0-0-157,s=255,c=3,t=15,pt=6,l=2,sg=0:0100
2689 TSF:MSG:SEND,157-157-0-0,s=255,c=0,t=17,pt=0,l=5,sg=0,ft=0,st=OK:2.3.2
2723 TSF:MSG:SEND,157-157-0-0,s=255,c=3,t=6,pt=1,l=1,sg=0,ft=0,st=OK:0
4761 TSF:MSG:SEND,157-157-0-0,s=0,c=0,t=4,pt=0,l=0,sg=0,ft=0,st=OK:
4796 TSF:MSG:SEND,157-157-0-0,s=255,c=3,t=11,pt=0,l=11,sg=0,ft=0,st=OK:DimmableLED
4835 TSF:MSG:SEND,157-157-0-0,s=255,c=3,t=12,pt=0,l=3,sg=0,ft=0,st=OK:1.1
4855 MCO:REG:REQ
4870 TSF:MSG:SEND,157-157-0-0,s=255,c=3,t=26,pt=1,l=1,sg=0,ft=0,st=OK:2
5076 TSF:MSG:READ,0-0-157,s=255,c=3,t=27,pt=1,l=1,sg=0:1
5093 MCO:PIM:NODE REG=1
5099 MCO:BGN:STP
5122 TSF:MSG:SEND,157-157-0-0,s=0,c=2,t=3,pt=0,l=0,sg=0,ft=0,st=OK:
5140 MCO:BGN:INIT OK,TSP=1

I am trying to send data from an MQTT Gateway to a node. I expect the receive() method in my node sketch to be activated when I send an MQTT message to my MQTT gateway.
The mysgw receives this message, as shown in the logs.
My sketch in the node (id=156) has a sensor with id 7 of type V_DIMMER.
I assume the mysgw should forward this message to my node, but no sign of any activity on my node 156.

Do I have to call wait(time, command, datatype)? In the example sketch for the dimmer, there is no wait() call at all.
I have implemented the receive() method, but that doesn't seem to get called if I send an MQTT message to my gateway.
BTW: this gateway is working perfectly with a few other nodes and sensors I have in my MySensors network.

What am I doing wrong?

My sketch:

// Enable debug prints to serial monitor
#define MY_DEBUG 
#define MY_BAUD_RATE 38400


// Enable and select radio type attached
//#define MY_RADIO_RF24
#define MY_RADIO_RFM69
// If using mysensors version<2.3.0 and want to use NEW DRIVER (OH SO BAD NAMING):
#define MY_RFM69_NEW_DRIVER
// if you use MySensors 2.0 use this style 
//#define MY_RFM69_FREQUENCY   RF69_433MHZ
//#define MY_RFM69_FREQUENCY   RF69_868MHZ
//#define MY_RFM69_FREQUENCY   RF69_915MHZ

// Do not use default 100 to avoid conflicts with neighbors that won't change this id
#define MY_RFM69_NETWORKID 197

//#define MY_RFM69_FREQUENCY   RFM69_433MHZ
#define MY_RFM69_FREQUENCY   RFM69_868MHZ
//#define DEFAULT_RFM69_CS_PIN     (PB2)                      //!< DEFAULT_RFM69_CS_PIN
// default #define MY_RF69_IRQ_PIN PD2


//#define MY_RADIO_NRF24

// Comment it out for Auto Node ID #, original was 0x90=144
#define MY_NODE_ID 156

#include <MySensors.h>  
#include <DallasTemperature.h>
#include <OneWire.h>

#define COMPARE_TEMP 1 // Send temperature only if changed? 1 = Yes 0 = No

#define ONE_WIRE_BUS 8 // Pin where dallas sensor is connected 
#define MAX_ATTACHED_DS18B20 1

#define LED_PIN LED_BUILTIN
#define FADE_DELAY 10  // Delay in ms for each percentage fade up/down (10ms = 1s full-range dim)

unsigned long SLEEP_TIME = 3000; // Sleep time between reads (in milliseconds)


OneWire oneWire(ONE_WIRE_BUS); // Setup a oneWire instance to communicate with any OneWire devices (not just Maxim/Dallas temperature ICs)
DallasTemperature sensors(&oneWire); // Pass the oneWire reference to Dallas Temperature. 
float lastTemperature[MAX_ATTACHED_DS18B20];
int numSensors=0;
bool receivedConfig = false;
bool metric = true;
// Initialize temperature message
MyMessage msg(0,V_TEMP);
static int16_t currentLevel = 0;  // Current dim level...
MyMessage dimmerMsg(0, V_DIMMER);
MyMessage lightMsg(0, V_LIGHT);

void before()
{
  wdt_disable();
  pinMode(LED_PIN,OUTPUT);
  // Startup up the OneWire library
  Serial.println("before(): between wdt_disable and sensors.begin");
  sensors.begin();
  Serial.println("before(): after sensors.begin");
}

void setup()  
{ 
  // Pull the gateway's current dim level - restore light level upon node power-up
  request( 0, V_DIMMER );
  // requestTemperatures() will not block current thread
  //sensors.setWaitForConversion(false);
  Serial.print("End setup for node number ");Serial.println(MY_NODE_ID);
}

void presentation() {
  // Send the sketch version information to the gateway and Controller
  //Serial.print("Start presentation for node number ");Serial.print(MY_NODE_ID);
  sendSketchInfo("FanAttic_156", "1.1");

  // Fetch the number of attached temperature sensors  
  numSensors=1; // !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
  // !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!numSensors = sensors.getDeviceCount();


  present(7,S_DIMMER);
  // Present all sensors to controller
  for (int i=0; i<numSensors && i<MAX_ATTACHED_DS18B20; i++) {   
     present(i+1, S_TEMP);
  }
}

void loop() {
  //wdt_reset();
  Serial.println("Start loop");
  float temperature=37.9;
  send (msg.setSensor(0).set(temperature,1));
  Serial.println("msg sent");
  int16_t conversionTime = sensors.millisToWaitForConversion(sensors.getResolution());
  //sleep(SLEEP_TIME);
  wait(conversionTime);
  wait(SLEEP_TIME,C_SET,V_PERCENTAGE);
  Serial.println("End loop");
  analogWrite(LED_PIN,(int)128);
}
void receive(const MyMessage &message) {
  
    Serial.print( "Message received" );
  if (message.getType() == V_LIGHT || message.getType() == V_DIMMER || message.getType() == V_PERCENTAGE) {

    //  Retrieve the power or dim level from the incoming request message
    int requestedLevel = atoi( message.data );

    // Adjust incoming level if this is a V_LIGHT variable update [0 == off, 1 == on]
    requestedLevel *= ( message.getType() == V_LIGHT ? 100 : 1 );

    // Clip incoming level to valid range of 0 to 100
    requestedLevel = requestedLevel > 100 ? 100 : requestedLevel;
    requestedLevel = requestedLevel < 0   ? 0   : requestedLevel;

    Serial.print( "Changing level to " );
    Serial.print( requestedLevel );
    Serial.print( ", from " );
    Serial.println( currentLevel );

    fadeToLevel( requestedLevel );

    // Inform the gateway of the current DimmableLED's SwitchPower1 and LoadLevelStatus value...
    send(lightMsg.set(currentLevel > 0));

    // hek comment: Is this really nessesary?
    send( dimmerMsg.set(currentLevel) );
  }
}
/***
 *  This method provides a graceful fade up/down effect
 */
void fadeToLevel( int toLevel )
{

  int delta = ( toLevel - currentLevel ) < 0 ? -1 : 1;

  while ( currentLevel != toLevel ) {
    currentLevel += delta;
    analogWrite( LED_PIN, (int)(currentLevel / 100. * 255) );
    delay( FADE_DELAY );
  }
}

My mqtt call:

# // Topic structure: MY_MQTT_PUBLISH_TOPIC_PREFIX/NODE-ID/SENSOR-ID/CMD-TYPE/ACK-FLAG/SUB-TYPE
# carl/mysensors/156/0/????/?????/?????

# try to set dim level (test script/sketch)
# V_PERCENTAGE 3 command type
mqtt pub --host rpi3.home --mqttVersion 3 --topic 'carl/mysensors/156/7/1/0/3' --message 35

The output of mysgw (I also show some other messages, my problem is only about the sensor-id=7 messages):

Aug  4 23:25:51 rpi3 mysgw: TSF:MSG:READ,156-156-0,s=0,c=1,t=0,pt=7,l=5,sg=0:37.9
Aug  4 23:25:51 rpi3 mysgw: GWT:TPS:TOPIC=mysensors/all/156/0/1/0/0,MSG SENT
Aug  4 23:25:53 rpi3 mysgw: GWT:IMQ:TOPIC=carl/mysensors/156/7/1/0/3, MSG RECEIVED
Aug  4 23:25:53 rpi3 mysgw: GWT:TPS:TOPIC=mysensors/all/0/156/7/1/0,MSG SENT
Aug  4 23:25:54 rpi3 mysgw: TSF:MSG:READ,156-156-0,s=0,c=1,t=0,pt=7,l=5,sg=0:37.9
Aug  4 23:25:54 rpi3 mysgw: GWT:TPS:TOPIC=mysensors/all/156/0/1/0/0,MSG SENT
Aug  4 23:25:58 rpi3 mysgw: TSF:MSG:READ,156-156-0,s=0,c=1,t=0,pt=7,l=5,sg=0:37.9
Aug  4 23:25:58 rpi3 mysgw: GWT:TPS:TOPIC=mysensors/all/156/0/1/0/0,MSG SENT
Aug  4 23:25:58 rpi3 mysgw: TSF:MSG:READ,153-153-0,s=4,c=1,t=23,pt=0,l=4,sg=0:2.92
Aug  4 23:25:58 rpi3 mysgw: GWT:TPS:TOPIC=mysensors/all/153/4/1/0/23,MSG SENT

The debug output of the sketch (nothing to see about sensor-id=7):

Start loop
2039105 TSF:MSG:SEND,156-156-0-0,s=0,c=1,t=0,pt=7,l=5,sg=0,ft=0,st=OK:37.9
msg sent
End loop
Start loop
2042230 TSF:MSG:SEND,156-156-0-0,s=0,c=1,t=0,pt=7,l=5,sg=0,ft=0,st=OK:37.9
msg sent
End loop
Start loop
2045356 TSF:MSG:SEND,156-156-0-0,s=0,c=1,t=0,pt=7,l=5,sg=0,ft=0,st=OK:37.9
msg sent
End loop
Start loop
2048538 TSF:MSG:SEND,156-156-0-0,s=0,c=1,t=0,pt=7,l=5,sg=0,ft=0,st=OK:37.9
msg sent
End loop
Start loop
2051663 TSF:MSG:SEND,156-156-0-0,s=0,c=1,t=0,pt=7,l=5,sg=0,ft=0,st=OK:37.9
msg sent

My gateway has been compiled on a rpi3 with:

./configure --my-transport=rfm69 --my-rfm69-frequency=868 \
--my-gateway=mqtt --my-controller-ip-address=127.0.0.1 \
--my-mqtt-publish-topic-prefix=mysensors/all \
--my-mqtt-subscribe-topic-prefix=+/mysensors \
--my-mqtt-client-id=mygateway1 \
--my-mqtt-user=mysensors \
--my-mqtt-password=<PASS>

Finally, after reading the data sheet, I see the problem with the watchdog timer: its value can not be read.
Timer2 would be a nice option. I will have a look at that. Thanks for your answers!

@Yveaux I mean to modify the MySensors code, adding a new feature to MySensors. The accuracy can be very good if it is implemented in the right way.

Something like this, ignoring some details to be solved:
Sleep() sets a variable inSleep to true, starts the watchdog counter and goes to sleep.
When millis() is called and inSleep is true, millis() must add the elapsed time (as calculated from the watchdog timer) to the millis counter and return this value. This can only happen in an ISR.
Before returning, the sleep() function adds the elapsed time (as calculated from the watchdog timer) to the millis counter and sets inSleep to false, in one atomic action.

Accuracy can be as good as one or two millis per sleep I guess, max one at start and max one at stop.

What is the problem in modifying the millis() function in the way I proposed? Why is it impossible to add the watchdog count to the millis counter?
I know that currently this is not supported, but it would be a nice feuture to have.

Re: gw.sleep(...) and milis()
I have a use case on an ATMega328 where I want to know the millis, also after a sleep.

I have been thinking about it and this crossed my mind:

A solution could be to modify the millis() function. If it is called while the sleep function was active (that can only happen in an ISR I guess) it could set the millis counter to the saved millis at sleep start, plus the elapsed time since sleep start (i.e. the sleep timer count). It should also do so at sleep end.
Unfortunately I do not know much about the internals of the MySensors and Arduino software, so my suggestion might be complete nonsens.

My preferred solution would be the use of a real time OS, But that seems very complicated.

Yes, the pin change intgerrupt gives me what I want. The sleep call also wakes up on this interrupt, I thought it would only wake up in INT0 or INT1.

So for now my experiments with FreeRTOS end.

@electrik thanks for the hint!

@skywatch There should be no reason why a RTOS would drain batteries quickly, one could even expect the opposite. In a RTOS there is no need to loop, it can simply wait for any interrupt to arrive. That is theory. But I suspect FreeRTOS not doing a low-power sleep, and instead simply looping the cpu. There might be possibilities to do a low-power sleep (in the loop() method).
Connecting diodes to other pins is not very simple in my situation. I have this easypirmultisensorbox (https://github.com/EasySensors/easyPIRmultisensorsBox2) and soldering will not be easy:-).
The PIR sensor in this thing is connected to pin 7. In the MySensors sleep method only pins 2 and 3 can be used for interrupts.
This Arduino forum post https://forum.arduino.cc/index.php?topic=704977.0 didn't give me much hope. It is bare Arduino that already is problematic with FreeRTOS, and I guess that MySensors will make things even more complicated.
So maybe I will take a look at dirtier solutions, like writing my own sleep method. If I don't find a solution within a few months, I might even resort to one of the worst, dirtiest and appalling software mechanism ever: polling.

@jenspr said in Use FreeRTOS?:

The mysensor-library s building some kind of own operating system It hijacks a lot of things If you want to use them, you need to know how. For example save&read from EEPROM.
Of course it is possible to handle everything.

FreeRTOS would set the system on a standard base.
Mysensor could "use" the functionality of FreeRTOS including different platforms and focus on the software for radio communication, security, etc..

But FreeRTOS is also not very easy to understand and there are some tricky things to consider. In the end is it also complicated to understand everything as soon you need to go in detail. i am pretty sure that you would have had this or other questions if mysensors would set up on FreeRTOS.

In the end a normal mysensor node just wants to send some collected data over the air and is not doing a lot of other things in parallel.
So I dont know if it would make it more easy or more complicated.

Maybe it is worse a thought if somebody starts to think about mysensor 3.0?

Yes, I totally agree with you. The "problem" is, that I am a big fan of real time solutions, it was the subject of my final project at school. Things won't be easier, mistakes are easily made in concurrent programming.
I see these advantages of using a real time OS:

• better hardware support (more interrupts)
• nice features to support elegant real-time concurrent programming (queues, semaphores etc.)
  The disadvantages I see:
• extra software, more complexity (download, documentation to read)
• concurrent programming is error-prone

I am going to spend a few hours trying FreeRTOS with MySensors for my application, it looks like there are not many people who have tried it.

Carl

@skywatch I would like to see the "diode matrix solution", but I cannot find it. Is it in Arduino or in MySensors?

Re: The "new" Arduino Pro IDE

Currently I am running into a limitation of MySensors. Sleeping can be done on only two interrupts. I am using an Arduino ATMega 328.
The bare Arduino platform doesn’t provide a solution.
A real time OS could provide a neat solution and imho that would be the right place to implement such things as interrupts.
The first OS I found was FreeRTOS, there is a library available in the Arduino IDE.
It would be nice if MySensors had a mode to run on FreeRTOS.
Is there anybody who already built this?
I don’t have enough knowledge and time to develop this on my own. maybe someone can help me.

@hakha4 said in RPI Gateway Error:

I have updated Buster and Domoticz works ok but can't get the gw to work. Btw is it normal (ok) to get a number of warnings when compiling??

I am getting the same warnings (about string truncation), that doesn't make it "normal", but I have a working gateway in Buster on my raspberry pi (raspbian). The error is about the radio. You have a rf24, I use a rfm69, so that is quite different.
Another difference is the gateway version. Mine is using protocol version 2.4.0-alpha:

Apr 29 22:12:35 INFO  Starting gateway...
Apr 29 22:12:35 INFO  Protocol version - 2.4.0-alpha
Apr 29 22:12:35 DEBUG MCO:BGN:INIT GW,CP=RPNGL---,FQ=NA,REL=0,VER=2.4.0-alpha
Apr 29 22:12:35 DEBUG TSF:LRT:OK
Apr 29 22:12:35 DEBUG TSM:INIT
Apr 29 22:12:35 DEBUG TSF:WUR:MS=0
Apr 29 22:12:35 DEBUG TSM:INIT:TSP OK
Apr 29 22:12:35 DEBUG TSM:INIT:GW MODE
Apr 29 22:12:35 DEBUG TSM:READY:ID=0,PAR=0,DIS=0
Apr 29 22:12:35 DEBUG MCO:REG:NOT NEEDED
Apr 29 22:12:35 DEBUG MCO:BGN:STP
Apr 29 22:12:35 DEBUG MCO:BGN:INIT OK,TSP=1
Apr 29 22:12:35 DEBUG GWT:RMQ:CONNECTING...
Apr 29 22:12:35 DEBUG connected to 127.0.0.1
Apr 29 22:12:35 DEBUG GWT:RMQ:OK
Apr 29 22:12:35 DEBUG GWT:TPS:TOPIC=mysensors/all/0/255/0/0/18,MSG SENT
Apr 29 22:12:35 DEBUG TSM:READY:NWD REQ
Apr 29 22:12:35 DEBUG ?TSF:MSG:SEND,0-0-255-255,s=255,c=3,t=20,pt=0,l=0,sg=0,ft=0,st=OK:
Apr 29 22:12:40 DEBUG TSF:MSG:READ,153-153-0,s=4,c=1,t=23,pt=0,l=4,sg=0:3.75
Apr 29 22:12:40 DEBUG GWT:TPS:TOPIC=mysensors/all/153/4/1/0/23,MSG SENT

I am using this version of MySensors (branch development):

git status
On branch development
Your branch is up to date with 'origin/development'.

The last commits that I have are:

git log
commit 397b70f37b4fc5ffd15dddf49364cce2408e5bb5 (HEAD -> development, origin/development, origin/HEAD)
Author: Henrik Ekblad <henrik.ekblad@gmail.com>
Date:   Mon Jan 6 15:47:04 2020 +0100

    Fix logparser frequency regexp (#1380)

commit 9040272ccbf57026780ed10f589a5498db729ae9
Author: Olivier <tekka007@users.noreply.github.com>
Date:   Mon Dec 9 22:43:02 2019 +0100

    Development version 2.4.0-alpha (#1373)

So maybe you can try this newer development branch to build your gateway.

Looks like this was solved in branch develop at Jan, 6, 2019:
98355b2d ( tekka 2019-01-06 18:14:10 +0100 158) delay(1000);

@mfalkvidd The development branch has the same behavior. An edit in core/MyGatewayTransportMQTTClient.cpp (I added a "delay(3000)" after line 148 in reconnectMQTT(void)) made it a lot better!

mysgw (the mysensors gateway) sends a continuous flow of error messages to the log file if mysgw is started and the configured mqtt broker is not available. I am running a quite recent version (built on Dec 5, 2018), branch master.
My problem with mysgw is the excessive logging. I know I can resolve it by configuring my mqtt broker, but mysgw should not produce such a massive amount of logging if the mqtt broker is not available.

Dec  5 11:19:13 raspberrypi mysgw: failed to connect
Dec  5 11:19:13 raspberrypi mysgw: GWT:RMQ:MQTT RECONNECT
Dec  5 11:19:13 raspberrypi mysgw: connect: Connection refused
Dec  5 11:19:13 raspberrypi mysgw: failed to connect
Dec  5 11:19:13 raspberrypi mysgw: GWT:RMQ:MQTT RECONNECT
Dec  5 11:19:13 raspberrypi mysgw: connect: Connection refused
Dec  5 11:19:13 raspberrypi mysgw: failed to connect
Dec  5 11:19:13 raspberrypi mysgw: GWT:RMQ:MQTT RECONNECT
Dec  5 11:19:13 raspberrypi mysgw: connect: Connection refused
Dec  5 11:19:13 raspberrypi mysgw: failed to connect

Started on the command line (only first lines copied, there are hundreds of lines per second):

root@raspberrypi:/home/pi# mysgw
Dec 05 11:26:42 INFO  Starting gateway...
Dec 05 11:26:42 INFO  Protocol version - 2.3.0
Dec 05 11:26:42 DEBUG MCO:BGN:INIT GW,CP=RPNGL---,VER=2.3.0
Dec 05 11:26:42 DEBUG TSF:LRT:OK
Dec 05 11:26:42 DEBUG TSM:INIT
Dec 05 11:26:42 DEBUG TSF:WUR:MS=0
Dec 05 11:26:42 DEBUG TSM:INIT:TSP OK
Dec 05 11:26:42 DEBUG TSM:INIT:GW MODE
Dec 05 11:26:42 DEBUG TSM:READY:ID=0,PAR=0,DIS=0
Dec 05 11:26:42 DEBUG MCO:REG:NOT NEEDED
Dec 05 11:26:42 DEBUG MCO:BGN:STP
Dec 05 11:26:42 DEBUG MCO:BGN:INIT OK,TSP=1
Dec 05 11:26:42 DEBUG GWT:RMQ:MQTT RECONNECT
Dec 05 11:26:42 ERROR connect: Connection refused
Dec 05 11:26:42 ERROR failed to connect
Dec 05 11:26:42 DEBUG GWT:RMQ:MQTT RECONNECT
Dec 05 11:26:42 ERROR connect: Connection refused
Dec 05 11:26:42 ERROR failed to connect
Dec 05 11:26:42 DEBUG GWT:RMQ:MQTT RECONNECT
Dec 05 11:26:42 ERROR connect: Connection refused
Dec 05 11:26:42 ERROR failed to connect
Dec 05 11:26:42 DEBUG GWT:RMQ:MQTT RECONNECT
Dec 05 11:26:42 ERROR connect: Connection refused
Dec 05 11:26:42 ERROR failed to connect
Dec 05 11:26:42 DEBUG GWT:RMQ:MQTT RECONNECT
Dec 05 11:26:42 ERROR connect: Connection refused
Dec 05 11:26:42 ERROR failed to connect
Dec 05 11:26:42 DEBUG GWT:RMQ:MQTT RECONNECT
Dec 05 11:26:42 ERROR connect: Connection refused
Dec 05 11:26:42 ERROR failed to connect

@scalz A useful addition could be that the controller only runs on Windows. I bumped into this limitation after quite a few succesful steps. Is there a Linux or Java alternative?