Boards esp8266 v3.1.0 or Newer Doesn't Work with MySensors

[d-smes]

Was trying to get a ESP8266-based Gateway working and kept getting a "exit status 1" error that then said "Error compiling for board xxx" where xxx is any ESP8266 board. Once I downgraded esp8266 boards file to version 3.0.2, the compile error was gone, and everything worked fine.

The only clue as to what's wrong with newer versions is this text from the compile window (except for first and last line, everything was in orange text):

"C:\\Users\\Doug\\AppData\\Local\\Arduino15\\packages\\esp8266\\tools\\xtensa-lx106-elf-gcc\\3.1.0-gcc10.3-e5f9fec/bin/xtensa-lx106-elf-g++" -D__ets__ -DICACHE_FLASH -U__STRICT_ANSI__ -D_GNU_SOURCE -DESP8266 "@C:\\Users\\Doug\\AppData\\Local\\Temp\\arduino_build_788282/core/build.opt" "-IC:\\Users\\Doug\\AppData\\Local\\Arduino15\\packages\\esp8266\\hardware\\esp8266\\3.1.0/tools/sdk/include" "-IC:\\Users\\Doug\\AppData\\Local\\Arduino15\\packages\\esp8266\\hardware\\esp8266\\3.1.0/tools/sdk/lwip2/include" "-IC:\\Users\\Doug\\AppData\\Local\\Arduino15\\packages\\esp8266\\hardware\\esp8266\\3.1.0/tools/sdk/libc/xtensa-lx106-elf/include" "-IC:\\Users\\Doug\\AppData\\Local\\Temp\\arduino_build_788282/core" -c "@C:\\Users\\Doug\\AppData\\Local\\Arduino15\\packages\\esp8266\\hardware\\esp8266\\3.1.0/tools/warnings/extra-g++" -Os -g -free -fipa-pta -Werror=return-type -mlongcalls -mtext-section-literals -fno-rtti -falign-functions=4 -std=gnu++17 -MMD -ffunction-sections -fdata-sections -fno-exceptions -DMMU_IRAM_SIZE=0x8000 -DMMU_ICACHE_SIZE=0x8000 -DNONOSDK22x_190703=1 -DF_CPU=80000000L -DLWIP_OPEN_SRC -DTCP_MSS=536 -DLWIP_FEATURES=1 -DLWIP_IPV6=0 -DARDUINO=10813 -DARDUINO_ESP8266_WEMOS_D1MINI -DARDUINO_ARCH_ESP8266 "-DARDUINO_BOARD=\"ESP8266_WEMOS_D1MINI\"" "-DARDUINO_BOARD_ID=\"d1_mini\"" -DFLASHMODE_DIO "-IC:\\Users\\Doug\\AppData\\Local\\Arduino15\\packages\\esp8266\\hardware\\esp8266\\3.1.0\\cores\\esp8266" "-IC:\\Users\\Doug\\AppData\\Local\\Arduino15\\packages\\esp8266\\hardware\\esp8266\\3.1.0\\variants\\d1_mini" "-IC:\\Users\\Doug\\Documents\\Arduino\\libraries\\MySensors" "-IC:\\Users\\Doug\\AppData\\Local\\Arduino15\\packages\\esp8266\\hardware\\esp8266\\3.1.0\\libraries\\SPI" "-IC:\\Users\\Doug\\AppData\\Local\\Arduino15\\packages\\esp8266\\hardware\\esp8266\\3.1.0\\libraries\\ESP8266WiFi\\src" "-IC:\\Users\\Doug\\AppData\\Local\\Arduino15\\packages\\esp8266\\hardware\\esp8266\\3.1.0\\libraries\\EEPROM" "C:\\Users\\Doug\\AppData\\Local\\Temp\\arduino_build_788282\\sketch\\LoRa_GatewayESP8266OTA_RFM95_v1.ino.cpp" -o "C:\\Users\\Doug\\AppData\\Local\\Temp\\arduino_build_788282\\sketch\\LoRa_GatewayESP8266OTA_RFM95_v1.ino.cpp.o"
In file included from C:\Users\Doug\Documents\Arduino\libraries\MySensors/MySensors.h:235,
                 from C:\Users\Doug\Documents\Arduino\LoRa_GatewayESP8266OTA_RFM95_v1\LoRa_GatewayESP8266OTA_RFM95_v1.ino:174:
C:\Users\Doug\Documents\Arduino\libraries\MySensors/core/MyGatewayTransportEthernet.cpp: In function 'bool gatewayTransportAvailable()':
C:\Users\Doug\Documents\Arduino\libraries\MySensors/core/MyGatewayTransportEthernet.cpp:430:44: warning: 'WiFiClient WiFiServer::available(uint8_t*)' is deprecated: Renamed to accept(). [-Wdeprecated-declarations]
  430 |     clients[i] = _ethernetServer.available();
      |                                            ^
In file included from C:\Users\Doug\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.0\libraries\ESP8266WiFi\src/ESP8266WiFi.h:40,
                 from C:\Users\Doug\Documents\Arduino\libraries\MySensors/hal/architecture/ESP8266/MyHwESP8266.h:23,
                 from C:\Users\Doug\Documents\Arduino\libraries\MySensors/hal/architecture/ESP8266/MyHwESP8266.cpp:20,
                 from C:\Users\Doug\Documents\Arduino\libraries\MySensors/MySensors.h:58,
                 from C:\Users\Doug\Documents\Arduino\LoRa_GatewayESP8266OTA_RFM95_v1\LoRa_GatewayESP8266OTA_RFM95_v1.ino:174:
C:\Users\Doug\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.0\libraries\ESP8266WiFi\src/WiFiServer.h:85:14: note: declared here
   85 |   WiFiClient available(uint8_t* status = NULL) __attribute__((deprecated("Renamed to accept().")));
      |              ^~~~~~~~~
In file included from C:\Users\Doug\Documents\Arduino\libraries\MySensors/MySensors.h:235,
                 from C:\Users\Doug\Documents\Arduino\LoRa_GatewayESP8266OTA_RFM95_v1\LoRa_GatewayESP8266OTA_RFM95_v1.ino:174:
C:\Users\Doug\Documents\Arduino\libraries\MySensors/core/MyGatewayTransportEthernet.cpp:445:48: warning: 'WiFiClient WiFiServer::available(uint8_t*)' is deprecated: Renamed to accept(). [-Wdeprecated-declarations]
  445 |   EthernetClient c = _ethernetServer.available();
      |                                                ^
In file included from C:\Users\Doug\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.0\libraries\ESP8266WiFi\src/ESP8266WiFi.h:40,
                 from C:\Users\Doug\Documents\Arduino\libraries\MySensors/hal/architecture/ESP8266/MyHwESP8266.h:23,
                 from C:\Users\Doug\Documents\Arduino\libraries\MySensors/hal/architecture/ESP8266/MyHwESP8266.cpp:20,
                 from C:\Users\Doug\Documents\Arduino\libraries\MySensors/MySensors.h:58,
                 from C:\Users\Doug\Documents\Arduino\LoRa_GatewayESP8266OTA_RFM95_v1\LoRa_GatewayESP8266OTA_RFM95_v1.ino:174:
C:\Users\Doug\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.1.0\libraries\ESP8266WiFi\src/WiFiServer.h:85:14: note: declared here
   85 |   WiFiClient available(uint8_t* status = NULL) __attribute__((deprecated("Renamed to accept().")));
      |              ^~~~~~~~~
In file included from C:\Users\Doug\Documents\Arduino\libraries\MySensors/MySensors.h:441,
                 from C:\Users\Doug\Documents\Arduino\LoRa_GatewayESP8266OTA_RFM95_v1\LoRa_GatewayESP8266OTA_RFM95_v1.ino:174:
C:\Users\Doug\Documents\Arduino\libraries\MySensors/hal/architecture/ESP8266/MyMainESP8266.cpp: In function 'bool can_yield()':
C:\Users\Doug\Documents\Arduino\libraries\MySensors/hal/architecture/ESP8266/MyMainESP8266.cpp:95:9: error: 'cont_can_yield' was not declared in this scope; did you mean 'can_yield'?
   95 |  return cont_can_yield(g_pcont);
      |         ^~~~~~~~~~~~~~
      |         can_yield
C:\Users\Doug\Documents\Arduino\libraries\MySensors/hal/architecture/ESP8266/MyMainESP8266.cpp: In function 'void esp_yield_within_cont()':
C:\Users\Doug\Documents\Arduino\libraries\MySensors/hal/architecture/ESP8266/MyMainESP8266.cpp:101:2: error: 'cont_yield' was not declared in this scope; did you mean 'can_yield'?
  101 |  cont_yield(g_pcont);
      |  ^~~~~~~~~~
      |  can_yield
C:\Users\Doug\Documents\Arduino\libraries\MySensors/hal/architecture/ESP8266/MyMainESP8266.cpp: In function 'void loop_task(ETSEvent*)':
C:\Users\Doug\Documents\Arduino\libraries\MySensors/hal/architecture/ESP8266/MyMainESP8266.cpp:199:26: error: invalid operands of types 'void' and 'int' to binary 'operator!='
  199 |  if (cont_check(g_pcont) != 0) {
      |      ~~~~~~~~~~~~~~~~~~~ ^~ ~
      |                |            |
      |                void         int
Using library MySensors at version 2.3.2 in folder: C:\Users\Doug\Documents\Arduino\libraries\MySensors 
``

[mfalkvidd]

@d-smes yes, almost every upgrade of the esp core breaks compatibility with MySensors. The symptoms are wildly different for different esp versions. Some versions give different compilation errors or warnings, some versions compile fine but fail when the esp boots. This makes it very hard for anyone to realize what problem they are having, and also makes it hard to search the forum for "similar" problems.

There are two pull requests attempting to fix this:
https://github.com/mysensors/MySensors/pull/1513
https://github.com/mysensors/MySensors/pull/1524

The one by virtual-maker makes changes that make it possible to use existing sketches, but this fix is likely to break again, just like it has done multiple times before.
The one from d-a-v (who is a big contributor to the esp8266 Arduino core) should be a more permanent solution, but will require changes to all user sketches. (See here for more details by virtual-maker)

The contents of the pull requests and the different interactions between MySensors and the esp8266 core are way too complex for me to comprehend so I can't explain why this is such a big problem. But my gut feeling is that none of the solutions are desirable, but the current situation is not desirable either.

The only viable solution so far seems to be to reiterate the recommendation to use ESP8266 core version 2.6.2 which is listed in the release notes [1] [2] but as you've discovered yourself this is not very effective because nobody reads/remembers it.

An alternative could be to remove support for esp in the next MySensors release. Doing so would stop people from spending time scratching their head trying to figure out what is wrong, but I doubt that would make people very happy either.

[monte]

@mfalkvidd I guess that's the downside of a decision to make library do everything automagically without need to have any constructors or call update functions in loop, or declare callbacks. For the novice user it's for sure beneficial, but I don't think it's necessary. The are many arduino libraries like web servers and display libraries, that have many users, but don't try to hide everything from them behind scenes, and it makes them easily modifiable for one's own needs and also makes easier support and development.

[mfalkvidd]

@monte yes, could be. I prefer control myself, which is why I have never used NodeManager (which was created because MySensors was seen as exposing too much of the underlying stuff).

From what I understood, much of the early success for MySensors came from being easy to use, but that was long before I became aware of MySensors existance.

A big rewrite of MySensors might be a solution. The challenge would be for interested people to gather enough people, and plan and implement that rewrite together. Recent interest levels suggest that such an effort would be unlikely.

[monte]

@mfalkvidd nowadays there are so many cheap and easy solutions for what mysensors does, that I doubt it will gain any interest from non-enthusiasts as it is now. And it also lacks something to attract enthusiasts who can contribute anything back.

[mfalkvidd]

@monte agreed. That’s why I think there is only one viable solution.

[d-smes]

@mfalkvidd Thanks for the prompt reply and your persistent monitoring of the forums. Correct- I didn't even realize there were compatibility issues until I myself fell into this hole. Thanks for pointing it out.
After I posted, I tried to get a ESP32-based gateway going and ran into the same compile error. Then, I also tried uploading a blink sketch and that too yielded a compile error. This prompted me to update my Arduino IDE to v1.8.19 (from 1.8.13) but that didn't help. So in this case, it's not a MySensors issue, but an issue with ESP32 boards in general (at least on my machine). It sucks that advanced boards have become beyond the reach of hobbyists like me...

[monte]

@d-smes no, it IS a MySensors issue, that's the point. Another point is that probably nobody will fix it anytime soon.

[JeeLet]

@monte said in Boards esp8266 v3.1.0 or Newer Doesn't Work with MySensors:

il existe tellement de solutions bon marché et faciles pour ce que font mysensors

as rich in possibilities as this does not exist :)