Zigbee gateway with support for multiple vendors?

[NeverDie]

@rejoe2 Thanks for your post. You raise a good point about conformance not equating to performance. For instance, no matter what the radio standard, if the sensor has a lousy antenna, the range will be impaired. Or, in your example, extremely bad energy management leading to a dead battery within hours.

Is sending the raw received Zigbee 3.0 packet to MQTT sufficient? Or does it have to be parsed and then the parsed data forwarded to MQTT? If the former, then maybe I could throw together a simple Zigbee C2530, CC2531, or CC2540 (not sure which) zigbee to serial transceiver like (https://www.aliexpress.com/item/1005003103244824.html?spm=a2g0o.productlist.0.0.7ab93c6cbS7cMw&algo_pvid=4645ab4e-ac1e-4803-919d-f93dd10803fa&algo_exp_id=4645ab4e-ac1e-4803-919d-f93dd10803fa-7&pdp_ext_f={"sku_id"%3A"12000024754357316"}&pdp_npi=2%40dis!USD!!3.72!!!2.11!!%400b0a555e16524414970292922ea46f!12000024754357316!sea) together with a Wemos D1 Mini to simply read the packet and forward it to MQTT . Would that be sufficient? i.e. the same/similar idea as this:
https://www.openhardware.io/view/394/Minimalist-LoRa-Ra-01-Shield-for-WeMos-D1-Mini
or this:
https://www.openhardware.io/view/392/Minimalist-RFM69HW-Shield-for-Wemos-D1-Mini

but using a serial Zigbee 3.0 transceiver module instead. If it's as simple as that, then I could bang out a Wemos adapter probably pretty fast in KiCAD and post it to openhardware.io. If that would work, I think I'd prefer it over a USB adapter, since a USB in turn would have to be plugged into a raspberry pi or something (unless you could plug it directly into your Home Assistant server and still get good range), which seems like a lot of overhead for doing something so simple. Also, this way you could put your zigbee 3.0 transceiver (maybe more than one) close enough to your Zigbe 3.0 sensors that you wouldn't drop packets. On the other hand, if Zigbee 3.0 implies 2.4Ghz band, then maybe sitting it right next to an esp8266 isn't going to work so well.... Hmmmm..... thinking about it now, I suppose an ethernet gateway would be better for that reason wouldn't it? In which case, yeah, I finally see the appeal of a Zigbee-to-usb dongle plugged into a pi or something like that.

Anyhow, when it comes to cheap Aliexpress wireless sensors, branding does seem to carry some weight. As near as I can tell, Sonoff seems like one brand that's willing to be fairly open with its stuff (or at least was in the past. Not sure if it's still true or not).

[rejoe2]

@NeverDie

Is sending the raw received Zigbee 3.0 packet to MQTT sufficient?

Although I've done quite a lot of funny "practictioners testing" with different ZigBee stuff, I'm not very deep into all the details. So here's something like a personal summary wrt. to the technical aspects - more from a pure consumer view though:

• using a "simple MQTT interface" might be possible, but most likely processing the data isn't fun at all;
• with Tasmota (https://tasmota.github.io/docs/Zigbee/) there's already a ZigBee2mqtt implementation running on ESP8266/ESP32 base. I personally didn't like that solution, especially as there had not been any "over time consistency" in the messages. Once you "rejoin" the ZigBee mesh with a device, this one gets a new identifier - no possibility to track which one it has been in the past; (additionally the json structure of the messages is basically a mess, too, imo)
• especially CC253x is very limited when used as coordinator, there's better silicon available for that purpose. For diy solutions it's a well-known chip, see here and mysensors-ZigBee-diy-discussion.
• wrt. to "coordinator software" using one of the long-term established solutions imo is the easier way than developing one your own. Choose between deconz, zigbee2mqtt or (perhaps, no experience with that) openhab and you're done. The later two afaik can use the "raw serial stream" of quite a lot of common devices - this is e.g. why I originally bought the "Lidl Starter Set": The bridge has not only a powerfull coordinator chip on board, but also a LAN interface and might be used together with them after beeing hacked (no link at hand, sorry).

Obviously, doing a really structured job on coordinating (and visualizing) the mesh network requires some more computing power, so this will exceed at least what's possible with an ESP8266, but as one has typically running a more powerful box for the automation itself, running e.g. deconz or zigbee2mqtt on top of that doesn't really make a difference.

Apart from that:

• What's also very obscure is the question, if "bindings" are possible between specific devices (direct communication between two devices without controller software interaction). And - even if that's possible - if it's really used when building "groups" within the controller software...
• As soon as there's enough "router" devices, the network itself seems to be rather stable. In the beginning, I sometimes had trouble especially with "tradfri" bulbs that had to be unpowered from time to time (could also have improved by newer firmware or changing the location they were used, didn't investigate much in that) (same with one specific Xiaomi temp/hum-sensor).

[NeverDie]

@rejoe2 Yipes! Thanks for your detailed reply. it does sound a lot more complicated than I had supposed. My naive approach to making a gateway probably wouldn't work then.

[rejoe2]

@NeverDie said in Zigbee gateway with support for multiple vendors?:

@rejoe2 Yipes! Thanks for your detailed reply. it does sound a lot more complicated than I had supposed. My naive approach to making a gateway probably wouldn't work then.

You're welcome!
Originally I started quite similar and had the idea to write a plugin for FHEM that uses the output of a CC253x directly (similar to what openhab seems to do), but finally gave up before even really beginning...
At that time, the ConBee II was the only powerfull USB adopter available, so I decided to switch to deconz from zigbee2mqtt (CC2531, ConBee wasn't supported at that point in time). I still don't like the strict split between "sensors" and "lights" resulting in 2 or more "devices" in the long-time existing "bridge"-implementation in FHEM for one and the same hardware (e.g. a plug with power measuring will be 3 devices in the end), but that's not really that important to change horses again (zigbee2mqtt will use the full hardware address, if you refrain from using "friendly names" (a real euphemism, btw.))

[monte]

@NeverDie I think this is what you are looking for:
https://tasmota.github.io/docs/Zigbee/#hardware
https://github.com/esphome/feature-requests/issues/1111
Also look at "Network" category: https://www.zigbee2mqtt.io/guide/adapters/
If you want to make something yourself, you need to make UART to LAN adapter basically. Home Assistant can use network socket as serial address https://www.home-assistant.io/integrations/zha/#zigate-or-sonoff-zbbridge-devices. So basically you think in the right direction, just skip MQTT part:)

[hlehoux]

@NeverDie avoid tuya devices: they are intendly made to work only with the tuya app.
people try to work this around , for instance in ZHA quirks, but this is a nightmare of reverse engineering

[mfalkvidd]

Thanks everyone.

I now have a working Zigbee network. Used zigbee2mqtt, sonoff zigbee 3.0 usb dongle, a bunch of sonoff temp/hum sensors. I was able to join three existing Ikea trådfri devices to the same network, where two of them act as routers to make the network more reliable.

I'm impressed with the reliability of the network, and the battery life of the sensors (powered by coin cell batteries and sometimes updating as often as every 10 seconds). This looks reliable enough to be used in projects.

[mfalkvidd]

I spoke too soon. One of the SNZB-02 (the one in the attic) stopped reporting. Before it dropped off, it reported a lqi of 105 and 100% battery. The Ikea tradfri router is about 6m from the attic device.

I put a second device in the same location and pressed the join button. It registed in the network but has never reported any values.

I also put a device in the car. I understand this is not a normal use case, but I thought the device would rejoin the network every time the car is back home. But that doesn't seem to work.

So so far, the robustness of the network doesn't seem to be as good as I hoped. Maybe Zigbee is not the right solution for projects.

[rejoe2]

@mfalkvidd said in Zigbee gateway with support for multiple vendors?:

I spoke too soon. One of the SNZB-02 (the one in the attic) stopped reporting. Before it dropped off, it reported a lqi of 105 and 100% battery. [...]
So so far, the robustness of the network doesn't seem to be as good as I hoped. Maybe Zigbee is not the right solution for projects.

As already mentionned:
@rejoe2 said in Zigbee gateway with support for multiple vendors?:

So basically, imo in most 3.0 cases with "simple devices" like sensors and bulbs the question is not whether a specific device can be integrated, it's other issues that matter:

• the quality itself. I had some very disappointing buys, e.g. an rgbw bulb from Lidl with painfull white light color, noname Xiaomi-motionsensor clone (battery empty after some hours?), same with Sonoff motion sensors, that additionally didn't work at all...

Especially the SonOff stuff seems to be disappointing. For the rest, it mostly depends on the stability of the network itself (amount of "always powered" router type devices).

But finally, I still prefer using (wired) MySensors for some of the things that's still in the pipeline to come somewhen in time :grin:...

[novicit]

@mfalkvidd Please keep us informed on how this goes for you.

A few months ago I also implemented a zigbee network for similar reasons. The nodes are very inexpensive, small, and coin cell powered. I am using these for common applications like temp or motion. That way I can use MySensors nodes for the more custom or complicated applications. So far I have had no problems, actually am surprised at how reliable and fast it is. I have tried to limit the number of different vendors as I understand that can present a problem. I opted to use ZHA (Home Assistant) instead of MQTT, so far works great. Using Sonoff USB 3.0, all other Sonoff zigbee nodes, Sengled bulbs, and Aqara cube, vibration & water detectors. If I run into any problems will also post here.

[Using Home Assistant, Intel NUC, Zwave, MySensors 2.4 & 915, Zigbee, ESP]

[hlehoux]

What is also most interesting with zigbee is a very important number of devices, and low price of them compared to zwave for instance.