@soloam great work.
Note: If you are able to upgrade to Java 8, you should no longer need to set XX:MaxPermSize
If you want to dig deeper into the memory usage, use the command jstat -gcutil to get access to a lot of information.
I don't know why but now OTA works better; I still get quite some NACKs on the gateway but at least most of the times the OTA is successful even with signing enabled. With ACK enabled the transfer speed is at least 4/5 times slower
I have 2 controllers connected to 2 ethernet gateways running all on the same rpi. I don't know really what happens to the mqtt broker if it gets clogged with a queue of undeliverable messages, you have to debug it and see if the mqtt grows too much
If you want to use esp8266 as nodes, there are also some alternative firmwares like espurna, espeasy or tasmota (pretty much the same as espurna), while you can show data on the node-red dashboard connecting with mqtt
However, when a sensor calls back to the controller, both HomeAssistant and myController send out replies, and due to this, i think the messages clash, and the controllers endup sending it multiple times.
I do not know your preference about, which controller wants to response for your sensors request. Suppose you want MyController only on listen mode, you have to change MQTT Topic publish to some dummy topic on MyController gateway settings.