Sensor database and display

[bkenobi]

I have an Uno set up to receive data from sensors of various types. The data is collected and then broadcast to my MQTT broker (Mosquitto) running on RPi. I am currently using HomeGenie to display the data, but I don't really like the looks. Also, the data only sticks around for a short time (not sure how long), so it's not really searchable I don't think.

What I want to do is save the data and have it available for viewing in a better way. I'd divide this question into those 2 parts (maybe linked).

1. Recommendations for better visualization
   My first thought was to use Pachube or WeatherUnderground, but those type of cloud storage servers seem to be temporary. If I don't store the data locally, then it appears I would lose it on a regular basis. I've seen Cacti and it looks great, but it apparently doesn't work with the web server HG uses. I am less than a novice with web servers (HG set it up and I haven't touched it), so I don't know if a second server (Apache) would conflict with whatever HG uses and I don't think HG can be switched to use anything else. I can work with the HG devs to see if the your recommendation(s) would be compatible or could be included in some other way.

2. Recommendations for data storage
   My goal is to save data at different levels of detail depending on age (e.g., current day by the minute, current week or month by the hour, all data by day). I can write a file with the data with a script, but I'm not sure about parsing it to keep only relevant data. If the data visualization approach from 1 is good enough, then I don't really need to worry about the data storage as much I suppose.

NOTE: I assume this topic has been covered many times but I didn't see a sub-forum for it and I failed to determine usable search terms. So, if someone could point me in the right direction with a link or two, that would be great.

[Fabien]

I have the similar setup (Uno, mosquitto on ODROID C1).
To display data you can use :

• emoncms.Easy to install. To feed emoncms you can use mqttwarn or node-red flow
• grafana : the best with influxdb as storage engine (influxdb is designed for time series databases) but hard to setup on armv7 device.
• freeboard. Not good fro graphs but nice jauges.

[bkenobi]

I looked at your recommendations when provided, but was busy with other things so forgot to reply. I now have 3 systems of sensors reporting to the Uno (Blueline power meter reader, Acurite 5n1 weather station, Acurite 592TX temperature sensors (1 currently, but up to 3 channels))and will be adding 1 or more local sensors to it as well (barometric pressure currently, but probably temperature also at a minimum). I have all of this sending data to the RPi via MQTT and am working on optimizing the location for the Uno for best reception/least packet loss. I'm also planning on tracking UPS performance as well as RPi CPU/memory.

That being said, I looked into emoncms again this morning and it appears that the data is stored on the cloud somewhere. If so, that's not ideal due to reasons I originally posted. I thought this would be the option I would utilize until I saw that. I've looked at Grafana/influxdb and they look great, but it will take more effort to figure them out. I decided against freeboard since I specificially want plots and not gauges.

I considered using rrdtools, but it looks like it's only capable of storing at a single frequency and then purges anything over a certain age (maintains a certain database size and loops to overwrite the oldest data next). There's a number of tutorials/guides, but I haven't found one that's a close enough match to proceed. I have a working HA server at the moment, so I'm leery of installing too much and risk corrupting things. When I do proceed, I'll certainly back up the SD first!