Plotting graphs with plot.ly service with luup scene using REST API

[samppa]

I have been searching for the easiest graphing solution to see how sensor data behaves and I think I have found it - at least it looks promising.

After creating plot.ly account create API key in https://plot.ly/settings/api

Then create scene with following luup that makes the REST API request to upload new the value into the plot.
So the example here is that I record time series one data point at a time and trigger the scene with 5 minute interval schedule,

Replace <username>, <APIKey>, <nameForGraph> and <graphTitle> below with your (hard coded) values and the datavelue is of course the sensor data that you want to plot (I plot humidity).

LUUP CODE:

local datavalue = luup.variable_get("urn:micasaverde-com:serviceId:HumiditySensor1", "CurrentLevel", 83)
local http = require("socket.http")
http.TIMEOUT = 5
local ts = os.date("%Y-%m-%d %X")
result, status = http.request("https://plot.ly/clientresp",'un=<username>&key=<APIKey>&origin=plot&platform=lisp&args=[{"x":["'..ts..'"],"y":['..datavalue..']}]&kwargs={"filename":"<nameForGraph>","fileopt":"extend","style":{"type":"scatter"},"traces":[0],"layout":{"title":"<graphTitle>"},"world_readable":false}')

[samppa]

Next I would like to plot a value from PLEG condition. But can those be accessed from scene luup code?

[Nuubi]

Excellent, thanks! This is much needed.
For some reason, can't get this to work...LUA error.
I'm trying to send MySensors data to Plotly...
Anyway, does e.g. <username> mean that you write username, or <username> to the script? (Oh yeah, username referring to the actual username )

[samppa]

@Nuubi
Yes, the username and API key are shown on the plot.ly api settings page.

Here is my code that works for 5 lines in one graph and appends data every 5 minutes:

---

-- Get variables
local BRHum = luup.variable_get("urn:micasaverde-com:serviceId:HumiditySensor1", "CurrentLevel", 83)
local BRTemp = luup.variable_get("urn:upnp-org:serviceId:TemperatureSensor1", "CurrentTemperature", 84)
local ACBoost = 100 * luup.variable_get("urn:upnp-org:serviceId:SwitchPower1", "Status", 106)
local ACMech = 100 * luup.variable_get("urn:upnp-org:serviceId:SwitchPower1", "Status", 107)
local ACEff = luup.variable_get("urn:upnp-org:serviceId:Dimming1", "LoadLevelStatus", 88    )

local ts = os.date("%Y-%m-%d %X")

-- Send to plot
local http = require("socket.http")
http.TIMEOUT = 5
result, status = http.request("https://plot.ly/clientresp",'un=<username>&key= <APIKey>&origin=plot&platform=lisp&args=[{"x":["'..ts..'"],"y":['..BRHum..'],"name":"Humidity"},{"x":["'..ts..'"],"y": ['..ACBoost..'],"name":"AC Boost"},{"x":["'..ts..'"],"y":['..BRTemp..'],"name":"Temperature"},{"x":["'..ts..'"],"y":['..ACMech..'],"name":"Mech Sensor"},{"x":["'..ts..'"],"y":['..ACEff..'],"name":"AC Efficiency"}]&kwargs={"filename":"BathroomHumidity","fileopt":"extend","style":{"type":"scatter"},"traces":[0,1,2,3,4],"layout":{"title":"Bathroom Humidity"},"world_readable":false}')

---

I know the JSON syntax is cryptic in one line without pretty formatting..

[korttoma]

@samppa this is sweet . Thank you so much for sharing this!

[Nuubi]

Uh...oh...still.. do you need to initialize the graph or what? Still not getting this to work, sorry for being such a dummie :-E

[korttoma]

You should use username and not <username>. The same goes for the API key.

In UI5 you can test your Lua in Vera from APPS -> Develop Apps -> Test Luup code

Here is a few more examples of variable_get:
local InAirHum = luup.variable_get("urn:micasaverde-com:serviceId:HumiditySensor1", "CurrentLevel", 347)
local OutTemp = luup.variable_get("urn:upnp-org:serviceId:TemperatureSensor1","CurrentTemperature", 229)
local InAirTemp = luup.variable_get("urn:upnp-org:serviceId:TemperatureSensor1","CurrentTemperature", 346)
local InAirQ = luup.variable_get("urn:upnp-org:serviceId:VContainer1", "Variable1", 348)

[korttoma]

I'm having trouble with one variable to. I would like to have the status of a qubino switch in the same graph.
I tried this:
local VentOn = 100 luup.variable_get("urn:upnp-org:serviceId:BinaryLight1", "Status", 356)

It gives me the value 100 in the graph. Please let me know what this 100 in front of the luup.variable_get does. If I remove it the Luup code fails.

[samppa]

@korttoma said:

I'm having trouble with one variable to. I would like to have the status of a qubino switch in the same graph.
I tried this:
local VentOn = 100 luup.variable_get("urn:upnp-org:serviceId:BinaryLight1", "Status", 356)

It gives me the value 100 in the graph. Please let me know what this 100 in front of the luup.variable_get does. If I remove it the Luup code fails.

Actually there should be asterisk character between 100 and the luup.variable_get and the (forum editor does not show that) since the returned value 1 or 0 is scaled to 100 so try:

local VentOn = 100 * luup.variable_get("urn:upnp-org:serviceId:BinaryLight1", "Status", 356)

Then you get 100 if binary light is on. Remove the 100 if you like it to be 1 or 0.

[samppa]

@Nuubi said:

Uh...oh...still.. do you need to initialize the graph or what? Still not getting this to work, sorry for being such a dummie :-E

You just send the data and the graph named with "filename":"xxxx" gets created if not yet existing.

[korttoma]

@samppa said:

Actually there should be asterisk character between 100 and the luup.variable_get and the (forum editor does not show that) since the returned value 1 or 0 is scaled to 100 so try:

local VentOn = 100 * luup.variable_get("urn:upnp-org:serviceId:BinaryLight1", "Status", 356)
Then you get 100 if binary light is on. Remove the 100 if you like it to be 1 or 0.

I thought it was a multiplier of some kind. Anyhow the code fails allso if I add the asterisk so I thik it has a problem with:

luup.variable_get("urn:upnp-org:serviceId:BinaryLight1", "Status", 356)

Any suggestions what I should try instead? Tried also this but no luck:

luup.variable_get("urn:micasaverde-com:serviceId:BinaryLight1", "Status", 356)

[korttoma]

Had a chat with @BulldogLowell and he taught me that you can hover the mouse on a variable to check the complete variable name so it turned out I had to use the following:

luup.variable_get("urn:micasaverde-com:serviceId:SwitchPower1", "Status", 356)

[Nuubi]

@samppa Yep, works now, thanks!
The problem turned out to be the actual value I wanted to send. Need to figure out how to read MySensors based values in LUA...

[BulldogLowell]

@Nuubi

have you tried luup.inet.wget( )

local brightness = luup.variable_get("urn:upnp-org:serviceId:Dimming1", "LoadLevelStatus", 118)
luup.inet.wget("192.168.1.50//?brightLevel="..brightness.."&", 1)

[korttoma]

Anyone figured out a way to add another variable to an old graph without deleting the file from plot.ly? Seems like it does not include the new variable unless you delete and create a new file.

[samppa]

@korttoma said:

Anyone figured out a way to add another variable to an old graph without deleting the file from plot.ly? Seems like it does not include the new variable unless you delete and create a new file.

I have noticed the same thing; If you want to add a trace you need to delete the file..
So perhaps it is wise to be prepared and add a few extra traces for some future use.

[korttoma]

@samppa said:

perhaps it is wise to be prepared and add a few extra traces for some future use.

Good suggestion for a workaround but it does not seem to pick up name changes for the trace either. Tried also to edit one of the plots on the website but when I wanted to save I could not save the changes to the file used by "Vera". I had to save the file with a different name so no editing seems to be possible.

[korttoma]

Since the nightly heal at 2 in the morning no data is transferred to plot.ly any more anyone else see this and know how to solv it?

[Dwalt]

@korttoma

My plots stopped last night as well, last data point transmitted was at 12:03am GMT, but not after a nightly heal.

-Dwalt

[korttoma]

12 GMT is 2 in my timezone so I guess it was just a coincidence that it happened at the time of the heel. Maybe plot.ly blocked us for some reason.

[samppa]

My graphs stopped as well. I have asked what is going on.. Let's see when it gets back up again..

[korttoma]

I sent them some feedback to but have not heard anything yet. Maybe we should try the "Streaming API" instead?

[Dwalt]

@korttoma

I tried to set up a new graph using the REST API and could not get it to register with Plot.ly. Without any explanation from Plot.ly, I tried sending the same data streams to ThingSpeak.com and it set up with out any trouble. The API is slightly different but just as easy to use. I set up a scene to run every 15 minutes with the following code:

-- Get Light Levels
local LR_LIGHT_LEVEL = luup.variable_get("urn:micasaverde-com:serviceId:LightSensor1", "CurrentLevel", 47)
local BR_LIGHT_LEVEL = luup.variable_get("urn:micasaverde-com:serviceId:LightSensor1", "CurrentLevel", 55)

- - Get Temps
local LR_TEMP = luup.variable_get("urn:upnp-org:serviceId:TemperatureSensor1", "CurrentTemperature", 46)
local BR_TEMP = luup.variable_get("urn:upnp-org:serviceId:TemperatureSensor1", "CurrentTemperature", 54)
local OUT_TEMP = luup.variable_get("urn:upnp-org:serviceId:TemperatureSensor1", "CurrentTemperature", 23)

-- Send data to ThingSpeak.com
local http = require("socket.http")
http.TIMEOUT = 5
result, status = http.request("http://api.thingspeak.com/update?key=Insert Your API Key here&field1="..LR_TEMP.."&field2="..BR_TEMP.."&field3="..OUT_TEMP.."&field4="..LR_LIGHT_LEVEL.."&field5="..BR_LIGHT_LEVEL.."", "run=run") ````

The service has a limited functionality to manipulate your data but the base configuration is easy to tweak and can be made public or private.  See [here for my sample data](https://thingspeak.com/channels/25629).

...plus it is free to use.

[samppa]

@korttoma

Now plot.ly is back up again.
Apparently they upgraded the infrastructure and broke something. Hopefully that does not happen too often..

[korttoma]

Yes, now it is working again. Maybe you should try again @Dwalt ?

[Dwalt]

@korttoma
I noticed it back online right after I posted my Thingspeak scene. I had not deleted my Plot.ly scenes and they continued to send http requests during the outage and when Plot.ly came back online, my data resumed plotting.

I have both services running right now plotting similar data and I will leave them both running to test reliability and quirks. It seems the time reporting drifts a little in Plot.ly. I am also going to test Beebotte.

[korttoma]

There was an outage again last night. No data uploaded for 3h. Was it just me or plot.ly?

[Dwalt]

@korttoma I got it too.

[derrij]

This post is deleted!

[korttoma]

Did plot.ly change something or did I brake my own ploting somehow? Anyone else having problems?

[Dwalt]

@korttoma

My plots are still working.

[korttoma]

@Dwalt would you mind charing your lua? Remember to remove your username and api key

Here is a nexample of what I'm using and it stoped working on 9th of this month.

local OutTemp = luup.variable_get("urn:upnp-org:serviceId:TemperatureSensor1","CurrentTemperature", 229)
local TankTopp = luup.variable_get("urn:upnp-org:serviceId:TemperatureSensor1","CurrentTemperature", 174)
local TankMitten = luup.variable_get("urn:upnp-org:serviceId:TemperatureSensor1","CurrentTemperature", 179)
local TankBotten = luup.variable_get("urn:upnp-org:serviceId:TemperatureSensor1","CurrentTemperature", 175)
local Solpanel = luup.variable_get("urn:upnp-org:serviceId:TemperatureSensor1","CurrentTemperature", 28)
local Solpump = luup.variable_get("urn:micasaverde-com:serviceId:SecuritySensor1", "Tripped", 212) * 50
local CirkPump = luup.variable_get("urn:micasaverde-com:serviceId:EnergyMetering1", "Watts", 98)


local ts = os.date("%Y-%m-%d %X")

local http = require("socket.http")
http.TIMEOUT = 8
result, status = http.request("https://plot.ly/clientresp", 'un=useruser&key=keykeyszx&origin=plot&platform=lisp&args=[{"x":["'..ts..'"],"y":['..OutTemp..'],"name":"Ute Temp"},{"x":["'..ts..'"],"y":['..TankTopp..'],"name":"Tank Topp"},{"x":["'..ts..'"],"y":['..TankMitten..'],"name":"Tank Mitten"},{"x":["'..ts..'"],"y":['..TankBotten..'],"name":"Tank Botten"},{"x":["'..ts..'"],"y":['..Solpanel..'],"name":"Solpanel Temp"},{"x":["'..ts..'"],"y":['..Solpump..'],"name":"Sol Pump"},{"x":["'..ts..'"],"y":['..CirkPump..'],"name":"Cirkulationspump"}]&kwargs={"filename":"Varme","fileopt":"extend","style":{"type":"scatter"},"traces":[0,1,2,3,4,5,6],"layout":{"title":"Värme Systemet"},"world_readable":false}')```

I get 2 values correct to the graph but after this it somehow screws up the time axis of the graph.

[Dwalt]

@korttoma

Here is my code.

This only tracks light level from one sensor. I did have a data outage that lasted 3 days last month (5/24 - 5/27) and I did not notice it at the time so I am unable to determine exact cause. It resumed plotting after 3 days and I assume the error was on plotly's side. Plotly had some 'offline' issues last year as well.

local Light_level = luup.variable_get("urn:micasaverde-com:serviceId:LightSensor1", "CurrentLevel", 47)
local http = require("socket.http")
http.TIMEOUT = 5
local ts = os.date("%Y-%m-%d %X")
result, status = http.request("https://plot.ly/clientresp",'un=XXXXX&key=XXXXX&origin=plot&platform=lisp&args=[{"x":["'..ts..'"],"y":['..Light_level..']}]&kwargs={"filename":"MHLux","fileopt":"extend","style":{"type":"scatter"},"traces":[0],"layout":{"title":"MHLux_Test"},"world_readable":false}')```

[korttoma]

Bingo. You only have one data source. That I can get to work also. I had over 10 different data in one graph and it was working fine but it is not anymore.

[5546dug]

@samppa just wondering what the numbers at the end of the luup code refers to is 83. 347 etc?
I am reading and what to try to graph my temp/hum.

[Dwalt]

@5546dug

The numbers are the device id numbers within Vera. They tell the 'Get' command exactly which sensors/devices to pull the data from. They can be found under the 'Advanced' tab under each devices' 'settings'.

[korttoma]

I gave up on plot.ly. I can no longer send many data to one graph using their API so I started looking in to ThingSpeak.com that @Dwalt mentioned. I seems like you can use plugins to create graphs with multiple series, seems to work nicely.

[BulldogLowell]

@korttoma

maybe try the new Maker channel on IFTTT and send data to google spreadsheet.

then just feed your plotting software (or excel) from the data on your google drive.

[NeverDie]

@korttoma said:

I gave up on plot.ly. I can no longer send many data to one graph using their API so I started looking in to ThingSpeak.com that @Dwalt mentioned. I seems like you can use plugins to create graphs with multiple series, seems to work nicely.

Did you get it to work?

[korttoma]

@NeverDie ThingSpeak is workin but not plot.ly

[NeverDie]

Here Plot.Ly has been working:
https://plot.ly/~WhiteRabbit/997

About 6 months or so ago it seemed to stop plotting, and it wouldn't resume even if I rebooted the Arduino. So, I requested all new stream tokens, plugged those into the Arduino program, rebooted, and it has been working since. What I like about Plot.ly is that I can both pan and zoom the data.

I'm presently driving Plot.ly using just the Arduino API. I get the impression I'd have finer grained control if I were to drive it using Python instead.

[korttoma]

@NeverDie Thanks for posting, I created a new API key and deleted my old plot and so far so good, it is working again.
Like you I also prefer the UI and plots produced by plot.ly but I'm not convinced by the reliability so far.

[Hoffan]

Hi Guys

How much work is it in Plot.ly to get in to work

Becuase i test to Copy the examples upper here and put my INFO in the text
But nothing happens, Must i change a lot of thing on the Plot.ly page

[NeverDie]

@korttoma said:

@NeverDie Thanks for posting, I created a new API key and deleted my old plot and so far so good, it is working again.
Like you I also prefer the UI and plots produced by plot.ly but I'm not convinced by the reliability so far.

I share your concern. It has been a bumpy ride. If it weren't for the pan and zoom UI, I would have tried switching to another free service.

[Hoffan]

Any one that can explain for me how to get plot.ly to work, I really don't get it

[korttoma]

@Hoffan if you have an APIKey and username from plot.ly all you need to do is to change the following from the first post in this thread:

("urn:micasaverde-com:serviceId:HumiditySensor1", "CurrentLevel", 83) If you don't know what this should be let me know so I can explain further.

<username>
<APIKey>
<nameForGraph>
<graphTitle>

You should make sure you remove all the < >

Also I noticed that you can only have one private graph so if you already have one the second one will not get created.
Try changing "world_readable":false to "world_readable":true

Now test your lua code from apps -> develop apps -> test loop code (lua). Do you get any errors?

[Hoffan]

@korttoma

So if i understand this right, If i have a User and a APIKey, i dont need to do
anything on plot.ly page

I will test to change false to True when i get back home tonight

[korttoma]

@Hoffan no you should not need to change anything at plot.ly your graph should just appear among "YOUR RECENT FILES" on the right when you go to "https://plot.ly/".

[Hoffan]

@korttoma
Big thanks. Now its working

[samppa]

It seems that plot.ly has new limitations to the free API in their price plan:

• Only one free private graph
• API rate limits: 50 per day or 30 per hour

Then you have to pay 20 USD/month to get unlimited access.
Damn. There goes the cheap solution.

[Nuubi]

@samppa Yeah, one another, again. Personally, started with Pachube a few years ago. Gone, a long time ago.

[korttoma]

Thanks @samppa I was wondering why my plots were not working properly, this explains it.

[samppa]

I converted my code to use ThingSpeak.
It is even simpler to call since the formatting is done on the website. Here is how I use it:

local url = "https://api.thingspeak.com/update?api_key=_YOUR_KEY_HERE_"
result, status = http.request(url,'field1='..HotWaterTemp..'&field2='..HotWaterPump..'&field3='..HotWaterLimit..'&field4='..PumpTime)

On the https://thingspeak.com website you just define the "channel" that receives all the data and copy the example "plugin" and modify it to render your graph.
Quite easy. And free at least so far..

[korttoma]

I converted some to ThingSpeak now also but I could not get it to work using your example @samppa ,don't know why but this works for me:

local http = require("socket.http")

result, status = http.request("http://api.thingspeak.com/update?key=<API_KEY>&field1="..TankTopp.."&field2="..TankMitten.."&field3="..TankBotten.."&field4="..StatTemp.."&field5="..StatMode.."&field6="..Solpanel.."&field7="..Solpump.."&field8="..CirkPump.."", "run=run")```

[MikeF]

Like others, I've (regrettably) given up on plot.ly, and gone over to ThingSpeak.

I'm using the following Python code to send values from Domoticz:

thingURL =  'https://api.thingspeak.com/update?api_key=_YOUR_KEY_HERE'
url = thingURL + "&field1=%s&field2=%s&field3=%s&field4=%s&field5=%s" \
	% (inside, target, outside, lounge, heating)
f = urllib2.urlopen(url)
f.close()

[samppa]

@korttoma The only difference I see is that you put the entire url with parameters in the first argument of the http.request call and "run=run" as the second (don't know why you have that?). I use the base url as the first argument and the field list as the second

 r, s = http.request(baseUrl, "field1=XX&field2=XX") 

Hope that fixes the issue.
Also You have to setup the "Channel" and "Plugin" in the website accordingly and name the fields there.