Graphing sensor data

[pete1450]

This is still in very early stages but I wanted to share what I've got up and running.

I don't even have a controller yet but I've been meaning to use the nrf24l01+ for quite some time and finding this site with everything worked out gave me the push to pull them out of the parts bin. I'm a data hog who likes to have a record of everything so I figured I'd get a logging server set up for when I actually get the other sensors going(at minimum temp/PIR for every room).
For proof-of-concept I pulled an ultrasonic sensor out and attached it to a pro mini with the distance sketch. A nano was attached to another nrf24l01+ and sent an id to the sensor so it would quit whining. At that point I could see data coming across and moved on to what this post is really about.
The pogoplug line of devices are meant to be an easy way to make your own cloud storage by plugging a USB drive into it and then the Pogoplug into your router. Whats great is that underneath they're just an Arm processor running stripped down linux. That means they can be reloaded with other versions of linux and run as a low power server.

The best part? You can find them for $6 on sale.

I'm not going to go into all the detail of getting everything set up; Google can help you get the pieces working together. I'll just give the high level overview. If you get stuck, a search for 'ArchLinux thing-im-trying-to-install' will get you to a wiki page with the exact commands you need to run.

• I went with this guide for getting ArchLinux running. Be aware that you need to find a guide specific to your pogoplug model/processor. I happene to be using the Mobile which has only one USB port so followed the directions in that guide to run the OS off a SD card.

• I like Perl so the libraries I used in the code were Device::SerialPort, DBD:SQLite, and GD::Graph. One or two failed on install through cpanminus if I recall because of a lack of memory(Pogoplug mobile only has 128mb). Creating a swap file took care of that.

• CGI is a quick way to make dynamic content and using a scripting language like Perl lets you just "print" whatever you want to a page. You can use things like

  <img src="/makegraph.cgi?">
  in a html page to place the output of your script(in this case your graph) there. Multiple lines would put multiple graphs on one page.

• Database used is sqlite

• Web server is lighttpd

Code to grab data off the gateway and store it. This runs continuously in the background.

use Device::SerialPort;
use DBI;
my $port = Device::SerialPort->new("/dev/ttyUSB0") or die $!;
my $dbh = DBI->connect(          
	"dbi:SQLite:dbname=/srv/http/distancetest", 
	"",
	"",
	{ RaiseError => 1}
) or die $DBI::errstr;
 
# 19200, 81N on the USB ftdi driver
$port->baudrate(115200); # you may change this value
$port->databits(8); # but not this and the two following
$port->parity("none");
$port->stopbits(1);
 
#get rid of some junk data
my $tEnd = time()+2; # 2 seconds in future
while (time()< $tEnd) {
  my $c = $port->lookfor();
  next if $c eq "";
  last;
}
while (1) {
	# Poll to see if any data is coming in
	my $char = $port->lookfor();
 
	# If we get data, then print it
	if ($char) {
		print "$char\n";
		if($char =~ /1;1;1;0;13/){
			@tmparr = split(/;/, $char);
			$tmpval = $tmparr[5];
			print "$tmpval\n";
			if($tmpval !~ //){ #sometimes a blank value comes in
				$dbh->do("INSERT INTO distance(distance) VALUES(" . $tmpval . ")") or die $!;
			}
		}
	}
	# Uncomment the following lines, for slower reading,
	# but lower CPU usage, and to avoid
	# buffer overflow due to sleep function.
 
	# $port->lookclear;
	# sleep (1);
}

This is the cgi script that generates the graphs. Ideally there would be fields to fill in that are passed to the script for determining ranges... later.

#!/usr/bin/perl -w
use CGI ':standard';
use DBI;
use GD::Graph::lines;
use strict;
#use CGI::Carp qw(fatalsToBrowser);

my $dbh = DBI->connect(          
	"dbi:SQLite:dbname=/srv/http/distancetest", 
	"",
	"",
	{ RaiseError => 1}
) or die $!;

my $sth = $dbh->prepare("SELECT * FROM distance");
$sth->execute() or die "Cannot execute: " . $sth->errstr(  );

my $row;
my @data;
my @out1;
my @out2;

while ($row = $sth->fetchrow_arrayref()) {
	push(@out1, @$row[0]);
	push(@out2, @$row[1]);
}
#@out1 = (1,2,3,4,5);
#@out2 = (1,2,3,4,5);

$data[0] = \@out1;
$data[1] = \@out2;

$sth->finish();
$dbh->disconnect();

my $skip = int ((@data * 50)/(600-50) + 1);

my $mygraph = GD::Graph::lines->new(600, 300);
$mygraph->set(
	x_label     => 'Timestamp',
	y_label     => 'cm',
	title       => 'Distance measuted at time',
	line_types  => [1],
	line_width  => 2,
	x_labels_vertical => 1,
	x_label_skip => 10,
	dclrs       => ['blue'],
) or warn $mygraph->error;

$mygraph->set_legend_font(GD::gdMediumBoldFont);
$mygraph->set_legend('sensor1');
my $myimage = $mygraph->plot(\@data) or die $mygraph->error;

print "Content-type: image/png\n\n";
#open(OUTPUT, ">$0.png") or die "Can't open $0.png: $!\n";
#print OUTPUT $myimage->png();
print $myimage->png;

And below is a graph from data that the sensor has reported today. The x axis needs work because that range is just a set spacing for each event; it's not representative of time.

Once I get a controller(hinted strongly to my wife for a Christmas Veralite), the nano will attach to that and I'll update all sensors to provide data in response to a request. A separate pro mini will be attached to the Pogoplug and send requests at set times for things like temperature. It will also recieve store-data requests for any commands a Vera would send to a sensor/control or new states something like a door sensor would have, I got this up in a bout a day of on and off work(when I should be wrapping Christmas presents) so anyone more linux savy should be able to knock this out quick. Anyone less: again, Google is your friend. It was a lot of copy/paste on my part anyway.

[pete1450]

Saving this space for updates

[tbowmo]

@pete1450

How do you plan to power the sensors? If they should be battery powered, you won't get long operation time if you are going to use the polling scheme for getting sensors, because the radios need to be powered up in order to receive the pull requests

I would let the sensors send in its data at a regular interval instead, without polling. This way you can put the radio to sleep when not needed, and save some of that precious energy in the batteries.

Of course, this is just my 2 cents

[pete1450]

No, youre right, that wouldnt work for battery powered. I chop the ends off old cell phone chargers and try to keep a room's sensors down to a single box. The only one I've considered being battery powered is the "Mailbox opened" one but thats another project. The polling is really just to allow getting realtime data when you want it. If I think about it more, Ill probably leave it out. I was just thinking out loud near the end.