Sending image-data over the MySensors network.

Oitzu

Hello,
i'm currently working on a project that requires sending various sensor data AND and an image (jpeg), shot by a serial camera, over an nrf24 arduino network to an raspberry pi.

Does somebody ever tried to realise that with the mysensors network?

I've read the MySensors API Documentation and looked at the git repository
https://github.com/mysensors/Arduino/tree/master/NodeJsController
regarding streams, but couldn't find a clue how to do this.

hek

You can send binary data (without hacking in the core) by doing calling msg.set() with a buffer and length argument.

https://github.com/mysensors/Arduino/blob/master/libraries/MySensors/MyMessage.cpp#L170

But you'll have to chop up the image data in small chunks (< MAX_PAYLOAD) yourself.

Oitzu

Hm... i see. I thought there could be maybe a hidden functionallity to directly pipe and stream it trough the MySensors network.
But this could work as well, i just need to chomp it up and puzzle it together on the other side.

blacey

@Oitzu said:

Hm... i see. I thought there could be maybe a hidden functionallity to directly pipe and stream it trough the MySensors network.
But this could work as well, i just need to chomp it up and puzzle it together on the other side.

Or you could add a new streaming API to the core and create a pull-request ;)

Oitzu

I poked arround a little bit in the code.
Well there seems to be a defined message type for streaming (C_STREAM) in the librarys, and a sub-message-type for pictures? (ST_IMAGE) or is "image" in this context maybe a firmware-image?

Maybe we should first agree to a method how to handle stuff like this?
I do not think i have the right knowledge about this project to handle this all by myself.

Oitzu

I now got finally the time and hardware to actually test this.
It works fine with msg.set() with a payload of 8 Byte.
On the other end of the communication i read the output, of a mysensors serial gateway, with a little perl script, parsed the payload out and wrote it into a file.
The perl script got extemly slow with high cpu load over time, at this moment i don't know why.

Sending the message-header each time seems a little bit overhead to me, but if i see this right the firmware-update does basicly the same?

hek

Yes, most of the message header is needed to route packets, which is needed when transferring firmware or image data also.

But I agree, it is not optimal utilisation of the header when using STREAM and should probably be handled in a more elegant way in the future.

Oitzu

Just if anyone is interested in sending image-data over the mysensors network:
I now have some sort of working proof on concept.
I'm able to take a picture with a serial camera connected to a arduino nano and transmit it over my MySensors network.
The transmitting of the taken picture takes about 40 seconds, has a resolution of 640x320 and a size of 50kb.

hek

Cool! You should post parts needed and example code :)

Oitzu

The Perl-Script i'm using instead of a controller, because obviously no controller supports this yet.

#!/usr/bin/perl
use Device::SerialPort;
use POSIX qw(strftime);
use Data::Dumper;
use Time::HiRes qw (sleep);
use Time::HiRes qw(time);
my $port = Device::SerialPort->new("/dev/ttyUSB1");
$| = 1;
use Time::HiRes qw(time);

$port->baudrate(115200);
$port->databits(8);
$port->parity("none");
$port->stopbits(1);

my $buffer;
my %transfers;

print "Waiting for incoming transfers...\n";

while (1) {
	my ($count,$saw)=$port->read(256);
	if($count > 0 || length($buffer) > 0)
	{
		$buffer .= $saw;
		my ($cmd, $rest) = split(/\n/, $buffer, 2);
		if(defined($rest))
		{
			if(@m = $cmd =~ /([0-9]*);([0-9]*);([0-9]*);([0-9]*);([0-9]*);(.*)/g)
			{
				my ($node_id, $child_sensor_id, $message_type, $ack, $sub_type, $payload) = @m;
		#		print "node_id: $node_id, child_sensor_id: $child_sensor_id, message_type: $child_sensor_id, ack: $ack, sub_type: $sub_type, payload: $payload \n";
				if($sub_type eq "25")
				{
					if($payload eq "START" && $transfers{$node_id}{'state'} == 0)
					{
						print "New transfer incoming from node $node_id.\n";
						$transfers{$node_id}{'start_time'} = time();
						$transfers{$node_id}{'state'} = 1;
						$transfers{$node_id}{'count'} = 0;
						my $time = strftime("%Y-%m-%d-%H-%M-%S", localtime);
						open ($transfers{$node_id}{'FH'}, ">>node-".$node_id."-".$time.".jpg");
						binmode($transfers{$node_id}{'FH'});
					}
					
					if($payload eq "END" && $transfers{$node_id}{'state'} == 1)
					{
						close($transfers{$node_id}{'FH'});
						$transfers{$node_id}{'state'} = 0;
						my $duration = time() - $transfers{$node_id}{'start_time'};
						my $bps = $transfers{$node_id}{'count'} * 24 / $duration;
						my $pps = $transfers{$node_id}{'count'} / $duration;
						print "Transfer from $node_id finished.\n";
						print "$duration seconds, $bps Bytes/s, $pps Packets/s\n";
					}
				}
				elsif($sub_type eq "24")
				{
					if($transfers{$node_id}{'state'} == 1)
					{
						$transfers{$node_id}{'count'}++;
						print { $transfers{$node_id}{'FH'} } pack('H*',$payload);
					}
				}
			}
			$buffer = $rest;
		}
	}
        #Check for timeout
        while(($node_id) = each %transfers)
        {
	        my $duration = time() - $transfers{$node_id}{'start_time'};
               	if($duration > 120 && $transfers{$node_id}{'state'} == 1)
                {
                	close($transfers{$node_id}{'FH'});
                        $transfers{$node_id}{'state'} = 0;
                        print "Warning: Timeout from node $node_id. Transfer cancelled.\n";
                }
	}
	sleep (0.001);
}

The sketch:

#include <MySensor.h>
#include <SPI.h>
#include <Adafruit_VC0706.h>
#include <SoftwareSerial.h>

#undef DEBUG

SoftwareSerial cameraConnection = SoftwareSerial(6,7);

Adafruit_VC0706 cam = Adafruit_VC0706(&cameraConnection);

#define CHILD_ID 3

MySensor gw;
MyMessage msg(CHILD_ID,V_VAR1);
MyMessage msg2(CHILD_ID,V_VAR2);

void setup()  
{  
  gw.begin();

  gw.present(CHILD_ID, S_CUSTOM);  
  
  if (cam.begin()){}
  else { return; } //Abort the transfer if camera does not initialize

  cam.setImageSize(VC0706_640x480);
  delay(3000);
  snapAndSend();
  cam.reset();
}

//Do nothing.
void loop() 
{

}

void snapAndSend()
{
  cam.takePicture();
  uint16_t jpgLen = cam.frameLength();
  Serial.print("Sending Picture of ");
  Serial.print(jpgLen, DEC);
  Serial.println(" Bytes.");
  gw.send(msg2.set("START"));
  while (jpgLen > 0)
  {                     //Send off 24 bytes of data at a time
    uint8_t *buffer;
    uint8_t bytesToRead = min(24, jpgLen);
    buffer = cam.readPicture(bytesToRead);
    gw.send(msg.set(buffer, bytesToRead));
    jpgLen -= bytesToRead;
  }
  Serial.println("Done.");
  gw.send(msg2.set("END"));
}

Hardware:
Typical MySensors-Node and a Serial TTL Camera like this:
http://aliexpress.com/item/NEW-RS232-TTL-JPEG-Digital-Serial-Port-CCTV-Camera-Module-SCB-1-with-video-out-Support/1975852463.html

2015-07-30 19.38.29 - Kopie.jpg

hek

Awesome! Really simple and clean example (and perl script!).

Oitzu

Thanks. :)
There was much try and error involved to get this running.
But i'm still looking forward for suggestions/discussion about the implementation of a STREAM feature.

What do you guys think about a fixed routing while streaming?
The init of the stream could be started by a special payload, when the gateway receives this message it generates a "stream-id". It would link this stream-id to the node-id and sensor-id and send the stream-id back to the sensor. Repeaters in the middle of the way, to the node, would also remember the link betweend stream-id and node-id.
After this point the node would begin to stream its content only with the stream-id assigned, which would free up to 30bytes per package.

The gateway itself could reverse this process and send out "normal" looking "commands".

The stream-id will be freed at the moment the stream is completed or a timeout occurs.

This ist just a suggestion by me.

hek

Yes, would be nice to free up some payload space for this type of data.

Just a bit worried about allocating things (dynamically or statically) for the stream information (beside the extra code). Pretty cramped already.

What happens if you just increase the RF24 datarate in MyConfig? Do you lose much data?

tekka

@Oitzu nice, I'll think about implementing this in MYSController for testing purposes.

Oitzu

@hek said:

Yes, would be nice to free up some payload space for this type of data.

Just a bit worried about allocating things (dynamically or statically) for the stream information (beside the extra code). Pretty cramped already.

I think by deactivating not used features with pre-compile directives this problem could be solved. Need to look into this again. But i remember there are large chunks of unused code compiled in if you, by example, only want to use the lib for a "pure" node.

What happens if you just increase the RF24 datarate in MyConfig? Do you lose much data?

Didn't tried yet. The target for this application is a long range communication. So i keeped the datarate low in my tests.
BUT i need to note: Most of the 40 seconds, at this moment, is probably a result of the slow UART connection to the camera. The camera-chip has spi but it isn't breaked out on the board. There are boards out there with the same chip + SPI! Users reported the transfer over SPI to the MCU takes about 1 second!

@tekka nice to hear. :) Let me know if i can support somehow.

Oitzu

Update!
I changed the camera to a faster SPI model.
http://de.aliexpress.com/item/ArduCAM-M-2MP-Camera-Shield-OV2640-2MP-SPI-Camera-module/32307177889.html?ws_ab_test=201407_5,201444_5,201409_3

The camera delivers a 2MP picture in under 1 seconds over SPI.

But i'm running in a dead end, with MySensors not supporting proper streams.
A 2MP (100kb) takes about 20 seconds with MySensors.
Main problem atm is IMHO with the mysensors transmitting style:

	rf24.powerUp();
	rf24.stopListening();
	rf24.openWritingPipe(TO_ADDR(to));
	bool ok = rf24.write(data, len, to == BROADCAST_ADDRESS);
	rf24.startListening();
	return ok;

would be more effective to do something like:

	rf24.powerUp();
	rf24.stopListening();
	rf24.openWritingPipe(TO_ADDR(to));
///loop here over more than 1 packet

rf24.writeFast(data, len, to == BROADCAST_ADDRESS);

//end of loop
	rf24.startListening();
	return ok;

The stopListening -> openWritingPipe -> startListening just takes to much time.

Unfortunately i failed a try to implement this... it seems i'm just not experienced enough.

thomas1412

Hi,

I'm currently trying to use your solution, @Oitzu.
The Node-part is working fine. Have a little problem on the Pi-side.
The RF-device is directly connected to the Pi, so I'm using a Serial Gateway script on the Pi (https://github.com/mysensors/Raspberry). The serial data is then send over /dev/ttyUSB20, that is working fine.
I know that I need to adjust the following lines of the Perl script:

use Device::SerialPort;
use POSIX qw(strftime);
use Data::Dumper;
use Time::HiRes qw (sleep);
use Time::HiRes qw(time);
my $port = Device::SerialPort->new("/dev/ttyUSB1");

But it seems I'm just to stupid to do this by myself :grin:
I would suggest that I can delete the first line and change the last line to something like this:

my $port = "/dev/ttyUSB20";

Can someone help me with the right syntax?

Thanks!

Oitzu

@thomas1412 said:

Hi,

I'm currently trying to use your solution, @Oitzu.
The RF-device is directly connected to the Pi, so I'm using a Serial Gateway script on the Pi (https://github.com/mysensors/Raspberry). The serial data is then send over /dev/ttyUSB20, that is working fine.
I know that I need to adjust the following lines of the Perl script:
use Device::SerialPort;
use POSIX qw(strftime);
use Data::Dumper;
use Time::HiRes qw (sleep);
use Time::HiRes qw(time);
my $port = Device::SerialPort->new("/dev/ttyUSB1");
But it seems I'm just to stupid to do this by myself :grin:
I would suggest that I can delete the first line and change the last line to something like this:
my $port = "/dev/ttyUSB20";
Can someone help me with the right syntax?

The RPi Serial Gateway creates a symlink "/dev/ttyMySensorsGateway" you should use that as Device for more consistent results.

I srsly don't know why you would delete the first line. I'm pretty sure you need to use the Device::Serialport Perl-Module. ;)

Also try to run the script as root. The created device files tend to have root-only access rights.

thomas1412

Hi,

thanks for the quick reply and your help
It turned out my problem was the not installed serialport perl library.. Now I'm a step further.
Unfortunately there's another problem now.. :smile:
After sending three or four packages the next six packages are failing to send. And this over and over again.
Seems that the SerialGateway is too slow in some way..

send: 254-254-0-0 s=255,c=0,t=17,pt=0,l=5,sg=0,st=ok:1.5.1
send: 254-254-0-0 s=255,c=3,t=6,pt=1,l=1,sg=0,st=ok:0
sensor started, id=254, parent=0, distance=1
send: 254-254-0-0 s=3,c=0,t=23,pt=0,l=0,sg=0,st=ok:
Sending Picture of 13048 Bytes.
send: 254-254-0-0 s=3,c=1,t=25,pt=0,l=5,sg=0,st=ok:START
send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=ok:FFD8FFFE00244900020D0000000000000000000000000000
send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=ok:00F0004001070032120B510451040000FFDB008400020101
send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=ok:020101020201020202020203040303020203050404030406
send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=fail:0607070606060607080A0907070A080606090C090A0A0B0B
send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=fail:0B0B07080C0D0C0B0D0A0B0B0B0102020203020305030305
send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=fail:0B0706070B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B
send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=fail:0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B
send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=fail:0B0B0B0B0B0BFFFE0005000000FFC000110800F001400301
send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=fail:2100021101031101FFC401A2000001050101010101010000```

hek

Might be a powering issue of the radio. Put a bigger cap on it.

Sending image-data over the MySensors network.

7

11.7k

11.2k

113.1k