Sending image-data over the MySensors network.

[Oitzu]

Hello,
i'm currently working on a project that requires sending various sensor data AND and an image (jpeg), shot by a serial camera, over an nrf24 arduino network to an raspberry pi.

Does somebody ever tried to realise that with the mysensors network?

I've read the MySensors API Documentation and looked at the git repository
https://github.com/mysensors/Arduino/tree/master/NodeJsController
regarding streams, but couldn't find a clue how to do this.

[hek]

You can send binary data (without hacking in the core) by doing calling msg.set() with a buffer and length argument.

https://github.com/mysensors/Arduino/blob/master/libraries/MySensors/MyMessage.cpp#L170

But you'll have to chop up the image data in small chunks (< MAX_PAYLOAD) yourself.

[Oitzu]

Hm... i see. I thought there could be maybe a hidden functionallity to directly pipe and stream it trough the MySensors network.
But this could work as well, i just need to chomp it up and puzzle it together on the other side.

[blacey]

@Oitzu said:

Hm... i see. I thought there could be maybe a hidden functionallity to directly pipe and stream it trough the MySensors network.
But this could work as well, i just need to chomp it up and puzzle it together on the other side.

Or you could add a new streaming API to the core and create a pull-request

[Oitzu]

I poked arround a little bit in the code.
Well there seems to be a defined message type for streaming (C_STREAM) in the librarys, and a sub-message-type for pictures? (ST_IMAGE) or is "image" in this context maybe a firmware-image?

Maybe we should first agree to a method how to handle stuff like this?
I do not think i have the right knowledge about this project to handle this all by myself.

[Oitzu]

I now got finally the time and hardware to actually test this.
It works fine with msg.set() with a payload of 8 Byte.
On the other end of the communication i read the output, of a mysensors serial gateway, with a little perl script, parsed the payload out and wrote it into a file.
The perl script got extemly slow with high cpu load over time, at this moment i don't know why.

Sending the message-header each time seems a little bit overhead to me, but if i see this right the firmware-update does basicly the same?

[hek]

Yes, most of the message header is needed to route packets, which is needed when transferring firmware or image data also.

But I agree, it is not optimal utilisation of the header when using STREAM and should probably be handled in a more elegant way in the future.

[Oitzu]

Just if anyone is interested in sending image-data over the mysensors network:
I now have some sort of working proof on concept.
I'm able to take a picture with a serial camera connected to a arduino nano and transmit it over my MySensors network.
The transmitting of the taken picture takes about 40 seconds, has a resolution of 640x320 and a size of 50kb.

[hek]

Cool! You should post parts needed and example code

[Oitzu]

The Perl-Script i'm using instead of a controller, because obviously no controller supports this yet.

#!/usr/bin/perl
use Device::SerialPort;
use POSIX qw(strftime);
use Data::Dumper;
use Time::HiRes qw (sleep);
use Time::HiRes qw(time);
my $port = Device::SerialPort->new("/dev/ttyUSB1");
$| = 1;
use Time::HiRes qw(time);

$port->baudrate(115200);
$port->databits(8);
$port->parity("none");
$port->stopbits(1);

my $buffer;
my %transfers;

print "Waiting for incoming transfers...\n";

while (1) {
	my ($count,$saw)=$port->read(256);
	if($count > 0 || length($buffer) > 0)
	{
		$buffer .= $saw;
		my ($cmd, $rest) = split(/\n/, $buffer, 2);
		if(defined($rest))
		{
			if(@m = $cmd =~ /([0-9]*);([0-9]*);([0-9]*);([0-9]*);([0-9]*);(.*)/g)
			{
				my ($node_id, $child_sensor_id, $message_type, $ack, $sub_type, $payload) = @m;
		#		print "node_id: $node_id, child_sensor_id: $child_sensor_id, message_type: $child_sensor_id, ack: $ack, sub_type: $sub_type, payload: $payload \n";
				if($sub_type eq "25")
				{
					if($payload eq "START" && $transfers{$node_id}{'state'} == 0)
					{
						print "New transfer incoming from node $node_id.\n";
						$transfers{$node_id}{'start_time'} = time();
						$transfers{$node_id}{'state'} = 1;
						$transfers{$node_id}{'count'} = 0;
						my $time = strftime("%Y-%m-%d-%H-%M-%S", localtime);
						open ($transfers{$node_id}{'FH'}, ">>node-".$node_id."-".$time.".jpg");
						binmode($transfers{$node_id}{'FH'});
					}
					
					if($payload eq "END" && $transfers{$node_id}{'state'} == 1)
					{
						close($transfers{$node_id}{'FH'});
						$transfers{$node_id}{'state'} = 0;
						my $duration = time() - $transfers{$node_id}{'start_time'};
						my $bps = $transfers{$node_id}{'count'} * 24 / $duration;
						my $pps = $transfers{$node_id}{'count'} / $duration;
						print "Transfer from $node_id finished.\n";
						print "$duration seconds, $bps Bytes/s, $pps Packets/s\n";
					}
				}
				elsif($sub_type eq "24")
				{
					if($transfers{$node_id}{'state'} == 1)
					{
						$transfers{$node_id}{'count'}++;
						print { $transfers{$node_id}{'FH'} } pack('H*',$payload);
					}
				}
			}
			$buffer = $rest;
		}
	}
        #Check for timeout
        while(($node_id) = each %transfers)
        {
	        my $duration = time() - $transfers{$node_id}{'start_time'};
               	if($duration > 120 && $transfers{$node_id}{'state'} == 1)
                {
                	close($transfers{$node_id}{'FH'});
                        $transfers{$node_id}{'state'} = 0;
                        print "Warning: Timeout from node $node_id. Transfer cancelled.\n";
                }
	}
	sleep (0.001);
}

The sketch:

#include <MySensor.h>
#include <SPI.h>
#include <Adafruit_VC0706.h>
#include <SoftwareSerial.h>

#undef DEBUG

SoftwareSerial cameraConnection = SoftwareSerial(6,7);

Adafruit_VC0706 cam = Adafruit_VC0706(&cameraConnection);

#define CHILD_ID 3

MySensor gw;
MyMessage msg(CHILD_ID,V_VAR1);
MyMessage msg2(CHILD_ID,V_VAR2);

void setup()  
{  
  gw.begin();

  gw.present(CHILD_ID, S_CUSTOM);  
  
  if (cam.begin()){}
  else { return; } //Abort the transfer if camera does not initialize

  cam.setImageSize(VC0706_640x480);
  delay(3000);
  snapAndSend();
  cam.reset();
}

//Do nothing.
void loop() 
{

}

void snapAndSend()
{
  cam.takePicture();
  uint16_t jpgLen = cam.frameLength();
  Serial.print("Sending Picture of ");
  Serial.print(jpgLen, DEC);
  Serial.println(" Bytes.");
  gw.send(msg2.set("START"));
  while (jpgLen > 0)
  {                     //Send off 24 bytes of data at a time
    uint8_t *buffer;
    uint8_t bytesToRead = min(24, jpgLen);
    buffer = cam.readPicture(bytesToRead);
    gw.send(msg.set(buffer, bytesToRead));
    jpgLen -= bytesToRead;
  }
  Serial.println("Done.");
  gw.send(msg2.set("END"));
}

Hardware:
Typical MySensors-Node and a Serial TTL Camera like this:
http://aliexpress.com/item/NEW-RS232-TTL-JPEG-Digital-Serial-Port-CCTV-Camera-Module-SCB-1-with-video-out-Support/1975852463.html

[hek]

Awesome! Really simple and clean example (and perl script!).

[Oitzu]

Thanks.
There was much try and error involved to get this running.
But i'm still looking forward for suggestions/discussion about the implementation of a STREAM feature.

What do you guys think about a fixed routing while streaming?
The init of the stream could be started by a special payload, when the gateway receives this message it generates a "stream-id". It would link this stream-id to the node-id and sensor-id and send the stream-id back to the sensor. Repeaters in the middle of the way, to the node, would also remember the link betweend stream-id and node-id.
After this point the node would begin to stream its content only with the stream-id assigned, which would free up to 30bytes per package.

The gateway itself could reverse this process and send out "normal" looking "commands".

The stream-id will be freed at the moment the stream is completed or a timeout occurs.

---

This ist just a suggestion by me.

[hek]

Yes, would be nice to free up some payload space for this type of data.

Just a bit worried about allocating things (dynamically or statically) for the stream information (beside the extra code). Pretty cramped already.

What happens if you just increase the RF24 datarate in MyConfig? Do you lose much data?

[tekka]

@Oitzu nice, I'll think about implementing this in MYSController for testing purposes.

[Oitzu]

@hek said:

Yes, would be nice to free up some payload space for this type of data.

Just a bit worried about allocating things (dynamically or statically) for the stream information (beside the extra code). Pretty cramped already.

I think by deactivating not used features with pre-compile directives this problem could be solved. Need to look into this again. But i remember there are large chunks of unused code compiled in if you, by example, only want to use the lib for a "pure" node.

What happens if you just increase the RF24 datarate in MyConfig? Do you lose much data?

Didn't tried yet. The target for this application is a long range communication. So i keeped the datarate low in my tests.
BUT i need to note: Most of the 40 seconds, at this moment, is probably a result of the slow UART connection to the camera. The camera-chip has spi but it isn't breaked out on the board. There are boards out there with the same chip + SPI! Users reported the transfer over SPI to the MCU takes about 1 second!

@tekka nice to hear. Let me know if i can support somehow.

[Oitzu]

Update!
I changed the camera to a faster SPI model.
http://de.aliexpress.com/item/ArduCAM-M-2MP-Camera-Shield-OV2640-2MP-SPI-Camera-module/32307177889.html?ws_ab_test=201407_5,201444_5,201409_3

The camera delivers a 2MP picture in under 1 seconds over SPI.

But i'm running in a dead end, with MySensors not supporting proper streams.
A 2MP (100kb) takes about 20 seconds with MySensors.
Main problem atm is IMHO with the mysensors transmitting style:

	rf24.powerUp();
	rf24.stopListening();
	rf24.openWritingPipe(TO_ADDR(to));
	bool ok = rf24.write(data, len, to == BROADCAST_ADDRESS);
	rf24.startListening();
	return ok;

would be more effective to do something like:

	rf24.powerUp();
	rf24.stopListening();
	rf24.openWritingPipe(TO_ADDR(to));
///loop here over more than 1 packet

rf24.writeFast(data, len, to == BROADCAST_ADDRESS);

//end of loop
	rf24.startListening();
	return ok;

The stopListening -> openWritingPipe -> startListening just takes to much time.

Unfortunately i failed a try to implement this... it seems i'm just not experienced enough.

[thomas1412]

Hi,

I'm currently trying to use your solution, @Oitzu.
The Node-part is working fine. Have a little problem on the Pi-side.
The RF-device is directly connected to the Pi, so I'm using a Serial Gateway script on the Pi (https://github.com/mysensors/Raspberry). The serial data is then send over /dev/ttyUSB20, that is working fine.
I know that I need to adjust the following lines of the Perl script:

use Device::SerialPort;
use POSIX qw(strftime);
use Data::Dumper;
use Time::HiRes qw (sleep);
use Time::HiRes qw(time);
my $port = Device::SerialPort->new("/dev/ttyUSB1");

But it seems I'm just to stupid to do this by myself
I would suggest that I can delete the first line and change the last line to something like this:

my $port = "/dev/ttyUSB20";

Can someone help me with the right syntax?

Thanks!

[Oitzu]

@thomas1412 said:

Hi,

I'm currently trying to use your solution, @Oitzu.
The RF-device is directly connected to the Pi, so I'm using a Serial Gateway script on the Pi (https://github.com/mysensors/Raspberry). The serial data is then send over /dev/ttyUSB20, that is working fine.
I know that I need to adjust the following lines of the Perl script:

use Device::SerialPort;
use POSIX qw(strftime);
use Data::Dumper;
use Time::HiRes qw (sleep);
use Time::HiRes qw(time);
my $port = Device::SerialPort->new("/dev/ttyUSB1");

But it seems I'm just to stupid to do this by myself
I would suggest that I can delete the first line and change the last line to something like this:

my $port = "/dev/ttyUSB20";

Can someone help me with the right syntax?

The RPi Serial Gateway creates a symlink "/dev/ttyMySensorsGateway" you should use that as Device for more consistent results.

I srsly don't know why you would delete the first line. I'm pretty sure you need to use the Device::Serialport Perl-Module.

Also try to run the script as root. The created device files tend to have root-only access rights.

[thomas1412]

Hi,

thanks for the quick reply and your help
It turned out my problem was the not installed serialport perl library.. Now I'm a step further.
Unfortunately there's another problem now..
After sending three or four packages the next six packages are failing to send. And this over and over again.
Seems that the SerialGateway is too slow in some way..

send: 254-254-0-0 s=255,c=0,t=17,pt=0,l=5,sg=0,st=ok:1.5.1
send: 254-254-0-0 s=255,c=3,t=6,pt=1,l=1,sg=0,st=ok:0
sensor started, id=254, parent=0, distance=1
send: 254-254-0-0 s=3,c=0,t=23,pt=0,l=0,sg=0,st=ok:
Sending Picture of 13048 Bytes.
send: 254-254-0-0 s=3,c=1,t=25,pt=0,l=5,sg=0,st=ok:START
send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=ok:FFD8FFFE00244900020D0000000000000000000000000000
send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=ok:00F0004001070032120B510451040000FFDB008400020101
send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=ok:020101020201020202020203040303020203050404030406
send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=fail:0607070606060607080A0907070A080606090C090A0A0B0B
send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=fail:0B0B07080C0D0C0B0D0A0B0B0B0102020203020305030305
send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=fail:0B0706070B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B
send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=fail:0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B
send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=fail:0B0B0B0B0B0BFFFE0005000000FFC000110800F001400301
send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=fail:2100021101031101FFC401A2000001050101010101010000```

[hek]

Might be a powering issue of the radio. Put a bigger cap on it.

[thomas1412]

Adding a capacitor hasn't helped.
I changed the setup using now a serial gateway on a separate Arduino instead of having it directly on the Pi. This is working fine. So it seems that the serial gateway on the Pi was the problem..

[Oitzu]

@thomas1412 said:

This is working fine. So it seems that the serial gateway on the Pi was the problem..

I tested this, and yes, the Pi serial gateway seems to be the problem, clogging up on some point and stops.
The problem itself seems to be in the pseudo terminal created. If i rewrite the gateway that the data with type 24 (image data) is not written to the pseudo terminal the sending works fine.

The Pi serial gateway has atm afaik no maintainer and needs some love.

[Zied BEN HAMMOUD]

Hello, this is my first arduino project: sending a file (jpeg, pdf, txt) from an arduino sd card module to android phone using arduino uno and bluetooth module HC-05
It's possible with this hardware?

If it's possible can someone please send me the source code and android app use it to receive the file
thank you

[hek]

@Zied-BEN-HAMMOUD

I doubt anyone has the sourcecode ready for an app/sketch shuffling data over bluetooth. The MySensors library doesn't handle bluetooth communication so you should probably look elsewhere.

[dimi59]

Hello,
It’s possible to use à esp32-cam and add a module for transmit one photo ?
Thanks a lot

[rejoe2]

I've been looking around a bit, but beside the example Perl script provided by @thomas1412 and the STREAM type usage in the OTA feature, there seems not to exist any working solution using really the stream type message structure. But this is more or less receiving only on the node side afai understood.

As I'm about to do some renovation work on the FHEM integration (also written in Perl), I'd be interested in some guidelines about the best way to transfer e.g. jpeg data as stream from node side.
The sample sketch @thomas1412 provided uses not stream type, but V_VAR1 (data) and V_VAR2 (signalizing start and end).

I'd be happy if someone could provide a sample sketch using the newer (?) stream type sending style or a link to the relevant parts in the MySensors codebase...

[Guillermo Schimmel]

@Oitzu Great work. Looking forward to use it. I have a use for this (some remote roof drain that gets clogged with leaves). Thanks!