Sending image-data over the MySensors network.



  • Hello,
    i'm currently working on a project that requires sending various sensor data AND and an image (jpeg), shot by a serial camera, over an nrf24 arduino network to an raspberry pi.

    Does somebody ever tried to realise that with the mysensors network?

    I've read the MySensors API Documentation and looked at the git repository
    https://github.com/mysensors/Arduino/tree/master/NodeJsController
    regarding streams, but couldn't find a clue how to do this.


  • Admin

    You can send binary data (without hacking in the core) by doing calling msg.set() with a buffer and length argument.

    https://github.com/mysensors/Arduino/blob/master/libraries/MySensors/MyMessage.cpp#L170

    But you'll have to chop up the image data in small chunks (< MAX_PAYLOAD) yourself.



  • Hm... i see. I thought there could be maybe a hidden functionallity to directly pipe and stream it trough the MySensors network.
    But this could work as well, i just need to chomp it up and puzzle it together on the other side.


  • Admin

    @Oitzu said:

    Hm... i see. I thought there could be maybe a hidden functionallity to directly pipe and stream it trough the MySensors network.
    But this could work as well, i just need to chomp it up and puzzle it together on the other side.

    Or you could add a new streaming API to the core and create a pull-request ;)



  • I poked arround a little bit in the code.
    Well there seems to be a defined message type for streaming (C_STREAM) in the librarys, and a sub-message-type for pictures? (ST_IMAGE) or is "image" in this context maybe a firmware-image?

    Maybe we should first agree to a method how to handle stuff like this?
    I do not think i have the right knowledge about this project to handle this all by myself.



  • I now got finally the time and hardware to actually test this.
    It works fine with msg.set() with a payload of 8 Byte.
    On the other end of the communication i read the output, of a mysensors serial gateway, with a little perl script, parsed the payload out and wrote it into a file.
    The perl script got extemly slow with high cpu load over time, at this moment i don't know why.

    Sending the message-header each time seems a little bit overhead to me, but if i see this right the firmware-update does basicly the same?


  • Admin

    Yes, most of the message header is needed to route packets, which is needed when transferring firmware or image data also.

    But I agree, it is not optimal utilisation of the header when using STREAM and should probably be handled in a more elegant way in the future.



  • Just if anyone is interested in sending image-data over the mysensors network:
    I now have some sort of working proof on concept.
    I'm able to take a picture with a serial camera connected to a arduino nano and transmit it over my MySensors network.
    The transmitting of the taken picture takes about 40 seconds, has a resolution of 640x320 and a size of 50kb.


  • Admin

    Cool! You should post parts needed and example code :)



  • The Perl-Script i'm using instead of a controller, because obviously no controller supports this yet.

    #!/usr/bin/perl
    use Device::SerialPort;
    use POSIX qw(strftime);
    use Data::Dumper;
    use Time::HiRes qw (sleep);
    use Time::HiRes qw(time);
    my $port = Device::SerialPort->new("/dev/ttyUSB1");
    $| = 1;
    use Time::HiRes qw(time);
    
    $port->baudrate(115200);
    $port->databits(8);
    $port->parity("none");
    $port->stopbits(1);
    
    my $buffer;
    my %transfers;
    
    print "Waiting for incoming transfers...\n";
    
    while (1) {
    	my ($count,$saw)=$port->read(256);
    	if($count > 0 || length($buffer) > 0)
    	{
    		$buffer .= $saw;
    		my ($cmd, $rest) = split(/\n/, $buffer, 2);
    		if(defined($rest))
    		{
    			if(@m = $cmd =~ /([0-9]*);([0-9]*);([0-9]*);([0-9]*);([0-9]*);(.*)/g)
    			{
    				my ($node_id, $child_sensor_id, $message_type, $ack, $sub_type, $payload) = @m;
    		#		print "node_id: $node_id, child_sensor_id: $child_sensor_id, message_type: $child_sensor_id, ack: $ack, sub_type: $sub_type, payload: $payload \n";
    				if($sub_type eq "25")
    				{
    					if($payload eq "START" && $transfers{$node_id}{'state'} == 0)
    					{
    						print "New transfer incoming from node $node_id.\n";
    						$transfers{$node_id}{'start_time'} = time();
    						$transfers{$node_id}{'state'} = 1;
    						$transfers{$node_id}{'count'} = 0;
    						my $time = strftime("%Y-%m-%d-%H-%M-%S", localtime);
    						open ($transfers{$node_id}{'FH'}, ">>node-".$node_id."-".$time.".jpg");
    						binmode($transfers{$node_id}{'FH'});
    					}
    					
    					if($payload eq "END" && $transfers{$node_id}{'state'} == 1)
    					{
    						close($transfers{$node_id}{'FH'});
    						$transfers{$node_id}{'state'} = 0;
    						my $duration = time() - $transfers{$node_id}{'start_time'};
    						my $bps = $transfers{$node_id}{'count'} * 24 / $duration;
    						my $pps = $transfers{$node_id}{'count'} / $duration;
    						print "Transfer from $node_id finished.\n";
    						print "$duration seconds, $bps Bytes/s, $pps Packets/s\n";
    					}
    				}
    				elsif($sub_type eq "24")
    				{
    					if($transfers{$node_id}{'state'} == 1)
    					{
    						$transfers{$node_id}{'count'}++;
    						print { $transfers{$node_id}{'FH'} } pack('H*',$payload);
    					}
    				}
    			}
    			$buffer = $rest;
    		}
    	}
            #Check for timeout
            while(($node_id) = each %transfers)
            {
    	        my $duration = time() - $transfers{$node_id}{'start_time'};
                   	if($duration > 120 && $transfers{$node_id}{'state'} == 1)
                    {
                    	close($transfers{$node_id}{'FH'});
                            $transfers{$node_id}{'state'} = 0;
                            print "Warning: Timeout from node $node_id. Transfer cancelled.\n";
                    }
    	}
    	sleep (0.001);
    }
    

    The sketch:

    #include <MySensor.h>
    #include <SPI.h>
    #include <Adafruit_VC0706.h>
    #include <SoftwareSerial.h>
    
    #undef DEBUG
    
    SoftwareSerial cameraConnection = SoftwareSerial(6,7);
    
    Adafruit_VC0706 cam = Adafruit_VC0706(&cameraConnection);
    
    #define CHILD_ID 3
    
    MySensor gw;
    MyMessage msg(CHILD_ID,V_VAR1);
    MyMessage msg2(CHILD_ID,V_VAR2);
    
    void setup()  
    {  
      gw.begin();
    
      gw.present(CHILD_ID, S_CUSTOM);  
      
      if (cam.begin()){}
      else { return; } //Abort the transfer if camera does not initialize
    
      cam.setImageSize(VC0706_640x480);
      delay(3000);
      snapAndSend();
      cam.reset();
    }
    
    //Do nothing.
    void loop() 
    {
    
    }
    
    void snapAndSend()
    {
      cam.takePicture();
      uint16_t jpgLen = cam.frameLength();
      Serial.print("Sending Picture of ");
      Serial.print(jpgLen, DEC);
      Serial.println(" Bytes.");
      gw.send(msg2.set("START"));
      while (jpgLen > 0)
      {                     //Send off 24 bytes of data at a time
        uint8_t *buffer;
        uint8_t bytesToRead = min(24, jpgLen);
        buffer = cam.readPicture(bytesToRead);
        gw.send(msg.set(buffer, bytesToRead));
        jpgLen -= bytesToRead;
      }
      Serial.println("Done.");
      gw.send(msg2.set("END"));
    }
    

    Hardware:
    Typical MySensors-Node and a Serial TTL Camera like this:
    http://aliexpress.com/item/NEW-RS232-TTL-JPEG-Digital-Serial-Port-CCTV-Camera-Module-SCB-1-with-video-out-Support/1975852463.html

    2015-07-30 19.38.29 - Kopie.jpg


  • Admin

    Awesome! Really simple and clean example (and perl script!).



  • Thanks. :)
    There was much try and error involved to get this running.
    But i'm still looking forward for suggestions/discussion about the implementation of a STREAM feature.

    What do you guys think about a fixed routing while streaming?
    The init of the stream could be started by a special payload, when the gateway receives this message it generates a "stream-id". It would link this stream-id to the node-id and sensor-id and send the stream-id back to the sensor. Repeaters in the middle of the way, to the node, would also remember the link betweend stream-id and node-id.
    After this point the node would begin to stream its content only with the stream-id assigned, which would free up to 30bytes per package.

    The gateway itself could reverse this process and send out "normal" looking "commands".

    The stream-id will be freed at the moment the stream is completed or a timeout occurs.


    This ist just a suggestion by me.


  • Admin

    Yes, would be nice to free up some payload space for this type of data.

    Just a bit worried about allocating things (dynamically or statically) for the stream information (beside the extra code). Pretty cramped already.

    What happens if you just increase the RF24 datarate in MyConfig? Do you lose much data?


  • Admin

    @Oitzu nice, I'll think about implementing this in MYSController for testing purposes.



  • @hek said:

    Yes, would be nice to free up some payload space for this type of data.

    Just a bit worried about allocating things (dynamically or statically) for the stream information (beside the extra code). Pretty cramped already.

    I think by deactivating not used features with pre-compile directives this problem could be solved. Need to look into this again. But i remember there are large chunks of unused code compiled in if you, by example, only want to use the lib for a "pure" node.

    What happens if you just increase the RF24 datarate in MyConfig? Do you lose much data?

    Didn't tried yet. The target for this application is a long range communication. So i keeped the datarate low in my tests.
    BUT i need to note: Most of the 40 seconds, at this moment, is probably a result of the slow UART connection to the camera. The camera-chip has spi but it isn't breaked out on the board. There are boards out there with the same chip + SPI! Users reported the transfer over SPI to the MCU takes about 1 second!

    @tekka nice to hear. :) Let me know if i can support somehow.



  • Update!
    I changed the camera to a faster SPI model.
    http://de.aliexpress.com/item/ArduCAM-M-2MP-Camera-Shield-OV2640-2MP-SPI-Camera-module/32307177889.html?ws_ab_test=201407_5,201444_5,201409_3

    The camera delivers a 2MP picture in under 1 seconds over SPI.

    But i'm running in a dead end, with MySensors not supporting proper streams.
    A 2MP (100kb) takes about 20 seconds with MySensors.
    Main problem atm is IMHO with the mysensors transmitting style:

    	rf24.powerUp();
    	rf24.stopListening();
    	rf24.openWritingPipe(TO_ADDR(to));
    	bool ok = rf24.write(data, len, to == BROADCAST_ADDRESS);
    	rf24.startListening();
    	return ok;
    

    would be more effective to do something like:

    	rf24.powerUp();
    	rf24.stopListening();
    	rf24.openWritingPipe(TO_ADDR(to));
    ///loop here over more than 1 packet
    
    rf24.writeFast(data, len, to == BROADCAST_ADDRESS);
    
    //end of loop
    	rf24.startListening();
    	return ok;
    

    The stopListening -> openWritingPipe -> startListening just takes to much time.

    Unfortunately i failed a try to implement this... it seems i'm just not experienced enough.



  • Hi,

    I'm currently trying to use your solution, @Oitzu.
    The Node-part is working fine. Have a little problem on the Pi-side.
    The RF-device is directly connected to the Pi, so I'm using a Serial Gateway script on the Pi (https://github.com/mysensors/Raspberry). The serial data is then send over /dev/ttyUSB20, that is working fine.
    I know that I need to adjust the following lines of the Perl script:

    use Device::SerialPort;
    use POSIX qw(strftime);
    use Data::Dumper;
    use Time::HiRes qw (sleep);
    use Time::HiRes qw(time);
    my $port = Device::SerialPort->new("/dev/ttyUSB1");
    

    But it seems I'm just to stupid to do this by myself :grin:
    I would suggest that I can delete the first line and change the last line to something like this:

    my $port = "/dev/ttyUSB20";
    

    Can someone help me with the right syntax?

    Thanks!



  • @thomas1412 said:

    Hi,

    I'm currently trying to use your solution, @Oitzu.
    The RF-device is directly connected to the Pi, so I'm using a Serial Gateway script on the Pi (https://github.com/mysensors/Raspberry). The serial data is then send over /dev/ttyUSB20, that is working fine.
    I know that I need to adjust the following lines of the Perl script:

    use Device::SerialPort;
    use POSIX qw(strftime);
    use Data::Dumper;
    use Time::HiRes qw (sleep);
    use Time::HiRes qw(time);
    my $port = Device::SerialPort->new("/dev/ttyUSB1");
    

    But it seems I'm just to stupid to do this by myself :grin:
    I would suggest that I can delete the first line and change the last line to something like this:

    my $port = "/dev/ttyUSB20";
    

    Can someone help me with the right syntax?

    The RPi Serial Gateway creates a symlink "/dev/ttyMySensorsGateway" you should use that as Device for more consistent results.

    I srsly don't know why you would delete the first line. I'm pretty sure you need to use the Device::Serialport Perl-Module. ;)

    Also try to run the script as root. The created device files tend to have root-only access rights.



  • Hi,

    thanks for the quick reply and your help
    It turned out my problem was the not installed serialport perl library.. Now I'm a step further.
    Unfortunately there's another problem now.. :smile:
    After sending three or four packages the next six packages are failing to send. And this over and over again.
    Seems that the SerialGateway is too slow in some way..

    send: 254-254-0-0 s=255,c=0,t=17,pt=0,l=5,sg=0,st=ok:1.5.1
    send: 254-254-0-0 s=255,c=3,t=6,pt=1,l=1,sg=0,st=ok:0
    sensor started, id=254, parent=0, distance=1
    send: 254-254-0-0 s=3,c=0,t=23,pt=0,l=0,sg=0,st=ok:
    Sending Picture of 13048 Bytes.
    send: 254-254-0-0 s=3,c=1,t=25,pt=0,l=5,sg=0,st=ok:START
    send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=ok:FFD8FFFE00244900020D0000000000000000000000000000
    send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=ok:00F0004001070032120B510451040000FFDB008400020101
    send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=ok:020101020201020202020203040303020203050404030406
    send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=fail:0607070606060607080A0907070A080606090C090A0A0B0B
    send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=fail:0B0B07080C0D0C0B0D0A0B0B0B0102020203020305030305
    send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=fail:0B0706070B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B
    send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=fail:0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B0B
    send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=fail:0B0B0B0B0B0BFFFE0005000000FFC000110800F001400301
    send: 254-254-0-0 s=3,c=1,t=24,pt=6,l=24,sg=0,st=fail:2100021101031101FFC401A2000001050101010101010000```

  • Admin

    Might be a powering issue of the radio. Put a bigger cap on it.



  • Adding a capacitor hasn't helped.
    I changed the setup using now a serial gateway on a separate Arduino instead of having it directly on the Pi. This is working fine. So it seems that the serial gateway on the Pi was the problem..


Log in to reply
 

Looks like your connection to MySensors Forum was lost, please wait while we try to reconnect.