Sending Sensor data to Cloud with GSM module

[esawyja]

Hi all,
Any ideas on how to do this please, I have a serial gateway running with a GSM module and a "button pressed" sensor. When I press the button I can see this below on the serial monitor.
0;255;3;0;14;Gateway startup complete.
0;255;0;0;18;2.1.1
Serial Setup Complete!
Serial Setup to A6 Complete!
10;3;1;0;16;0
10;3;1;0;16;1
Without using a controller, how can I "capture" the sensor data and send it to my server in the cloud via the GSM module? I can get the communication with the GSM module working, its just the capture of the MySensors data that I have no idea of how to do...
Any ideas will be appreciated..

[mfalkvidd]

@esawyja maybe you can modify the mqtt gateway? It is built for sending data directly to a server. The MySensors example mqtt sketches assume that the gateway has ethernet or wifi connection, but it should be possible to make it work with a gsm connection.

[esawyja]

Thanks @mfalkvidd , Let me read up on that a bit

[gohan]

@esawyja there are multiple examples over internet and youtube on how to connect gsm modules to arduino and get internet IP connection. Also adafruit has some examples on gsm and mqtt.

[esawyja]

I've read about MQTT, but I'm afraid it is above my pay grade Is there no way of capturing the data what gets displayed on the serial port of the gateway, or rather, what does MySensors use as the serial port, all I need is some kind of trap what will say if I receive this
10;3;1;0;16;0 for example,
initialize the GSM module and post the sensor value using curl or something, could someone point me in the right direction please, my head is spinning

[gohan]

I think investing some time on learning how mqtt works, is going to make it simpler to manage it later on because mqtt was born to be used for applications like this one.

[esawyja]

Ok in a effort to learn MQTT, what I did is the following, I used my RFM69 sensor (that did work on a Serial Gateway, see the values received above) and with that to build a MQTT gateway, I used an Arduino Mega with a W5100 Ethernet Shield and connected the RFM69 radio to the Mega, using the pinouts of the RFM69 & Arduino page and also included a level converter on the NSS, SCK MOSI and MISO pins
Radio is powered by 3.3V and I uploaded this sketch below to the Mega

/**
 * The MySensors Arduino library handles the wireless radio link and protocol
 * between your home built sensors/actuators and HA controller of choice.
 * The sensors forms a self healing radio network with optional repeaters. Each
 * repeater and gateway builds a routing tables in EEPROM which keeps track of the
 * network topology allowing messages to be routed to nodes.
 *
 * Created by Henrik Ekblad <henrik.ekblad@mysensors.org>
 * Copyright (C) 2013-2015 Sensnology AB
 * Full contributor list: https://github.com/mysensors/Arduino/graphs/contributors
 *
 * Documentation: http://www.mysensors.org
 * Support Forum: http://forum.mysensors.org
 *
 * This program is free software; you can redistribute it and/or
 * modify it under the terms of the GNU General Public License
 * version 2 as published by the Free Software Foundation.
 *
 *******************************
 *
 * REVISION HISTORY
 * Version 1.0 - Henrik Ekblad
 *
 * DESCRIPTION
 * The W5100 MQTT gateway sends radio network (or locally attached sensors) data to your MQTT broker.
 * The node also listens to MY_MQTT_TOPIC_PREFIX and sends out those messages to the radio network
 *
 * LED purposes:
 * - To use the feature, uncomment WITH_LEDS_BLINKING in MyConfig.h
 * - RX (green) - blink fast on radio message recieved. In inclusion mode will blink fast only on presentation recieved
 * - TX (yellow) - blink fast on radio message transmitted. In inclusion mode will blink slowly
 * - ERR (red) - fast blink on error during transmission error or recieve crc error
 *
 * See http://www.mysensors.org/build/esp8266_gateway for wiring instructions.
 * nRF24L01+  ESP8266
 * VCC        VCC
 * CE         GPIO4
 * CSN/CS     GPIO15
 * SCK        GPIO14
 * MISO       GPIO12
 * MOSI       GPIO13
 *
 * Not all ESP8266 modules have all pins available on their external interface.
 * This code has been tested on an ESP-12 module.
 * The ESP8266 requires a certain pin configuration to download code, and another one to run code:
 * - Connect REST (reset) via 10K pullup resistor to VCC, and via switch to GND ('reset switch')
 * - Connect GPIO15 via 10K pulldown resistor to GND
 * - Connect CH_PD via 10K resistor to VCC
 * - Connect GPIO2 via 10K resistor to VCC
 * - Connect GPIO0 via 10K resistor to VCC, and via switch to GND ('bootload switch')
 *
  * Inclusion mode button:
 * - Connect GPIO5 via switch to GND ('inclusion switch')
 *
 * Hardware SHA204 signing is currently not supported!
 *
 * Make sure to fill in your ssid and WiFi password below for ssid & pass.
 */


// Enable debug prints to serial monitor
#define MY_DEBUG

// Enables and select radio type (if attached)
//#define MY_RADIO_NRF24
#define MY_RADIO_RFM69  // Define for using RFM69 radio
#define MY_RFM69_FREQUENCY RF69_433MHZ  
#define MY_IS_RFM69HW

#define MY_GATEWAY_MQTT_CLIENT

// Set this node's subscribe and publish topic prefix
#define MY_MQTT_PUBLISH_TOPIC_PREFIX "mygateway1-out"
#define MY_MQTT_SUBSCRIBE_TOPIC_PREFIX "mygateway1-in"

// Set MQTT client id
#define MY_MQTT_CLIENT_ID "mysensors-1"

// W5100 Ethernet module SPI enable (optional if using a shield/module that manages SPI_EN signal)
//#define MY_W5100_SPI_EN 4

// Enable Soft SPI for NRF radio (note different radio wiring is required)
// The W5100 ethernet module seems to have a hard time co-operate with
// radio on the same spi bus.
#if !defined(MY_W5100_SPI_EN) && !defined(ARDUINO_ARCH_SAMD)
#define MY_SOFTSPI
#define MY_SOFT_SPI_SCK_PIN 14
#define MY_SOFT_SPI_MISO_PIN 16
#define MY_SOFT_SPI_MOSI_PIN 15
#endif

// When W5100 is connected we have to move CE/CSN pins for NRF radio
#ifndef MY_RF24_CE_PIN
#define MY_RF24_CE_PIN 5
#endif
#ifndef MY_RF24_CS_PIN
#define MY_RF24_CS_PIN 6
#endif

// Enable these if your MQTT broker requires usenrame/password
//#define MY_MQTT_USER "username"
//#define MY_MQTT_PASSWORD "password"

// Enable MY_IP_ADDRESS here if you want a static ip address (no DHCP)
#define MY_IP_ADDRESS 192,168,1,88

// If using static ip you need to define Gateway and Subnet address as well
#define MY_IP_GATEWAY_ADDRESS 192,168,1,1
#define MY_IP_SUBNET_ADDRESS 255,255,255,0

// MQTT broker ip address or url. Define one or the other.
//#define MY_CONTROLLER_URL_ADDRESS "m20.cloudmqtt.com"
#define MY_CONTROLLER_IP_ADDRESS 192, 168, 1, 9

// The MQTT broker port to to open
#define MY_PORT 1883
#define MY_GATEWAY_W5100
/*
// Enable inclusion mode
#define MY_INCLUSION_MODE_FEATURE
// Enable Inclusion mode button on gateway
//#define MY_INCLUSION_BUTTON_FEATURE
// Set inclusion mode duration (in seconds)
#define MY_INCLUSION_MODE_DURATION 60
// Digital pin used for inclusion mode button
//#define MY_INCLUSION_MODE_BUTTON_PIN  3

// Set blinking period
#define MY_DEFAULT_LED_BLINK_PERIOD 300

// Flash leds on rx/tx/err
// Uncomment to override default HW configurations
//#define MY_DEFAULT_ERR_LED_PIN 16  // Error led pin
//#define MY_DEFAULT_RX_LED_PIN  16  // Receive led pin
//#define MY_DEFAULT_TX_LED_PIN  16  // the PCB, on board LED
*/

#include <Ethernet.h>
#include <MySensors.h>

void setup()
{
}

void presentation()
{
  // Present locally attached sensors here
}


void loop()
{
  // Send locally attached sensors data here
}

I then installed MQTT on a raspberry PI on IP 192.168.1.9, from the Mega's serial port, it looks to me that the radio initializes OK and the Mega is connected to my LAN with IP 192.168.1.88, see below

0;255;3;0;9;MCO:BGN:INIT GW,CP=RRNGA--,VER=2.1.1
0;255;3;0;9;TSM:INIT
0;255;3;0;9;TSF:WUR:MS=0
0;255;3;0;9;TSM:INIT:TSP OK
0;255;3;0;9;TSM:INIT:GW MODE
0;255;3;0;9;TSM:READY:ID=0,PAR=0,DIS=0
0;255;3;0;9;MCO:REG:NOT NEEDED
IP: 192.168.1.88
0;255;3;0;9;MCO:BGN:STP
0;255;3;0;9;MCO:BGN:INIT OK,TSP=1
IP: 192.168.1.88
0;255;3;0;9;Attempting MQTT connection...
0;255;3;0;9;MQTT connected
0;255;3;0;9;Sending message on topic: mygateway1-out/0/255/0/0/18

But when I activate the sensor, I don''t see anything as I did before on the Mega and also nothing in the MQTT server logs or subscription

root@raspberrypi:~# mosquitto_sub -v -t 'mygateway1-out/#'
mygateway1-out/0/255/0/0/18 2.1.1
mygateway1-out/0/255/0/0/18 2.1.1

and the log

root@raspberrypi:~# kill -9 28003
root@raspberrypi:~# /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf &
[1] 29149
root@raspberrypi:~# tail -f /var/log/mosquitto/mosquitto.log 
1490722311: New client connected from 192.168.1.88 as mysensors-1 (c1, k15).
1490722482: Client mysensors-1 has exceeded timeout, disconnecting.
1490722482: Socket error on client mysensors-1, disconnecting.
1490722739: Saving in-memory database to /var/lib/mosquitto/mosquitto.db.
1490722757: Socket error on client mosqsub/28798-raspberry, disconnecting.
1490722980: mosquitto version 1.4.11 (build date Mon, 20 Feb 2017 22:49:47 +0000) starting
1490722980: Config loaded from /etc/mosquitto/mosquitto.conf.
1490722980: Opening ipv4 listen socket on port 1883.
1490722980: Opening ipv6 listen socket on port 1883.
1490722984: New connection from ::1 on port 1883.





1490722984: New client connected from ::1 as mosqsub/29150-raspberry (c1, k60).
1490723068: New connection from 192.168.1.88 on port 1883.
1490723068: New client connected from 192.168.1.88 as mysensors-1 (c1, k15).
1490723375: Client mysensors-1 has exceeded timeout, disconnecting.
1490723375: Socket error on client mysensors-1, disconnecting.
1490723398: New connection from 192.168.1.88 on port 1883.

Any ideas please, must the Mega get a NODE_ID? I seem to recall I could only get sensor data on the serial gateway after a assigned a NODE_ID to the sensor, any advice please would be appreciated
Regards

[gohan]

Being gateway you don't need node ID