P1 Smart Meter (NTA8130) readout using Mysensors

[nielsvanharen]

Hey guys, since a few days i got my new smart meter installed which has a p1 dataport, that sends telegram message using a serial connection.

I used to read my energy values using an adjusted pulse power meter setup with an reflection sensor (CNY70) but i thought it would be nice to readout the p1 port and use mysensors to coop with it.

I want to share this project, because i didn't find anything on mysensors about it. and want to share my experiences with it. so we can hopefully learn from each other

To understand what is going on, i read alot about the p1 port in combination with an arduino (UNO) to see what kind of information is sent. i have got alot of usefull information from http://12volt.kloppenburgweb.nl/p1-uitlezen-met-arduino/

we can use the SoftSerial library or an even better one, called AltSoftSerial to read out the data that is sent from the power meter.

the data that is sent, is inverted, so we need a small circuit to put it back to normal data. I used the following schematic.

i replaced the BS170 with a more standard NPN transistor , in my case the BC547.

the data that i received has the following format.

(sorry for some dutch descriptions, but it basicly has some information like, current usage,power_high,power_low,gas meter, etc)

I used the altsoftserial library, because it parses the data in a nicer way then the softserial library, but i has a huge disadvantage, which bothers me alot. i will tell about it later on.

the program to test my arduino uno with the p1 power port.

 #include <AltSoftSerial.h>
#include <SPI.h>

// AltSoftSerial always uses these pins:
//
// Board          Transmit  Receive   PWM Unusable
// -----          --------  -------   ------------
// Teensy 2.0         9        10       (none)
// Teensy++ 2.0      25         4       26, 27
// Arduino Uno        9         8         10
// Arduino Mega      46        48       44, 45
// Wiring-S           5         6          4
// Sanguino          13        14         12

AltSoftSerial altSerial;

int msg[1];
const uint64_t pipe = 0xE8E8F0F0E1LL;

const int requestPin =  4;         
char input; // incoming serial data (byte)
bool readnextLine = false;
#define BUFSIZE 75
char buffer[BUFSIZE]; //Buffer for serial data to find \n .
int bufpos = 0;
long mEVLT = 0; //Meter reading Electrics - consumption low tariff
long mEVHT = 0; //Meter reading Electrics - consumption high tariff
long mEAV = 0;  //Meter reading Electrics - Actual consumption
long mEAT = 0;  //Meter reading Electrics - Actual generated electricity (Solar panels)
long mG = 0;   //Meter reading Gas
int tarief = 3; //Meter actual rate

void setup() {
  Serial.begin(115200);
  delay(1000);

  altSerial.begin(115200);
  delay(1000);

}


void loop() {
  long tl = 0;
  long tld =0;
  

  if (altSerial.available()) {
    input = altSerial.read();
    char inChar = (char)input;
    // Fill buffer up to and including a new line (\n)
    buffer[bufpos] = input&127;
    bufpos++;

    if (input == '\n') { // We received a new line (data up to \n)


      // 1-0:1.8.2 = Electricity consumption high tariff (DSMR v4.0)
      if (sscanf(buffer,"1-0:1.8.2(%ld%.%ld%*s" , &tl, &tld) >0 ) {
        tl *= 1000;
        tl += tld;
        mEVHT = tl;    
        //mEVHT = tl * 1000 + tld;
        if (mEVHT > 0) {

          //mEVHT = 0;
        }
      }

        if (sscanf(buffer,"0-0:96.14.0(%ld.%ld*s" , &tl) >0  )   //)
        {
          tarief = tl;
        }


      if (sscanf(buffer,"1-0:1.8.1(%ld.%ld" ,&tl, &tld)==2){
        tl *= 1000;
        tl += tld;
        mEVLT = tl;

        if (mEVLT > 0) {


        }
      }

      // 1-0:1.7.0 = Electricity consumption actual usage (DSMR v4.0)
      if (sscanf(buffer,"1-0:1.7.0(%ld.%ld" ,&tl , &tld) == 2)
      { 
        mEAV = (tl*1000)+tld;
        if (mEAV > 0) {

        }
      }

      // 0-1:24.2.1 = Gas (DSMR v4.0) on Kaifa MA105 meter
      if (strncmp(buffer, "0-1:24.2.1", strlen("0-1:24.2.1")) == 0) {
        if (sscanf(strrchr(buffer, '(') + 1, "%d.%d", &tl, &tld) == 2) {
          mG = (tl*1000)+tld; 

        }
      }

      // Empty buffer again (whole array)
      for (int i=0; i<75; i++)
      { 
        buffer[i] = 0;
      }
      bufpos = 0;

    }
     
    if (input == '!') {   //uitroepteken geeft einde van telegram aan, dus we gaan data versturen
      //SendNrf();
      printData();
      mEVHT=0;
      mEVLT = 0;
      mEAV = 0;
      mG = 0;
      tarief=3;
      //client.stop();
    } //Einde vraagteken detectie   

  } //Einde 'if AltSerial.available'

}
void printData(){
  Serial.println("********* BEGIN LOOP *********");  
  Serial.print("Elektra - meterstand verbruik HOOG tarief (Wh): ");
  Serial.println(mEVHT);  
  Serial.print("Elektra - meterstand verbruik LAAG tarief (Wh): ");
  Serial.println(mEVLT);  
  Serial.print("Elektra - actueel verbruik (W): ");
  Serial.println(mEAV);
  Serial.print("Gas - meterstand (liters): ");
  Serial.println(mG);

  Serial.print("Huidige tarief: ");
  if (tarief==1){
    Serial.println ("dal");
  }
  else {
    Serial.println("hoog");
  }
//  Serial.println(tarief);
  Serial.println("");
}

Now i want to use an arduino pro mini with the p1 port in stead of the UNO, to see if it works the same as i would expect. At first I used an Arduino Pro Mini 3.3V with an Atmega328. But sadly this does not work, probably because the Pro Mini 3.3V has a lower operating frequency then the UNO (8Mhz vs 16Mhz) and it takes the Arduino to much processing power to parse the serial information before it receives another update from the smartmeter. I came to this conclusion. because sometimes i get some text and sometimes i get corrupted text and then suddenly no text at all...

So i had another Arduino Pro mini laying around with a 16Mhz crystal (5V operating voltage) and tried again with success!

My next step was to integrate it with mysensors. I choose not to use it with battery power and avoid all interrupt procedures to get my data. So i simply listen to the dataport, and when i get my data, i will sent it to my mysensors gateway.

the code is as follows.

#define MY_RADIO_NRF24
#define MY_NODE_ID 110
#define MY_DEBUG 
#include <AltSoftSerial.h>
#include <SPI.h>
#include <MySensor.h>  

//SoftwareSerial portOne(8, 7);
AltSoftSerial altSerial;
int msg[1];
const uint64_t pipe = 0xE8E8F0F0E1LL;

const int requestPin =  4;         
char input; // incoming serial data (byte)
bool readnextLine = false;
#define BUFSIZE 75
#define CHILD_ID_WATT 0
#define CHILD_ID_KWH_LOW 1
#define CHILD_ID_KWH_HIGH 2
#define CHILD_ID_GAS 3
#define CHILD_ID_CURRENT 4

char buffer[BUFSIZE]; //Buffer for serial data to find \n .
int bufpos = 0;
long mEVLT = 0; //Meter reading Electrics - consumption low tariff
long mEVHT = 0; //Meter reading Electrics - consumption high tariff
long mEAV = 0;  //Meter reading Electrics - Actual consumption
long mEAT = 0;  //Meter reading Electrics - Actual generated electricity (Solar panels)
long mG = 0;   //Meter reading Gas
int tarief = 3; //Meter actual rate
MyMessage msgWatt(CHILD_ID_WATT, V_WATT);
MyMessage msgKwhLow(CHILD_ID_KWH_LOW, V_KWH);
MyMessage msgKwhHigh(CHILD_ID_KWH_HIGH, V_KWH);
MyMessage msgGas(CHILD_ID_GAS, V_VAR1);
MyMessage msgCurrent(CHILD_ID_CURRENT, V_VAR2);


void setup() {
  altSerial.begin(115200);
  delay(1000);
}


void presentation() {
  // Send the sketch version information to the gateway and Controller
  sendSketchInfo("Energy Gas", "1.0");
  present(CHILD_ID_WATT, S_POWER);
  present(CHILD_ID_KWH_LOW, S_POWER);
  present(CHILD_ID_KWH_HIGH, S_POWER);
  present(CHILD_ID_GAS, S_CUSTOM);
  present(CHILD_ID_CURRENT, S_CUSTOM);
}



void loop() {
  long tl = 0;
  long tld =0;
  if (altSerial.available()) {
      char input = altSerial.read();
      buffer[bufpos] = input&127;
      bufpos++;
      if (input == '\n') { // We received a new line (data up to \n)
        // 1-0:1.8.2 = Electricity consumption high tariff (DSMR v4.0)
        if (sscanf(buffer,"1-0:1.8.2(%ld%.%ld%*s" , &tl, &tld) >0 ) {
          tl *= 1000;
          tl += tld;
          mEVHT = tl;    
          //mEVHT = tl * 1000 + tld;
          if (mEVHT > 0) {
  
            //mEVHT = 0;
          }
        }
  
          if (sscanf(buffer,"0-0:96.14.0(%ld.%ld*s" , &tl) >0  )   //)
          {
            tarief = tl;
          }
  
  
        if (sscanf(buffer,"1-0:1.8.1(%ld.%ld" ,&tl, &tld)==2){
          tl *= 1000;
          tl += tld;
          mEVLT = tl;
  
          if (mEVLT > 0) {
  
  
          }
        }
  
        // 1-0:1.7.0 = Electricity consumption actual usage (DSMR v4.0)
        if (sscanf(buffer,"1-0:1.7.0(%ld.%ld" ,&tl , &tld) == 2)
        { 
          mEAV = (tl*1000)+tld;
          if (mEAV > 0) {
  
          }
        }
  
        // 0-1:24.2.1 = Gas (DSMR v4.0) on Kaifa MA105 meter
        if (strncmp(buffer, "0-1:24.2.1", strlen("0-1:24.2.1")) == 0) {
          if (sscanf(strrchr(buffer, '(') + 1, "%d.%d", &tl, &tld) == 2) {
            mG = (tl*1000)+tld; 
  
          }
        }
  
        // Empty buffer again (whole array)
        for (int i=0; i<75; i++)
        { 
          buffer[i] = 0;
        }
        bufpos = 0;
  
      }
       
      if (input == '!') {   //uitroepteken geeft einde van telegram aan, dus we gaan data versturen
        #ifdef DEBUG
          printData();
        #endif
        send(msgWatt.set(mEAV));
        send(msgKwhLow.set(mEVLT));
        send(msgKwhHigh.set(mEVHT));
        send(msgGas.set(mG));
        send(msgCurrent.set(tarief));
        mEVHT=0;
        mEVLT = 0;
        mEAV = 0;
        mG = 0;
        tarief=3;
        //client.stop();
      } //Einde vraagteken detectie   
  
    } //Einde 'if AltSerial.available'

}

void printData(){
  Serial.println("********* BEGIN LOOP *********");  
  Serial.print("Elektra - meterstand verbruik HOOG tarief (Wh): ");
  Serial.println(mEVHT);  
  Serial.print("Elektra - meterstand verbruik LAAG tarief (Wh): ");
  Serial.println(mEVLT);  
  Serial.print("Elektra - actueel verbruik (W): ");
  Serial.println(mEAV);
  Serial.print("Gas - meterstand (liters): ");
  Serial.println(mG);

  Serial.print("Huidige tarief: ");
  if (tarief==1){
    Serial.println ("dal");
  }
  else {
    Serial.println("hoog");
  }
//  Serial.println(tarief);
  Serial.println("");
}
 

but then trouble came along....
Because i am using AltSoftSerial my TX and RX ports are restricted to D8 and D9, where RX is D8. so im receiving my information and i want to send it to the mysensors gateway, but D9 is occupied by the NRF24L01 transmitter, i sometimes get half of my information send to the gateway, resulting in some failed messages sent.

I tried to use the normal SoftSerial library, where i simple define my own TX and RX ports, but the data i receive is unreadable for some strange reason... I Decided to let it go for the moment, as i get my information 8/10 in the correct way and will tweak the process a bit more later on.

In the end i get the following information in my openhab.

(i still have to tweak it a bit, my kwh numbers are 1000 off )

[Sparkman]

You can move D9 for the NRF radio to a different pin. Not sure which version of MySensors you are using, but check the relevant documentation on how to do so.

Cheers
Al

[AWI]

@nielsvanharen Like @Sparkman said; as you are using the development version, add something like below to the defines:

#define MY_RF24_CE_PIN 11								// Ceech board, 3.3v (7,8)  (pin default 9,10)
#define MY_RF24_CS_PIN 12

To get some extra speed you could also use the hardware serial. As you are only receiving data connect the Rx to the meter and Tx to your computer for reading the serial outputs. Also the sscanf function are need a lot of processing power... replacing them with some custom code and you can run it below 4Mhz

[GertSanders]

@AWI
Pin 11 and 12 are miso and mosi and already in use by the nrf24. Better to use 4,5,6 or 7

[AWI]

@GertSanders right wrong choice for the example...

[sincze]

@GertSanders I tried your trick indeed of moving the pin, however I think my issue of receiving NO data is something totally different. As your technical knowledge exceeds mine do you have any suggestions ?? see: my problem

[Oleg Vladimirov]

thank you for sharing, it's really useful. Very nice thread!