[contest] My 12 input high precision pulse counter (kWh/ W)

[A Former User]

Great project, ive made this project, i had to declare the chars this was not done in your script? i had to declare:
lcd.createChar(1, degCelcius);
lcd.createChar(2, hecto);
lcd.createChar(3, Pascal);
lcd.createChar(4, Lux);
lcd.createChar(5, visible);
lcd.createChar(6, infrared);
lcd.createChar(7, rel_humidity);

i declared them for eg:
uint8_t degCelcius[8] = {0x18,0x18,0x3,0x4,0x4,0x4,0x3};

i had to change the display adress to 0x3F and tested this with a script to verify and this works, display SCL is connected to pin A5 and SDA is connected to A4.

ive compiled it without errors and uploaded the sketsh to the arduino nano, when connecting the slave arduino (pro mini) board the Rx led will blink every 10seconds when the json data is received.
but the display stays like this:
http://forum.arduino.cc/index.php?action=dlattach;topic=157817.0;attach=40625
on the top row only blocks are visible, any idea why?

[AWI]

@RCF It was a long time ago.... can't remember the details. It could be that the display library does not work for your LCD. If I can find a minute I will review the sketch.

[A Former User]

I have the same display as in your post > see the first display on this page: http://arduino-info.wikispaces.com/LCD-Blue-I2C#v1
there is also a test script which works fine on adress 0x3F, but i did already change that adres in your script but the display shows blocks on the first line. the library is the same i believe.
it would be grat if you could help :).

by the way if i connect a RF module will the data sended be compatible with http://openenergymonitor.org/emon/guide the Emonbase unit?

[MarkV]

Could you explain how to connect the sensors 2 pins?
On http://openenergymonitor.org/emon/buildingblocks/12-input-pulse-counting they connect it like in the picture?

Is it by this also like that?? Because i can't see the pull down resistors on yours..

[AWI]

You noticed it right. I used the internal pull-up of the Arduino. Gnd is directly connected and input with the internal pull-up to vcc.

[MarkV]

So i don't need the extra pulldown resistor and could connect everything like drawn down below?

With the differance that i'm using a arduino mini wich runs on 3.3V and doesn't have a 5V point...

[AWI]

@MarkV sounds good. I am reworking the master sketch a little to cope with the new libraries and MySensors 1.5

[AWI]

Updated the master sketch... you need the load the latest libraries mentioned in the heading..(RotaryEncoder & arduinoJson). Adapt your settings to your display if needed (I2C address, etc.).
The sketch depends on a "read/ get" from the controller for the last (or user filled) kWh values. Many controllers are lacking this functionality (i.e. Domoticz). I have a version without this (useful ) syncing capability under construction. This will allow local changes in the history by means of the rotary encoder.

/*----------------------------------------------------------------------*
 * MySensors interface to AWI 12 input usage/ power Meter               *
 *  meter generates Power/ Usage/ Cumm. usage for 12 independent chann. *
 * 16x2 (I2C) Display shows consumption                                 *
 *                                                                      *
 *----------------------------------------------------------------------*/ 
 /*
//
// Licence: GNU GPL
//
// Author: AWI 2015
// Updates: AWI 20150815, new RotaryEncoder & ArduinoJson libraries
//
//
// continuously:
// 1. read JSON strings from Pulse12Counter slave , format {"m":meter,"c":count,"r":rate, "cA":countAccum}
// 2. calculate real time Power from "rate"
// 3. store values 
// 4. send to controller

// Get consistency with controller - controller should only be updated when initial values have been received:
// 1. read Wh values from controller (local last values should be in sync with controller)
// 2. compare with stored readings. Action accordingly: 
//    - if controller Wh > stored last reading: new startup from counter, send new Wh increment and adjust readings. 
//    - if controller Wh < stored last reading: something went wrong with last update, send updated values from counter
//
// Other:
// 1. display on local LCD, time & energy values, usage can be shown as Daily & Total
// 2. rotary encoder to browse the different displays
//
// Caution: pulse meter is connected to std serial (pin D0), Serial.print can still be used  
//    Disconnect pulse meter when programming via FTDI! (sync error)
 */
#include <MySensor.h>                 // MySensors network
#include <SPI.h>
#include <LiquidCrystal_I2C.h>        // display I2C
#include <Time.h> 
#include <Wire.h> 
#include <RotaryEncoder.h>          // RotaryEncoder to browse displays https://github.com/mathertel/RotaryEncoder
#include <ArduinoJson.h>           // used to parse the simple JSON output of the pulse meter https://github.com/bblanchon/ArduinoJson

// Constants
#define NO_METERS  5                // number of meters used (max 12)
#define NODE_ID    11             // fixed MySensors node ID
#define ENCODER_PINA 3            // RotaryEncoder pins
#define ENCODER_PINB 4

// *** Global variables
// stores LCD text
//char lastLCD1[21] = "Line1 - first   ";
//char lastLCD2[21] = "Line2 - second  ";

class pulseMeter              // meter class, store all relevant data (equivalent to struct)
{
public:                                  
  unsigned long UsageWh;                  // last (current) usage (in W) from pulse counter
  unsigned long UsageAccumWh;               // usage accumulator (to keep in sync) from pulse counter
  unsigned long PowerW;                 // actual power, calculated from pulse "rate"
  unsigned long DayUsageWh;           // daily usage for display
  bool  Updated ;             // flag to check if update from controller is received
};

pulseMeter pulseMeters[NO_METERS] ;     // define power meters
RotaryEncoder encoder(ENCODER_PINA, ENCODER_PINB); //, 1, 1, 3000);    // instance of RotaryEncoder: (Pin1, Pin2, Multiplier(), Stepsize(), pause(ms)]

// Json parser:  define parse object: <10> = number of tokens in JSON: ~10 per m (4 * [key, value] + key_total, value_total))
    char json[50]="{\"m\":1,\"c\":12,\"r\":120000,\"cA\":12345}";              // Storage for serial JSON string


// flags & counters 
bool timeReceived = false;
bool newDay = false;
unsigned long lastUpdate=0, lastRequest=0, lastDisplay=0, lastSyncKWH=0;  // loop timers for once in while events
int display_no = 0;             // determines what is displayed on second line, first line = time display;
int currentMeter = 0;           // actual meter for update & check, cycles through meters
int lastRotary = 0;    // last rotary encoder position
// *** Definition and initialisation
// define the MySensor network
MySensor gw;                    // pins used RFX24(default 9,10)
    
// Initialize messages for sensor network
MyMessage powerMsg(0,V_WATT);         // message to send power in W
MyMessage usageMsg(0,V_KWH);        // message to send usage in kWH

// Set the pins on the I2C chip used for LCD connections:
//                    addr, en,rw,rs,d4,d5,d6,d7,bl,blpol
LiquidCrystal_I2C lcd(0x27, 2, 1, 0, 4, 5, 6, 7, 3, POSITIVE);  // Set the LCD I2C address

// Temp Buffer to print readings on the LCD.
char output_buffer[6];
// OPTIONAL: Custom characters for display - Units (add W/ kWh?)
byte degCelcius[8] = {
  B01000,
  B10100,
  B01000,
  B00111,
  B00100,
  B00100,
  B00111,
  B00000
};
byte Pascal[8] = {
  B00000,
  B11100,
  B10100,
  B11100,
  B10010,
  B10111,
  B10101,
  B00000
};
byte hecto[8] = {
  B00000,
  B00000,
  B00000,
  B00100,
  B00110,
  B00101,
  B00101,
  B00000
};
byte Lux[8] = {
  B00000,
  B10000,
  B10101,
  B10010,
  B10010,
  B10101,
  B11100,
  B00000
};
/*byte visible[8] = {
  B00000,
  B00000,
  B00001,
  B00000,
  B10101,
  B10101,
  B01001,
  B00000
};
byte infrared[8] = {
  B00101,
  B00010,
  B10101,
  B00000,
  B10110,
  B10100,
  B10100,
  B00000,
};
*/
byte rel_humidity[8] = {
  B00101,
  B00010,
  B00101,
  B10000,
  B11000,
  B10100,
  B10100,
  B00000,
};


void setup(void)
{
    //Serial in Sensor network = 115200
    gw.begin(incomingMessage, NODE_ID, false);  // this node is fixed, no repeat
    //Send the sensor node sketch version information to the gateway
    gw.sendSketchInfo("AWI-12ChannelPulse", "1.0");
    // Register all Pulse counters to gw (they will be created as child devices from 0 to MAX-1)
    for (int x = 0; x < NO_METERS; x++){ 
      gw.present(x, S_POWER);             // present power meters to gateway
    delay(100);                         // give it some time to process
    }
 
    // Initializations
    Wire.begin();                 // I2C

    // Request latest time from controller at startup
    gw.requestTime(receiveTime);
    for (int x = 0; x < NO_METERS; x++){ 
      gw.request(x, V_KWH);             // for sync get current values from controller
      pulseMeters[x].Updated = false ;      // set meter status to not in sync
      gw.wait(100);                         // give it some time to process
    }
    // ** LCD display **
    lcd.begin(16, 2);             // LCD 2 lines * 16 char.
    lcd.setBacklight(HIGH);
    // send custom characters to display
    lcd.createChar(1, degCelcius);
    lcd.createChar(2, hecto);
    lcd.createChar(3, Pascal);
    lcd.createChar(4, Lux);
//    lcd.createChar(5, visible);
//    lcd.createChar(6, infrared);
    lcd.createChar(7, rel_humidity);
    lcd.setCursor(0, 0);             // Reset cursor position
}


void loop(void)
{
  unsigned long now = millis();       // Timer in loop for "once in a while" events
  gw.process() ;                      // process incoming messages
    // If no time has been received yet, request it every 10 second from controller
    // When time has been received, request update every hour
    if ((!timeReceived && now-lastRequest > 10*1000)
      || (timeReceived && now-lastRequest > 60*1000*60)) {
      // Request time from controller. 
       Serial.println("requesting time");
       timeReceived = false;
       gw.requestTime(receiveTime);  
       lastRequest = now;
    }
  // Check if new day has started (hour == 0) and reset day usage counters of meters
  if (hour()==0 && !newDay){
    newDay = true;
    for (int x = 0; x < NO_METERS; x++){ 
      pulseMeters[x].DayUsageWh = 0 ;   // reset daily counters
    }   
  } else {
    newDay = false;
  }
  
  // check if RotaryEncoder action for display selection
  encoder.tick();
  int enc = encoder.getPosition();
  if( enc > lastRotary) {
    if (display_no++ >= NO_METERS-1 ) display_no = 0;
    lastRotary = enc ;
  }
  else if (enc < lastRotary) {
    if (display_no-- <  0) display_no = NO_METERS - 1;
    lastRotary = enc ;
  }

  // Update display every second (mainly for time display)
  if (now-lastUpdate > 1000) {
    LCD_local_display();
    lastUpdate = now;
    }

  // If no "RotaryEncoder", Change display every 10 seconds
  //if (now-lastDisplay > 10000) {
  //  // change display
  //  display_no++;
  //  if (display_no >= NO_METERS){         // meter display for second line
  //    display_no = 0;
  //  }
  //  lastDisplay = now ;
  //
  // Every 10 seconds update one meter and check if controller is still in sync
  
  if (now-lastSyncKWH > 10000){
    // printPulsemeter(display_no);
    sendPowerUpdate(currentMeter);        // update the values for currentMeter
    if (currentMeter++ >= NO_METERS){     // increment and wrap current meter
      currentMeter = 0 ;
    }
    // take care of synchronization
    gw.request(currentMeter, V_KWH);        // get current values from controller
    lastSyncKWH = now;
  }
  // get readings from serial (sent every 10s)
  // format {"m":meter,"c":count,"r":rate, "cA":countAccum}
  // use JSON parser to process (could be replaced by simple split routine, but this works just fine)
  if(readLineJSON(Serial.read(), json, 80) > 0 ){   //dummySerial(), Serial.read()
  // if(readLineJSON(dummySerial(), json, 80) > 0 ){   //dummySerial(), Serial.read()

    Serial.println(json);
    storeMeterJSON(json);           //store the meter reading
  }
}

// This is called when a new time value was received
void receiveTime(unsigned long controllerTime) {
    // Ok, set incoming time 
    Serial.print("Time value received: ");
    Serial.println(controllerTime);
    setTime(controllerTime); // this sets the clock to the time from controller - which we do want periodically
    timeReceived = true;
}

// This is called when a kWh message is received 
void incomingMessage(const MyMessage &message) {
  // Expect few types of messages from controller, V_KWH for last value of kWh (for each meter)
  if (message.type==V_KWH) {
  // if message comes in, update the kWH reading for meter with value since last update
  // Write some debug info
  Serial.print("Last reading for sensor: ");
  Serial.print(message.sensor);                 // kWh is float
  Serial.print(", Message: ");
  Serial.println(message.getFloat());
  unsigned long ControllerUsageWh = message.getFloat()*1000 ;   // update current values
  if (ControllerUsageWh > pulseMeters[message.sensor].UsageWh){   // if controller value > current: meter restarted
    pulseMeters[message.sensor].UsageWh = ControllerUsageWh ;   //  update with controller value
  } // else: do nothing meter value is higher and therefore more recent
  pulseMeters[message.sensor].Updated = true;   // set updated status flag. Only process meters if in sync (value received).
  }
}
 
void sendPowerUpdate(int currentMeter)
// Sends update to controller for current meter 
// !! check needs to be implemented for Correct sync with controller...
{
  if  (pulseMeters[currentMeter].Updated){      // only send when there was a sync response from the controller
    gw.send(powerMsg.setSensor(currentMeter).set((int)pulseMeters[currentMeter].PowerW));
    gw.send(usageMsg.setSensor(currentMeter).set((float)pulseMeters[currentMeter].UsageWh/1000,3)); // send in kWh!
    printPulsemeter(currentMeter);
  }
}

void LCD_local_display(void)
// prints variables on LCD display with units
{
    char buf[17]; // buffer for 16 char display
    // start with time on first line
    snprintf(buf, sizeof buf, "%02d:%02d:%02d %02d-%02d", hour(), minute(), second(), day(), month());
    lcd.setCursor(0,0);               // LCD line 1
    lcd.print(buf);
  // print meter indicated by "display_no"
    snprintf(buf, sizeof buf, "m%2d %4dWh %4dW", display_no+1, (int)pulseMeters[display_no].DayUsageWh, (int)pulseMeters[display_no].PowerW );
  lcd.setCursor(0,1);               // LCD line 2
  lcd.print(buf);
}


int storeMeterJSON(char *json)
/* convert JSON to values and store in corresponding meter (if used)
 input: JSON string (can be wrong formatted), with length
 output: changed meter record number or -1 if error
 use JsonParser 
*/
{
  StaticJsonBuffer<50> jsonBuffer;
  // char njson[] = "{\"m\":1,\"c\":12,\"r\":120000,\"cA\":12345}";
    JsonObject& root = jsonBuffer.parseObject(json);
  if (!root.success())
    {
        Serial.println("JsonParser.parse() failed");
        return -1;
    }
  int m = (long)root["m"];
  if (m > NO_METERS){                 // meter value out of range for used meters 
    return -1 ;
  } else {                      // update meter values, Power is momentary, Usage is cumulative
    pulseMeters[m-1].UsageWh += (long)root["c"];    // pulsecount = Wh, accumulate
    pulseMeters[m-1].DayUsageWh += (long)root["c"];   // daily counter = Wh, accumulate
    pulseMeters[m-1].UsageAccumWh = (long)root["cA"]; // accumulated pulse count (only for sync and eventual error correction (serial))
    if ((long)root["r"] == 0){            // calculate power from pulse rate (ms) and truncate to whole Watts
      pulseMeters[m-1].PowerW = 0;
    } else {
      pulseMeters[m-1].PowerW = int(3600000000 / (long)root["r"]); // rate in microseconds
    }
    return m ;
  }
}

int readLineJSON(int readch, char *buffer, int len)
/* checks for JSON and when started append char tot buffer and checks for line completion 
usage:
  static char buffer[80];
  if (readline(Serial.read(), buffer, 80) > 0) { // line complete}
  returns simple JSON
  */
{
  static int pos = 0;
  int rpos;

  if (readch > 0) {
    switch (readch) {
      case '\n':                // Ignore new-lines
        break;
      case '\r':                // Return on CR
        rpos = pos;
        pos = 0;                  // Reset position index ready for next time
        return rpos;
      default:
        if (pos < len-1) {
          buffer[pos++] = readch;
          buffer[pos] = 0;
        }
    }
  }
  // No end of line has been found, so return -1.
  return -1;
}


int dummySerial()
// Acts as a dummy JSON serial character generator for debugging
// input: none
// output: preset JSON string
{
  static int pos = 0;
  char json[] = "{\"m\":1,\"c\":12,\"r\":120000,\"cA\":12345}\r{\"m\":2,\"c\":212,\"r\":2120000,\"cA\":212345}\r{\"m\":3,\"c\":212,\"r\":2120000,\"cA\":212345}\n\r";
  if (pos++ >= strlen(json)){
    pos = 0;
  } 
  return json[pos] ;
  
}


void printPulsemeter(int meter)
// prints the Pulsemeter record to serial out
{
  Serial.print("m:");
  Serial.print(meter);
  Serial.print(", power: ");
  Serial.print(pulseMeters[meter].PowerW);
  Serial.print(", usage: ");
  Serial.print(pulseMeters[meter].UsageWh);
  Serial.print(", day usage: ");
  Serial.print(pulseMeters[meter].DayUsageWh );
  Serial.print(", Ca: ");
  Serial.println(pulseMeters[meter].UsageAccumWh);
}

[A Former User]

Ive updated the libraries and uploaded it to my arduino nano, in the serial of my comport it says: radio init fail.

is the radio module needed to get it working? and does it connect with the emonPI of emon base http://openenergymonitor.org/emon/guide ?
now the display still shows blocks on the first line just like before, is this because the RF module is not present?

[AWI]

@RCF The radio is needed to have the MySensors stuff working otherwise you will get the "radio init fail" message and block the execution of the code. This sketch is not meant for anything else than MySensors. If you want to connect it to emon or anything else it can serve as inspiration...

[A Former User]

i just ordered one at ebay, where does your radio module communicated with? with a server ?
i think it should also be possible to connect pin 9 and 10 to an ethernet module right? i would like to setup an emoncs server for openremote (openremote.org) on my Synology NAS

[korttoma]

@RCF Maybe you should start here -> http://www.mysensors.org/about

[A Former User]

Yes i understand that this is for extra sensors http://www.mysensors.org/about/network but when i do not have any external sensors the radio module should be there anyway

[MarkV]

I've uploaded both sketches without errors, but can't get any readings when i check the serial monitor? Also it doesn't say anything about the communication with the gateway.

What's wrong? What should i check?

[AWI]

@MarkV Hi Mark, good hear you are trying to get it working. You need two arduino's which one is not showing serial information? You first need to get the "measurement" arduino working. This one is supposed to spit out JSON strings. Second you try the "master".

[MarkV]

I've connected everything, but it says:

11-11-0-0 s=255,c=0,t=17,pt=0,l=3,sg=0,st=fail:1.5

What does this mean?? What does the fail mean?

Also the sensors don't show up in Domoticz.

And now i'm getting al sort off strange signes:
-11-0-0 s=255,c=0,t=17,pt=0,l=3,sg=0,st=fail:1.5
11-11-0-0 s=255,c=3,t=6,pt=1,l=1,sg=0,st=fail:0
.W J‘õ11, parent=0, distance=1
LLKLLKj‚šõ255,c=3,t=11,pt=0,l=18,sg=0,st=fail:AWI-12ChannelPulse
LLKLLKj‚šõ255,c=3,t=12,pt=0,l=3,sg=0,st=fail:1.0
LLKLLKj‚šõ0,c=0,t=13,pt=0,l=0,sg=0,st=fail:
LLKLLKj‚šõ1,c=0,t=13,pt=0,l=0,sg=0,st=fail:
find parent
LLKLLK¦SSIªªšõ255,c=3,t=7,pt=0,l=0,sg=0,st=bc:
11-0-0 s=2,c=0,t=13,pt=0,l=0,sg=0,st=fail:
LLKLLKj‚šõ3,c=0,t=13,pt=0,l=0,sg=0,st=fail:
LLKLLKj‚šõ4,c=0,t=13,pt=0,l=0,sg=0,st=fail:
LLKLLKj‚šõ5,c=0,t=13,pt=0,l=0,sg=0,st=fail:
LLKLLKj‚šõ255,c=3,t=1,pt=0,l=0,sg=0,st=fail:
LLKLLKj‚šõ0,c=2,t=18,pt=0,l=0,sg=0,st=fail:
find parent
LLKLLK¦SSIªªšõ255,c=3,t=7,pt=0,l=0,sg=0,st=bc:
11-0-0 s=1,c=2,t=18,pt=0,l=0,sg=0,st=fail:
LLKLLKj‚šõ2,c=2,t=18,pt=0,l=0,sg=0,st=fail:
LLKLLKj‚šõ3,c=2,t=18,pt=0,l=0,sg=0,st=fail:
LLKLLKj‚šõ4,c=2,t=18,pt=0,l=0,sg=0,st=fail:
LLKLLKj‚šõ5,c=2,t=18,pt=0,l=0,sg=0,st=fail:
¢¥µ•5
LLKLLKj‚šõ255,c=3,t=1,pt=0,l=0,sg=0,st=fail:
find parent
LLKLLK¦SSIªªšõ255,c=3,t=7,pt=0,l=0,sg=0,st=bc:
11-0-0 s=1,c=2,t=18,pt=0,l=0,sg=0,st=fail:
Y‹é 11-11-0-0 s=255,c=0,Y‹é 11-11-0-0 s=25óYk½É�started, id=11,equesting time
JsonParser.parse() failed
¢¥µ•5
LLKLLKj‚šõ255,c=3,t=1,pt=0,l=0,sg=0,st=fail:
LLKLLKj‚šõ2,c=2,t=18,pt=0,l=0,sg=0,st=fail:
–‹é 11-11-0-0 s=25Í–‹é 11-11-255-255`¾ò–‹é 11-1Y‹é 11-ñequesting time
JsonParser.parse() failed

[AWI]

@MarkV it sounds like the radio cannot make a connection to the gateway. In many cases this comes from the power supply of the radio. But it can also be distance to gateway etc. Do you have other sensors working?

[MarkV]

Good evening,

It took a while but i wanted to check everything and see what i can do about it by myself.
And.... still no luck, except that i have a running ethernet GW, plus a DHT11 Humidity/temperature sensor up and running.

My pulse counter is also running and i'm getting output on the usb, normal output haha.
Also the gateway is able to see the counter.
Domoticz sees the gateway, the only problem is, that it doesn't add the sensor to it.
The strange part is that is does add the humidity/temperatuur arduino.

I've pressed the "add new sensors for 5 minuts" button and still no luck.

I've also installed MYScontroller and when i connect through tcp i get the following:

Arduino Sketch serial monitor:

What's wrong, what did i do wrong and what should i do to fix it??

[AWI]

@MarkV Good morning ;-) the issue you are having is probably due to the fact that Domoticz sometimes creates sensors after actual data is received (i.s.o. when presenting). The original sketch won't start sending data until it receives something. (line 111 in code above). Also Domoticz is not able to to "send" data values, I had to tweak it myself...
The whole dependency on the controller gave me a lot of headaches. Therefore I decided to rework the whole thing in an independent version. This one is running stable for a few weeks now with a "standard (beta)" version of Domoticz.

The code and hardware is similar but requires some attention. i.e. you need different and the newest libraries. Just drop me a note if you need any assistance.

[MarkV]

Thanks for the commet and i defently need some help.
Sorry i didn't see your "new" post about the pulse counter.

I'm uploading the sketches now.
I only got one question, what should i uncomment in the sketches when i'm not using the display and counter?

And when i got this working, is it possible to add to other sensors, a CNY70 (reflector sensor) to readout a water and gas meter? (but this is the next step ;-) )

I also installed the beta version from domoticz.