Gas Meter Reading Using a Magnetometer



  • I was trying to find a way to measure the amount of gas our gas meter was using. I checked out several ways but they all involved using either a hall sensor or a reed switch or a IR sensor or a light sensor. But all of these would not work for my application for a number a reasons. First my meter is outside, second there is no magnet in my meter, third the utility company will not allow us to attach any device to the dials or the electronics of the meter (US smart meter). By the way my meter is a Rockwell 250 (Siemens). So after much researching I'm using a HMC5883L magnetometer. It measures very low magnetic fields.

    Please forgive my lack of pictures and links but when I was assembling it I never thought about it. I can get some pictures of the finished sensor if needed.

    I found the sweet spot on my gas meter using an Android app on my phone called Sensors Box, which allows one to access the magnetometer inside of the phone. While the gas was flowing I just moved it around over the meter until I found a fluctuation that coincided with the gas flow. This is what I call the "sweet spot". I then built a sensor using the HMC5883L per (https://www.sparkfun.com/tutorials/301). But as of this writing I found that they are no longer making it but do have a replacement (https://www.sparkfun.com/products/retired/10530). I have it successfully connected to my serial gateway and am using Domoticz as a controller.

    The initial problem I found was trying to convert the output a the magnetometer to something useful. The data that comes out is stream of numbers from three vectors that goes up and down. After looking at them in the serial plotter of the Arduino IDE I found one vector (y in my case) that looked like a sine wave. Then with a lot of help from my son, who wrote most of the code for the Arduino, it seems to be putting out decent data. However it does need to be fine tuned.

    What we ended up doing is auto detecting the top an bottom of the sine wave, then dividing it up into 10 pieces to get useful pulses.

    I'm working on getting the Metric and English units out put correct but for now the metric units work OK. Any input is welcome:

    UPDATE: 3/12/17 - we have been using the most recent sketch (v.6) and it has been running for two weeks now without a problem. I do plan some updates to the code that I hope will include an interactive serial monitor to set up the sensor for the first use. Also if you have looked at the picture of the sensor it is quite large and I would like to make it a bit smaller.

    Does anyone have any idea how to make it smaller yet water proof?

    /*
     * 
     * 
     * 
     * 
     * Currently the autoDetectMaxMin in set to true which will find the TOP and BOTTOM of the wave, however if you want 
     * to use it the gas must be flowing.
     */
    
    
    
    #define MY_DEBUG
    #define MY_RADIO_NRF24
    
    #include <MySensors.h>                  
    #include <Wire.h>                       //I2C Arduino Library
    
    #define CHILD_ID 1                      //ID of the sensor child
    #define SLEEP_MODE false                //prevent sensor from sleeping
    #define address 0x1E                    //0011110b, I2C 7bit address of HMC5883
    
    int TOP = 0;                            //highest magnetic field registered from meter (Ga)Initialize low if using AutoDetectMaxMin
    int BOTTOM = 10000;                     //lowest magnetic field registered from meter (Ga) Initialize high if using AutoDetectMaxMin
    int tol = 50;
    unsigned long SEND_FREQUENCY = 30000;   // Minimum time between send (in milliseconds). We don't want to spam the gateway.
    
    bool metric = true;                     //sets units to Metric or English
    bool autoDetectMaxMin = true;           //lets Arduino decide the values for TOP and BOTTOM
    bool pcReceived = false;                //whether or not the gw has sent us a pulse count
    bool rising = true;                     //whether or not a pulse has been triggered
    bool inside = true;                     //whether the magnetic field is within TOP and BOTTOM limits
    unsigned long pulsecount = 0;           //total number of pulses measured ever
    unsigned long oldPulseCount = 0;        //old total
    double vpp = metric ? 0.17698 : 0.00625;//Volume of gas per pulse (Rockwell gas meter 250)
    unsigned long lastSend = 0;             //time since last transmission - msec
    double volume = 0;                      //Cumulative amount of gas measured
    const int len = 3;                      //number of flow rate measurements to save
    double flow [len];                      //array of previous gas flow rate measurements
    double avgFlow = 0;                     //average of all elements in flow array
    double oldAvgFlow = 0;                  //previous average flow
    int divider = 1;                        //Current divider
    int totDividers = 10;                    //Number of dividers
    int increment = (TOP - BOTTOM) / totDividers;   //space b/w dividers
    
    MyMessage flowMsg(CHILD_ID,V_FLOW);
    MyMessage volumeMsg(CHILD_ID,V_VOLUME);
    MyMessage lastCounterMsg(CHILD_ID,V_VAR1);
    
    void setup(){
      //Initialize Serial and I2C communications
      Serial.begin(9600);
      Wire.begin();
    
      // Fetch last known pulse count value from gw
      request(CHILD_ID, V_VAR1);
      
      //Put the HMC5883 IC into the correct operating mode
      Wire.beginTransmission(address); //open communication with HMC5883
      Wire.write(0x02); //select mode register
      Wire.write(0x00); //continuous measurement mode
      Wire.endTransmission();
      
      int y = 0;
      int oldy = 0;
    
      //WARNING: MAKE SURE GAS IS RUNNING IF USING THIS OPTION!!!
      if(autoDetectMaxMin){
        //determine max and min magnetic field strength over a few minutes
        lastSend = millis();
        
        while(millis() - lastSend < 300000){
          y = readMag();
          if(y > TOP){
            TOP = y;                        //update TOP if new max has been detected
          }
          else if(y < BOTTOM){
            BOTTOM = y;                     //update BOTTOM if new min has been detected
          }
        }
        
        TOP -= tol;                         //nudge TOP and BOTTOM so that they have a chance of being triggered
        BOTTOM += tol;
    
        increment = (TOP - BOTTOM) / totDividers;    //recalculate increment to match new TOP and BOTTOM
        autoDetectMaxMin = false;           //finished determining TOP and BOTTOM
      }
    
      Serial.print("Increment = ");
      Serial.println(increment);
      y = readMag();
      oldy = readMag();
      while(abs(y - oldy) < increment / 2){ //wait until difference b/w y and oldy is greater than half an increment
        y = readMag();
      }
      rising = (y > oldy);
      Serial.println(rising ? "Magnetic field is rising" : "Magnetic field is falling");
    }
    
    void presentation()
    {
        // Send the sketch version information to the gateway and Controller
        sendSketchInfo("Gas Meter", "0.4");
    
        // Register this device as Gas sensor
        present(CHILD_ID, S_GAS);
    }
    
    void loop(){
      if (!pcReceived) {
        //Last Pulsecount not yet received from controller, request it again
        request(CHILD_ID, V_VAR1);
        return;
      }
      //detecting magnetic pulses - Fractional Simple Method
      while(millis() - lastSend < SEND_FREQUENCY){
        int y = readMag();
    
        if(inside && rising && y > BOTTOM + divider * increment){
          divider++;
          pulsecount++;
        }
        else if(inside && !rising && y < TOP - divider * increment){
          divider++;
          pulsecount++;
        }
    
        if(inside && (y > TOP || y < BOTTOM)){        //switch directions once TOP or BOTTOM divider has been reached
          inside = false;                 //keep this from happening multiple times once signal exceeds TOP or BOTTOM
          Serial.println("OUTSIDE");
        }
        else if(!inside && (y < TOP - increment / 2 && y > BOTTOM + increment / 2)){
          rising = !rising;
          divider = 1;
          inside = true;
          Serial.println("INSIDE");
        } 
      }
      
      //shift all flow array elements to the right by 1, ignore last element
      for(int idx = len; idx > 0; idx--){
        flow[idx] = flow[idx - 1];
      }
      //calculate newest flow reading and store it as first element in flow array
      flow[0] = (double)(pulsecount - oldPulseCount) * (double)vpp * 60000.0 / (double)SEND_FREQUENCY;
      //display flow array state
      Serial.print("Flow Array State: [");
      for(int idx = 0; idx < len - 1; idx++){
        Serial.print(flow[idx]);
        Serial.print("|");
      }
      Serial.print(flow[len]);
      Serial.println("]");
      //calculate average flow
      avgFlow = 0;                                //reset avgFlow
      for(int idx = 0; idx < len; idx++){         //calculate sum of all elements in flow array
        avgFlow += flow[idx];
      }
      avgFlow /= len;                             //divide by number of elements to get average
      Serial.print("Average flow: ");             //display average flow
      Serial.println(avgFlow);
      //send flow message if avgFlow has changed
      if(avgFlow != oldAvgFlow){
        oldAvgFlow = avgFlow;
        send(flowMsg.set(avgFlow, 2));
      }
    
      //send updated cumulative pulse count and volume data, if necessary
      if(pulsecount != oldPulseCount){
        oldPulseCount = pulsecount;              //update old total
        
        //calculate volume
        volume = (double)oldPulseCount * (double)vpp / 1000.0;
    
        //send pulse count and volume data to gw
        send(lastCounterMsg.set(pulsecount));
        send(volumeMsg.set(volume, 3));
      }
    
      lastSend = millis();
      
    }
    
    void receive(const MyMessage &message)
    {
      if (message.type==V_VAR1) {
        unsigned long gwPulseCount=message.getULong();
        pulsecount = gwPulseCount;
        oldPulseCount = pulsecount;
        Serial.print("Received last pulse count from gw:");
        Serial.println(pulsecount);
        pcReceived = true;
        lastSend = millis();
      }
    }
    
    int readMag(){
      int x = 0, y = 0, z = 0;
      
      //Tell the HMC5883 where to begin reading data
      Wire.beginTransmission(address);
      Wire.write(0x03); //select register 3, X MSB register - was called Wire.send but the compiler had an error and said to rename to to Wire.write
      Wire.endTransmission();
    
      //Read data from each axis, 2 registers per axis
      Wire.requestFrom(address, 6);
      if(6<=Wire.available()){
        x = Wire.read()<<8; //X msb
        x |= Wire.read(); //X lsb
        z = Wire.read()<<8; //Z msb
        z |= Wire.read(); //Z lsb
        y = Wire.read()<<8; //Y msb
        y |= Wire.read(); //Y lsb
      }
    
      if(!autoDetectMaxMin){
        //show real-time magnetic field, pulse count, and pulse count total
        Serial.print("y: ");
        Serial.print(y);
        Serial.print(rising ? "  Rising, " : "  Falling, ");
        Serial.print("next pulse at: ");
        Serial.print(rising ? BOTTOM + divider * increment : TOP - divider * increment);
        Serial.print("  Current Number of Pulses: ");
        Serial.print(pulsecount - oldPulseCount);
        Serial.print("  Last Total Pulse Count Sent to GW: ");
        Serial.println(oldPulseCount);
      }
      else{
        //show real-time magnetic field, TOP, BOTTOM, and time left in auto-detect mode
        Serial.print("y: ");
        Serial.print(y);
        Serial.print("  TOP: ");
        Serial.print(TOP);
        Serial.print("  BOTTOM: ");
        Serial.print(BOTTOM);
        unsigned long remainingTime = 300000 + lastSend - millis();
        Serial.print("  Time remaining: ");
        Serial.print(remainingTime / 60000);
        Serial.print(":");
        remainingTime = (remainingTime % 60000) / 1000;
        if(remainingTime >= 10){
          Serial.println(remainingTime);
        }
        else{
          Serial.print("0");
          Serial.println(remainingTime);
        }
        
      }
      
      return y;
     
    }
    

  • Mod

    @dpcr Very nice project and creative use of a magnetometer!
    These things are very cheap and able to solve far more problems than just telling in what direction you're heading ;-)

    Could you post a shot of some of the sine data from the serial plotter? Just wondering what it looks like.
    Any idea about the resolution you get with this method? You describe half a sine to be divided in 10 steps, but how much gas usage does one sine represent?



  • Hello,

    i have the same Project, i use a HMC5983 magnetometer.
    i'm testing now, but it looks good.

    regards
    ThomasD
    0_1486845404196_IMG_20170211_213140~2.jpg



  • Hello,

    here my sketch to form a puls:

    /***************************************************************************
      This is a library example for the HMC5883 magnentometer/compass
    
      Designed specifically to work with the Adafruit HMC5883 Breakout
      http://www.adafruit.com/products/1746
     
      *** You will also need to install the Adafruit_Sensor library! ***
    
      These displays use I2C to communicate, 2 pins are required to interface.
    
      Adafruit invests time and resources providing this open source code,
      please support Adafruit andopen-source hardware by purchasing products
      from Adafruit!
    
      Written by Kevin Townsend for Adafruit Industries with some heading example from
      Love Electronics (loveelectronics.co.uk)
     
     This program is free software: you can redistribute it and/or modify
     it under the terms of the version 3 GNU General Public License as
     published by the Free Software Foundation.
     
     This program is distributed in the hope that it will be useful,
     but WITHOUT ANY WARRANTY; without even the implied warranty of
     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
     GNU General Public License for more details.
    
     You should have received a copy of the GNU General Public License
     along with this program.  If not, see <http://www.gnu.org/licenses/>.
    
     ***************************************************************************/
    
    #include <Wire.h>
    #include <Adafruit_Sensor.h>
    #include <Adafruit_HMC5883_U.h>
    
    /* Assign a unique ID to this sensor at the same time */
    Adafruit_HMC5883_Unified mag = Adafruit_HMC5883_Unified(12345);
    
    unsigned long t = 0;
    unsigned long counter = 0;
    bool trigger = 0;
    float out = 0.0;
    
    void setup(void) 
    {
      Serial.begin(115200);
      //Serial.println("HMC5883 Magnetometer Test"); Serial.println("");
      
      /* Initialise the sensor */
      if(!mag.begin())
      {
        /* There was a problem detecting the HMC5883 ... check your connections */
        Serial.println("Ooops, no HMC5883 detected ... Check your wiring!");
        while(1);
      }
      
      /* Display some basic information on this sensor */
      // displaySensorDetails();
    }
    
    void loop(void) 
    {
      /* Get a new sensor event */ 
      sensors_event_t event; 
      mag.getEvent(&event);
     
      /* Display the results (magnetic vector values are in micro-Tesla (uT)) */
      out = event.magnetic.y;
      if (out > -0.7 && trigger == 0)
      {
      trigger = 1;
      Serial.println(trigger);
      }
      if (out < -110.5 && trigger == 1)
      {
      trigger = 0;
      counter++;
      Serial.println(trigger);
      }
      
      if (millis() > (t+5000))
       {
       Serial.println(trigger);
       t = millis();
       }
     
      
      delay(250);
    }
    

    And here the Output, 1 Puls is 0.001 m3, first at slow flow and then with my max.

    0_1486850133246_gas-002.png

    regards
    ThomasD


  • Mod

    @ThomasDr do I understand it correctly that you get a full sine swing for each 0.001 m3 of gas usage?



  • @Yveaux ! The picture shows the sine wave produced by the HMC5883L. The orange line is the Y axis, which is the one I am using. The large visible sine wave is when the gas is flowing for the furnace. When the gas stops the line is creeping upwards, this is the pilot light on the water heater.
    0_1486851169495_Untitled1.png



  • @Yveaux Not quite. What I did was look at the actual gas meter and determine how many full "cycles" of the sign wave I received for 2 CuFt. What I got was 16 full cycles per 2 CuFt. I then divided that by 2 to get how many cycles per CuFt (8). Then I divided that by the number of divisions per one full cycle (20), 10 from TOP to BOTTOM and 10 from BOTTOM to TOP. This gave me the the CuFt of gas used per division or pulse. I then converted that to litre. This gave me the 0.17698 per litre. Hope this help and I hope my math is correct.



  • @Yveaux Sorry, it is 1 puls at 0.01 m3
    That depends on the gas meter you use, the BK4 gives a magnetic pulse per 0.01 m3.
    For this puls you can use a reed contact or a magnetometer.
    0_1486857534669_IMG_20170212_005723_649.jpg

    regards
    ThomasD


  • Mod

    @dpcr Thanks for the nice chart!
    It is indeed a clear sine wave. Having the sensor run for a while and checking with the actual meter counts will prove if your math was correct ;-)
    You could also use inverse sine to calculate the angle of the sine pulse, to have linear readout and probably even increase resolution.

    @ThomasDr I'm currently using a pulse counter (reed contact) to count the pulses of my meter (also 100/m3).
    Works well, but the magnetometer has the potential to increase the measurement resolution.


  • Mod

    Sorry for the question, but why would you need a magnetometer to have better resolution in the measurements when you only need to count pulses? Given of course that the reed switch provides accurate readings, because if it doesn't then I understand why to use the magnetometer


  • Mod

    @gohan well, it seems like a magnetometer is able to also give detailed information inbetween the pulses.
    For a pulse counter you only see a complete revolution as a single pules; it doesn't tell you how far the revolution is. The magnetometer creates a sine signal, which indicates where you are on this single revolution.


  • Mod

    I can see that, but looking at the graph if that climbing line is just the consumption of the pilot light, how would you benefit for this added resolution? I am just referring to the fact that this kind of sensor is used to keep track of gas usage during the day and not a live view of the actual flow even for the small amounts: we are talking 0.01 cubic meter for each pulse and a pilot light usually uses 3 cubic meters per hour, so we end up with a pulse every the 10-15 seconds and to me it seems a good resolution, or maybe I just don't see what are other possible benefits.



  • @gohan I had to use a magnetometer because my meter did not have a magnet in it. I tried a reed switch but it never actuated and my meter is outside. So I was stuck using a magnetometer which I mounted in a water proof plastic box. The issue I had was getting the sine wave to create a pulse that was usable for MySensors. I did look at using an inverse sine, or as someone else mentioned, use an exponential moving average of the data but they both failed. When the gas stops flowing, a pilot light as an example as seen in the attached picture, there is little to no gas flowing so the angle doesn't change much. It looks like no gas is flowing when it actually is. I decided to go with dividing the wave or meter movement into separate divisions to create a more accurate way of measuring the gas flow in all flow situations.

    I haven't had a chance to test the accuracy of my math against the gas meter but I will at some point. The issue I'm currently having is that the bellows inside of the meter that the magnetometer is reading do not move at the same rate at all times. This causes the FLOW output to change when I know the gas is flowing at a steady rate. The VOLUME doesn't seem to be effected too much. So what I did was to take a moving average of the last three FLOW readings. This helps a little.

    The hardware and ideas were done by me but the coding was done by my 24 year old son. I always enjoy a challenge. Please feel free to comment.


  • Mod

    Ok, now it makes sense. If you plot the data with the pilot using the gas, don't you still see a sine wave but with, of course, a longer period?


  • Mod

    @gohan I assume you do. By interpreting the sine and thereby increasing the resolution you might be able to see there's gas flowing, even if the rate is very low.


  • Mod

    good luck with the math then, I am calling myself out! :sweat_smile:
    Back in high school I was quite good at math, but now... :D



  • @gohan Yes you do, but what about the times when there is gas flowing. If I remember correctly there was about 20 or 25 minutes for full sine wave when only the pilot was running. That SEND_FREQUENCY (1500000) was too large when the gas was actually flowing to get an accurate measurement. I just wanted to get an accurate reading of the gas flow at all flow rates. Am I missing something?

    Here are some screenshots from Domoticz for the past week:
    1_1486917738604_Screenshot from 2017-02-12 11-40-09.png 0_1486917738603_Screenshot from 2017-02-12 11-40-26.png )



  • @gohan Hello,
    My thought is simpler.
    The original sensor is too expensive.
    With a Reedcontact I must find the right position.
    The magnetometer I could simply attach somewhere in the proximity.

    regards
    ThomasD


  • Mod

    @ThomasDr
    I see your point, but you pay the simplicity with a more complex coding ;-)



  • @ThomasDr Sorry not to get back to you when I saw your post, your project looks interesting. Does the HMC5983 magnetometer output something similar to the HMC5883L? I just used the HMC5883L magnetometer because I found it useful, I didn't run across it during my research.


Log in to reply
 

Looks like your connection to MySensors Forum was lost, please wait while we try to reconnect.