Wind speed monitoring

A blog post especially for my friend Tom.

I have a very minor interest in a cooperative wind farm based in Lincolnshire, who kindly publish the current site wind speed on their website. It’s useful, but there is no historical data so it’s hard to know whether it’s a good day for power generation or whether last week was better or worse than normal.  Time for an inelegant but effective approach using a bit of scripting and cosm

Here’s how:

The wind farm website displays the data in an iframe, so first I had to find the URL that pointed to the source.  Easily done, right click -> this frame -> view source  and make a note of the URL and have look at the code to see where/how the actual wind speed is displayed.

Armed with the above knowledge create a script that will grab the url and using a bit of regex extract the wind speed.  I used PHP as I had the code ready to go from a similar project I’d done in the past. Choose what works for you.

Next push the data up to cosm. Register as a user if you haven’t already,  create a feed,  add a data stream and grab the API key.

Originally I had planned to to use MQTT and I’m already user of Mosquito. There is a well written howto, but I couldn’t get this  to work and didn’t have time to investigate.   So plan B was to use the standard API, which has served me well for sometime.  The standard API simply requires the construction of a curl request that includes your API key, feed number, and the data itself.

The regex and curl request are then glued together with shell script. A cron job then executes every fifteen minutes which I felt was a sensible resolution and hopefully won’t upset the web site owners too much with my regular requests for data.

The results can be seen here.

I freely admit this is not the most efficient way of coding this. With more time I would have written the whole thing in PHP  or a single shell script, but it was quick to do and it works. Sometimes a hack is all you need…

 

 

 

White Hat Rally

Having had a lot of fun last year I’m taking part in the 2012 White Hat Rally.  This year it’s a weekend of fun traveling around the East Midlands with Olympic themed events, along with something for petrol heads and the historic side of IT security.  The chosen charity is a fine choice in the shape of Barnardo’s.  More updates to be posted in the build up and during the event in June. In the meantime please check out the website or the video for more info. Be part of it…

Utilities, Smart grids. and identity – white papers and announcements

Several interesting things today:

1)  Smart Grid Security  – http://www.pikeresearch.com/research/smart-grid-cyber-security
2)  Utilities sector security – http://q1labs.com/resource-center/white-papers/details.aspx?id=99
3)  White House cybersecurity/ trusted identities announcement  expected – http://blogs.csoonline.com/1479/white_house_cybersecurity_announcement_tomorrow

Weather and power monitoring

I’ve spoken to a few people recently about how I monitor my weather sensors and energy usage in my house and felt a write up was in order.  Firstly I should point out that my weather senor monitoring is very amateurish when compared to others who do this for a hobby, but is serves my purposes.

Step 1 – Senors

After a discussion with colleague who runs what I consider a professional setup I settled on Oregon sensors. I brought the WMR80  kit which included remote sensors for wind, rain, temperature  and humidity it was relatively inexpensive and has proved reliable although I’m not sure about the accuracy of the wind sensors.  The senors send a regular reading in the 433Mhz band to a base station in the house, which in itself have been very good at predicting snow for some reason.  In addition I own a Owl/CM119 home energy meter again this transmits on the 433Mhz band and uses the same protocol as the Oregon senors

Step 2 – Automate the reading of sensors

To do anything useful with the senors I need to get their data on to my server.  I discovered the ideal piece of kit for this is the RFXCOM. This handy little device is capable of picking up 433Mhz signals and depending on model will push the data over a USB connection or ethernet.  I chose USB, but would probably go with ethernet if I did this again as it would give me a bit more flexibility on device location.  Not only did this device pick up my senors but a few more in the neighbourhood including some bathroom scales, other temperature senors, and a hot tub sensor !  

The senors readings get pushed out in raw format of the Oregon Scientific protocol, so you need some software to do the translation to a human readable format unless you have lots of time on your hands and like cutting code. There are a few options available, the XPL-Perl suite, Heyu.  plus some more commercial options like Homeseer.  

Initially I used XPL-Perl, but this was a bit of a hack an involved getting specific modules reading the /dev/tty0, putting the output in to a log file, reading a log file etc.  However I was already a Heyu user for a couple of devices I control via X10 and Heyu  had partial support for my sensors. About 6 months into my experiment Heyu introduced support for all my models of sensors. This meant I could assign a senors to an X10 house/unit code and  extract readings with a single command.
 
Step 3 – Output the readings

I run a few scripts to generate various forms of output

1) I send an hourly reading weather sensor readings to twitter – see @empweather
2) Every 2 minutes I take temperature  senors readings from my garden, greenhouse, and garage. I display this data on a private webserver  and use Google’s Visualisation  API to give me some nice display output this.  This data is also sent to a mysql database so  I can look at longer term trends in the future
3) Every 2 minutes I send the external temperate reading and home energy usage to Pachube, this gives a nice simple graph that  allows me to spot short term trends. I can also access this via an iPhone app while on the move

Known issues:
Every now and then I’ll get a “null” reading and need to restart the Heyu process.  This was very frustrating as I was loosing a whole days worth of data on occasions.  I’ve since discovered there is an problem in the linux kernal I am running that causes USB connection to drop.  My server is due an upgrade at some point, so this will  get fixed in due course, in the meantime a house keeping script watches for failures of Heyu and simply restarts.

The Owl is not accurate, in fact it was sensing exactly half my power usage when compared against a single appliance on a single socket meter.  I think this might be to do with AC phases, but for now I can correct this in software <<< Update 5/5/11 – Following a battery change in the sensor and full reset I am now getting accurate readings .

Practical uses
I like things to pass the “so what” test, here are some benefits of this setup:

1) I and others can see what the weather is doing in my home village
2) I’ve been able to manage my electricity usage better and keep my electricity bills down.
3) I know how hot it’s been while I’ve been at work so can decide if I need to water the garden.

Future enhancements
Monitoring mains water usage
Monitoring gas usage
Automated garden watering
Make twitter output more “human” than just figures i.e. it’s hot, cold, warmer than yesterday

Resources:
Heyu – http://www.heyu.org
RFXCOM – http://www.rfxcom.com
Pachube – http://www.pachube.com
Twitter – http://twitter.com/empweather
Weather products – http://www.oregonscientific.co.uk/
Owl/eletrisave http://www.letsautomate.com/12516.cfm
Google Visualisations – http://code.google.com/apis/visualization/documentation/gallery.html

Sample outputs