Home Heating IoT Project - Software planning

For this project I want to be able to control the room heating in my house either manually or on a timed system.

I decided to plan it using a flowchart, as I couldn't get my head around the programming easily without visualising it.

Here's the first draft and I'm pleased how it turned out:

This image was made at lucidchart.com - so easy to use!

The earlier posts in this project can be found here: http://raspitech.blogspot.com/2016_10_01_archive.html

Electracker - More energy consumption analysis

Now that I have a few weeks of data, I wanted to get daily graphs to look for patterns. I amended the original code to create them:

Not as useful as I'd hoped, but the average consumption per day is definitely interesting.

The coding is pretty terrible, sorry. If I was starting from scratch I'd use datetime module functions to do the date work and include some error handling to stop the program crashing when data is missing for an hour. Because I was pushed for time and had a working program already in place  to modify, it was the quickest solution.

Code, such as it is,  is here:
https://github.com/jcwyatt/electracker/blob/master/elecanalDailyGraphs.py

#program to analyse electricity use over time by hour.

#open a file

#for each hour:
#read it line by line
#bucket into hour
#get total
#get average for each hour

#enhancements to follow: daily charts, date range charts, movie!

import csv
import os
import matplotlib
matplotlib.use('agg')
import matplotlib.pyplot as plt



startDay = 20
startMonth = 12

while True:
 elecByHour=[]
 dailyTotal=0
 dailyReadings=0

 for i in range (0,24):     #for each hour
    with open ('feeds.csv', 'rb') as csvfile:
     rawelecdata = csv.DictReader(csvfile)
     
     j=0
     hourelectot=0

     for row in rawelecdata:
      if int(row['created_at'][11:13])==i and int(row['created_at'][8:10])==startDay and int(row['created_at'][5:7])==startMonth:
       hourelectot += float(row['field1'])
       j +=1
     
    print(i,hourelectot/j,j) #useful to for debugging

    elecByHour.append(hourelectot/j) #add average for the current hour to list

    #calculate average for day:
    dailyTotal = dailyTotal + hourelectot
    dailyReadings = dailyReadings + j


 dailyAverage = dailyTotal/dailyReadings

#plot the graph:

 y = (elecByHour)
 N = len(y)
 x = range(N)
 width = 1/1.5
 plt.xlabel ('Time of Day / hr')
 plt.ylabel ('kW')
 plt.ylim((0,6))
 plt.title('Consumption for '+ str(startDay) + '/' + str(startMonth) + ' Average = ' + str(dailyAverage)+'kW')
 plt.bar(x, y, width, color="blue")
 plt.savefig('elecByHour' + str(startMonth) + str(startDay) + '.png')
 plt.close()
 

 #os.system('xviewer elecByHour.png &')

 startDay += 1

Pi Light and Movie Quote Alarm Clock

Sorry, No documentation on this one. Rough and ready test program.

Turns a blue LED on in the morning (and evening..don't ask) to wake me up, but also plays a random selection from a bank of quotes.  

from gpiozero import LED
from time import sleep
import datetime
import os
import random

led = LED(4)

overRide='auto'
amOn=datetime.time(05,50,00)
amOff=datetime.time(06,30,00)
pmOn=datetime.time(21,30,00)
pmOff=datetime.time(22,45,00)

print (pmOff)
soundTrigger=0
while True:
        while overRide=='auto':

                timeNow=datetime.datetime.now().time()
                soundchoice = str(random.randint(1,5))
                playsound=('aplay '+soundchoice+'.wav')
                if timeNow>pmOn and timeNow<pmOff or timeNow>amOn and timeNow<amOff:
                        led.on()
                        print('led on')
                        soundTrigger+=1
                        if soundTrigger==1:
                                os.system(playsound)
                else:
                        led.off()
                        print('led off')
                        soundTrigger=0
                sleep(60)

Live Energy Consumption in a 'Google Guage'

The author can't find a way to make this visualisation public. Wierd user interface at Thingspeak.

Electracker - Analysis of 2 weeks of data

Earlier this month I got serious about logging my home electrical energy consumption.

The Pi tasked with this job has run continuously for two weeks, happily logging data:

https://thingspeak.com/channels/182833

And is still doing so.

I wanted to analyse this data to get an average picture of the daily consumption, by hour:

A typical weekday looks like this:

The code is here:

https://raw.githubusercontent.com/jcwyatt/electracker/master/elecanal.py

The only thing I'd do to update this is to plonk the overall average consumption for the time period shown as a data box on the graph, e.g. "Average = 0.871 kW" for the main graph.

Circles in Minecraft - Minecraft Pi and Python

Circles in Minecraft?

Using Minecraft Pi, you can write code to place blocks. This means, in theory, you can code objects you could never build accurately - like giant circles.

I had a go at some maths around this in a spreadsheet using the formula for a circle



where x and y are coordinates.

It came out like this:

Which is circular if nothing else.

I don't like the gaps near the axes but with some clever programming (which I'll have to google!) I reckon eventually I could get rid of them.

Trig Method

I realised the numbers above follow 2 overlapping sine waves, so a quick search revealed this:

"A circle can be defined as the locus of all points that satisfy the equations

x = r cos(t)    y = r sin(t)

where x,y are the coordinates of any point on the circle, r is the radius of the circle and 
t is the parameter - the angle subtended by the point at the circle's center."
from  http://www.mathopenref.com/coordparamcircle.html (there's an interactive applet)



So  by carefully choosing the angular interval 't' you should be able to put blocks precisely where you want them in the circle.

Here's my first go in the spreadsheet:

You can see there are still gaps and the numbers in the columns are all over the place. Think the trig functions are working in radians but I entered angles in degrees. 

Using radians instead of degrees:

To plot more points, I'd have to reduce the interval in column 1 to give more points between 0 and 2.

Imagine that in Minecraft! 

I've a Pi 3 that I've just set up to run headless, so I'll move onto that next.....

The Results!

nested loop of expanding rising circles. Feels like an amphitheatre inside.

Simple cylinder - circles on top of circles

This one was weird - a cylinder with different block types in each layer. Unfortunately 2 of them were lava and water.

One massive circle. 

 The code for these:

from mcpi.minecraft import Minecraft
import math

mc = Minecraft.create()
#get players position
x,y,z=mc.player.getPos()

#move out of the way, there's something coming
mc.player.setPos (x,y+50,z)


r=15 #radius factor

for q in range (1,4):  #nested loop for when you want multiple circles
    for i in range(0,r*6):  #sets how many blocks comprise the shape
        j = float(i)/(r*3)  #gives max j of 2 (to give  2*pi radians for trig))

        blockXPos = r*math.cos(j*math.pi)  #sets x coordinate of block
        blockZPos = r*math.sin(j*math.pi)  #sets z coordinate of block

        mc.setBlock(blockXPos+x,y+q,blockZPos+z,1) #places a block in the circle

You could use code like this to plot other mathematical functions. Even 3D ones:

Its something called a quadric surface:

See http://tutorial.math.lamar.edu/Classes/CalcIII/QuadricSurfaces.aspx

Something about that makes me feel like I've earned a glass of wine!

Electracker - Domestic Energy Consumption Logger

I've written about this project before but I revived it after a visit to Jersey Tech Fair and meeting the nice chaps at Jersey Electricity. They tried to sell me 'Nest', a home heating automation system (they may have succeeded. Watch this space.) But we also discussed electricity consumption.

I've had a Pi attached to my electricity meter for over a year now, but doing nothing.

Now rekindled, it seems to be happily tracking my energy usage and logging the results at Thingspeak.

I'm going to leave it for a bit and then plan to do a frequency analysis on a few days worth of data.

https://thingspeak.com/channels/182833

I also added this widget to my phone:

https://play.google.com/store/apps/details?id=ua.livi.thingspeakmonitor&hl=en

So now I can see live updates of my energy consumption as long as I have wifi/3G:

The code is here at github:

https://github.com/jcwyatt/electracker/blob/master/electracker_bars.py

Things I've learnt:

1) The wifi connection was really flaky - an Edimax attached to a Pi Zero. I've replaced the Edimax with a bulky Netgear with an ariel but this seems quite erratic too. Tried a better power supply.  Seems slightly better now. Made it a real pain to SSH into the Pi and also writing to Thingspeak.

2) You can't have it all ways (yet). I was initially recording every interval between flashes, and getting a really accurate reading, but couldn't write this to Thingspeak as it took too long and caused missed readings and errors due to the delay.

Now I'm counting the number of flashes in a 2 minute period and writing a calculated consumption based on this to Thingspeak. Advantages are that it give a consistent rate of readings, which looks great on Thingspeak, and at high consumption rates it should be very accurate, albeit for an average consumption over 2 minutes. However at low consumption rates the accuracy drops off; the small number of flashes per 2 minute interval kills the resolution on the measurements.

Solution:


Some kind of hybrid where it records the interval between flashes and averages this out over 2 minutes, and then writes this average data to TS.

Should be possible:

<pseudocode>

while True:
     Reset total time to 0
     Reset flashes to 0
     Wait for a flash
     Start the timer
     For 2 minutes:
            Wait for a flash
            stop the timer
            record the flash interval
            start the timer
            flashes +=1
            total time = total time + flash interval
            wait 0.07s to check the LED is off again
            repeat
      average = totaltime/flashes
      Consumption = 1.125/average

      Write Consumption to Thingspeak
</pseudocode>

Home Heating IoT Project - Interface Designed

The interface is now built:

It uses PHP to write to a CSV and the interface reads from the current settings (in csv format) where present, and set these as the default selections.

Ideally I'd like to use AJAX to update the info live.

The html and css are here at git hub.

It's meant to be a mobile ready design but although it picks up the mobile version in a small desktop browser window, my phone displays only the desktop version.

Home Heating IoT Project - Project Planning

Detailed Requirements:

1) each heater should have 2 programmable time slots, for morning and evening heating

2) it should be possible to turn each heater on or off manually from the interface

3) each heater should have a manual override to turn/stay off or on regardless of program.

4) there should be an override to turn all heaters off.

5) it should be possible to set the heat level (using mark space ratio)

6) it could be possible to link the heat level to outside temp.

7) the program data will be stored on a local csv file (or google sheet?)

8) there will be a php-based interface with the csv file to allow changes to be made, and the current status of the heater to be known.

9) a python program will manage the relays, using the csv file to determine required state of the output.



Schedule data (csv):

sleep (all off)
room1, start1, end1, start2, end2, rm1override on, rm1override off, heatlevel1
room2, start1, end1, start2, end2, rm2override on, rm2override off, heatlevel2
room3, start1, end1, start2, end2, rm3override on, rm3override off, heatlevel3
room4, start1, end1, start2, end2, rm4override on, rm4override off, heatlevel4

Day of Random Pi Stuff

Today's Pi Jobs:

1) Prepare SD card for code club project so webcam can be fitted to the front of CamJam 3 robot. The kids have already built the bot and can remote control it via node-red. I used nginx for the first time with zero problems.

2) Reimage 4 new SD cards with latest Raspbian plus a few tweaks - pibrella and tightvncserver.

3) For home set up a camera on one pi that will send its pictures to another using a bash script and scp. The remote pi is feeding the pictures to the web.

I'll publish more details on these projects later.

Sweet Peas

My wife planted some sweet peas and put them in the conservatory to get started.

I was sorting through my Pi bits and bobs and found the camera module and a project was born.

I made a camera rig from an old bit of wood and a section of bicycle brake cable and used a PoundWorld specs to get the focal length shorter*:

And found the code for the time lapse online here:

(https://picamera.readthedocs.org/en/release-1.10/recipes1.html#capturing-timelapse-sequences)

and only made slight amendments: https://github.com/jcwyatt/sweetpea2016/blob/master/sweetpea.py


After 11 days I copied the files across with scp and deleted all the files smaller than 150k (the dark night time pics) with this great command:

find . -name "*.jpg" -size -150k -delete

Then it was just a matter of using ffmpeg to make the images into a video. A quick search confirmed that it was possible to do this on filenames organised by dates:


ffmpeg -framerate 5 -pattern_type glob -i '*.jpg' -c:v libx264 -r 30 sweetpea01.mp4


5 images per second, 30fps video and it literally took a few seconds to create this:

All done headless and with the command line.

Arduino - Acceleration Due to Gravity Testing Rig

I made the rig in the picture to measure acceleration due to gravity, just for fun.

The hardware is just a home made electromagnet powered through a ULN2003 (linking  2 channels to boost current). This was controlled using an Arduino Nano. I added an LED to indicate when the electromagnet was activated, and a button to activate it. The switch that detected the ball hitting the ground was also home made from a peg, some paperclips and tin foil:

The plan for the program was:

• Push a button on the rig to activate the electromagnet for long enough to attach a bearing but not so long it burns out the hardware (4 seconds seemed about right)
• Switch off the electromagnet automatically and simultaneously start a timer.
• When the pressure switch is activated by the ball hitting the ground, stop the timer.
• Work out the acceleration using a = 2s / t**2 
• (s= distance fallen, t is the time)

I started the code in Ardublock because it avoids typing errors, but the calculations refused to give a valid result even when the timing was giving valid readings. After a frustrating hour of trying to fix it within Ardublock, 3 minutes with the actual code in the Arduino IDE had it working well. The erroneous values are when the ball missed the switch and I had to trigger it manually.

 The code:

https://github.com/jcwyatt/gravityfalls

AM2302 Temperature and Humidity Sensor - Bat Project #01 - Durrell, Jersey

In this post I'll detail how I fared with getting a AM2302 working with the RPi.

This is the first step in a project I'm working on for Durrell Wildlife Conservation Trust. As well as working on a Gorilla feeder with my friend Max, they were also interested in data logging for their bat house:




I bought a sensor from eBay for a few pounds.

From what I can understand, the output from these devices is a digital stream of bits containing the relevant data, but it is not in a regular format like I2C, therefore you have to download a special library. I followed the instructions from Adafruit and it worked really well.

I wasn't that bothered about uploading my data to a google sheet, but since I discovered Thingspeak
a few days ago, I've been itching to try it. It's a great solution for the zoo project because the data goes straight to the web and is formatted and graphed automatically, and can be made public.

I found a great tutorial for linking python to thingspeak here:

http://www.australianrobotics.com.au/news/how-to-talk-to-thingspeak-with-python-a-memory-cpu-monitor

I just adapted the code and added it to the Adafruit example code. Resulting code is messy but worked really well.

I developed this using a model B+ but once it was running I shrunk it down onto a headless Pi Zero.

The thingspeak project site for the trial data (from my lounge) is here:

https://thingspeak.com/channels/103719

And you can embed the graphs using an iframe embed code. I presume this will update as data gets added, but I don't yet know for sure.




Hardware wise I set it up exactly as shown in the diagrams of the tutorial, but added a blue LED between the data and ground pins of the AM2302. It was a lucky guess and works nicely, flashing brightly as data is sent to the pi every minute.

(Sidenote: I've been trying out Node Red recently and I love it, but the instructions I could find online for getting the sensor to work in Node Red seem out of date and I ran into errors trying to install the nodes.)


The code:

#!/usr/bin/python
# Copyright (c) 2014 Adafruit Industries
# Author: Tony DiCola

# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:

# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.

# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
import sys
import httplib, urllib
import Adafruit_DHT
from time import sleep

# Parse command line parameters.
sensor_args = { '11': Adafruit_DHT.DHT11,
    '22': Adafruit_DHT.DHT22,
    '2302': Adafruit_DHT.AM2302 }
if len(sys.argv) == 3 and sys.argv[1] in sensor_args:
 sensor = sensor_args[sys.argv[1]]
 pin = sys.argv[2]
else:
 print 'usage: sudo ./Adafruit_DHT.py [11|22|2302] GPIOpin#'
 print 'example: sudo ./Adafruit_DHT.py 2302 4 - Read from an AM2302 connected to GPIO #4'
 sys.exit(1)

# Try to grab a sensor reading.  Use the read_retry method which will retry up
# to 15 times to get a sensor reading (waiting 2 seconds between each retry).

while True:
 humidity, temperature = Adafruit_DHT.read_retry(sensor, pin)

# Un-comment the line below to convert the temperature to Fahrenheit.
# temperature = temperature * 9/5.0 + 32

# Note that sometimes you won't get a reading and
# the results will be null (because Linux can't
# guarantee the timing of calls to read the sensor).  
# If this happens try again!
#Pushing data to Thingspeak
# python

        if humidity is not None and temperature is not None:
                print 'Temp={0:0.1f}*  Humidity={1:0.1f}%'.format(temperature, humidity)
        else:
                print 'Failed to get reading. Try again!'
                sys.exit(1)


 params = urllib.urlencode({'field1': temperature, 'field2': humidity,'key':'92P5L3PGPTT8ZE8N'})
 headers = {"Content-type": "application/x-www-form-urlencoded","Accept":"text/plain"}
 conn = httplib.HTTPConnection("api.thingspeak.com:80")
 conn.request("POST", "/update", params, headers)
 response = conn.getresponse()
 print response.status, response.reason
 data = response.read()
 conn.close()


 if humidity is not None and temperature is not None:
  print 'Temp={0:0.1f}*  Humidity={1:0.1f}%'.format(temperature, humidity)
 else:
  print 'Failed to get reading. Try again!'
  sys.exit(1)
 
 sleep(60)

Node Red

I've had a play with Node Red for the first time tonight. In a matter of minutes I was able to operate an LED through the Node Red interface, something I'd been wanting to do for a long time.

I couldn't be much simpler to use. I could do quite a lot of things in Node Red after just one evening playing around. The web side of things is still puzzling for now, with websockets and HTTP the next nut to crack.

I reckon I could control my home heating now just using Node Red and the right hardware.

LATER THAT SAME EVENING........

Couldn't let it lie so I had another play. Now the status of the LED can be output to wherever, e.g. twitter:

Output:

Copy and paste the code below if you want to try it (minus the twitter feed) - I used pin 11 (GPIO 17) to drive the LED.

[{"id":"d82e83d.f27d18","type":"inject","z":"2fc3a5ef.d03c5a","name":"LED on","topic":"","payload":"1","payloadType":"string","repeat":"","crontab":"","once":false,"x":295,"y":112,"wires":[["24897f1c.db768","1b8bd14c.e4742f"]]},{"id":"795f97e3.86a068","type":"inject","z":"2fc3a5ef.d03c5a","name":"LED off","topic":"","payload":"0","payloadType":"string","repeat":"","crontab":"","once":false,"x":297,"y":187,"wires":[["24897f1c.db768","1b8bd14c.e4742f"]]},{"id":"24897f1c.db768","type":"rpi-gpio out","z":"2fc3a5ef.d03c5a","name":"Yellow LED","pin":"11","set":true,"level":"0","out":"out","x":497,"y":146,"wires":[]},{"id":"4a42382a.b5bdc8","type":"debug","z":"2fc3a5ef.d03c5a","name":"LED status","active":true,"console":"false","complete":"payload","x":622,"y":286,"wires":[]},{"id":"1b8bd14c.e4742f","type":"change","z":"2fc3a5ef.d03c5a","name":"Create Human Readable Output","rules":[{"t":"change","p":"payload","from":"0","to":"LED is OFF","re":false},{"t":"change","p":"payload","from":"1","to":"LED is ON","re":false}],"action":"","property":"","from":"","to":"","reg":false,"x":356,"y":287,"wires":[["4a42382a.b5bdc8"]]}]

Hacking a ridiculously cheap laser.

I bought a couple of really cheap lasers off of eBay for a project I wanted to try at codeclub (Mission impossible style burglar detector using LDRs and lasers). They were about £1.50 each from China. Like this one but cheaper.

They were not easy to use in the chassis they came in, having a momentary push button switch, and eating batteries, so I set about taking one apart.

The chassis needed a bit of hacksaw work to split, and then the controlling chip and laser came out as one. I jammed on the switch with a cable tie, and taped the +ve wire to the body with insulating tape. The spring on the chip is the -ve:

Not quite sure of the power requirements but I think I would power it through a ULN2003 and an external battery supply if I was to include it in a project. Claims to be 1mW with 3V input. My multimeter read 25mA being supplied from the arduino 3.3V pin.

Not sure what project to include it in.

Robot Chassis - PoliceBot project

My son and I spent a bit of time this evening trying to get a robot chassis working.

My daughter and I built it a week or so ago:

This evening my son and I had a go at programming it to do something:

We used my new favourite programming tool - ardublock:

My son found some blue and white LEDs in my kit box, so we decided to make it into a police car.

It was really easy to get going.

I had a few problems with dodgy connections so we ended up soldering the power connections from the battery to the motors.

The next step is to activate the ultrasonic sensor on the front of the chassis.

Dodgy HC-SR04 Ultrasonic Distance Sensors

I've had a few issues trying get ultrasonic sensors working recently.

I'd had no problems previously when I'd used Flowol software to get one working with the boat project, so initially I suspected it was me trying to use Ardublock that was the problem, or just lack of skilled wiring on my part.

Turns out it was 4 dodgy HC-SR04 I'd bought through Amazon from China.

I built the circuit and tried the Newping sample sketch on one of the Amazon HC-SR04s. No reading on the serial monitor when using the Newping (well actually a continous zero reading)

I tried each of the sensors in turn, and as soon as I put an older SR04 I had lying around in the circuit, I immediately got good numbers in.

God to isolate the problem, but sucks to have to either return the parts or write them off and buy more.

At least knowing I had a good sensors let me try out the Ardublock program again, this time with some success, although it wouldn't read above 20cm.

Arduino with Bluetooth and ArduBlock

My daughter has a holiday project to build a robot, so we are adding Arduino controlled features.

I've tried to get bluetooth working with the arduino before with the boat project. But failed for whatever reason.

I recently discovered Ardublock, which is great way to get a program working without getting bogged down in the typing. It tops S4A because you can upload to the Arduino, and see the actual code it is creating:

This worked well and had the pin 13 LED blinking.

I attached the HC-06 bluetooth chip to the arduino and loaded up the Bluetooth Serial Controller App.

Once the HC-06 was paired (easy to do if you're used to pairing speakers and stuff regularly) the app was able to send '5' to the Arduino and get the LED blinking.

Now do do a few more adventurous things with the robot.

Tide Indicator Pi Project #9 - Calculation of Current Tide Completed (No bugs)

The last version I posted, v2.2, turned out not to work for all tide states, due to some maths errors. These have all been fixed and the code seems to work well.

I've think I've finally got the hang of updating to github, so here's the latest code:

tideproject/tidenow3.0.py

This is the output:

(datetime.datetime(2016, 1, 14, 21, 34, 13, 517280), u'10.5')
(datetime.datetime(2016, 1, 15, 4, 9, 13, 518325), u'1.9')
Tide is currently:  falling
Tidal Range =  -8.6
Current Tide :  9.55460304533


Next job is to have it running continuously and outputting to this webpage.

Tide Indicator Pi Project #8 - Calculation of Current Tide Completed

The program below seems to work!

Output:

('Next: ', (datetime.datetime(2016, 1, 3, 6, 18, 23, 116073), u'4.3'), ' is ', datetime.timedelta(0, 21180, 2472), ' away. /n Previous: ', (datetime.datetime(2016, 1, 2, 23, 49, 23, 115191), u'8.1'), ' was ', datetime.timedelta(0, 2159, 998410), ' ago.')
('Sum of both gaps is ', datetime.timedelta(0, 23340, 882))
('Tide is Currently: ', 'falling')
('tide difference = ', -3.8)
('lower tide value', 4.299999999999999)
('Normalised Time =', 2159, 23340, 0.29060405051843885)
0.958070971113
('Current tide : ', 7.940669690228617)

Code:

#version 1.0
#This program pulls tide data from the ports of Jersey Website
#Under a licence from the UKHO
#
#It then calculates the current tide using a simplified sinusoidal harmonic approximation
#By finding the two tide data points either side of now and working out the current tide height


import urllib2
from bs4 import BeautifulSoup
from time import sleep
import datetime as dt
import math

#open site and grab html

rawhtml = urllib2.urlopen("http://www.ports.je/Pages/tides.aspx").read(40000)
soup = BeautifulSoup(rawhtml, "html.parser")


#get the tide data (it's all in tags)

rawtidedata = soup.findAll('td')


#parse all data points (date, times, heights) to one big list
#format of the list is [day,tm,ht,tm,ht,tm,lt,tm,lt]

n=0
parsedtidedata=[]
for i in rawtidedata: 
 parsedtidedata.append(rawtidedata[n].get_text())
 n += 1

#extract each class of data (day, time , height) to a separate list (there are 10 data items for each day)

tidetimes=[]
tideheights=[]
tideday=[]
lastdayofmonth=int(parsedtidedata[-10])

for n in range(0,lastdayofmonth*10,10):

 tideday.append(parsedtidedata[n])
 tidetimes.extend([parsedtidedata[n+1],parsedtidedata[n+3],parsedtidedata[n+5],parsedtidedata[n+7]])
 tideheights.extend([parsedtidedata[n+2],parsedtidedata[n+4],parsedtidedata[n+6],parsedtidedata[n+8]])

#get time now:

currentTime = dt.datetime.now()


#create a list of all the tide times as datetime objects:

dtTideTimes=[]
tideDataList=[]

for j in range (0,lastdayofmonth*4):
 #print tidetimes[j][0:2], tidetimes[j][3:6]
 if tidetimes[j]=='**':
  dtTideTimes.append('**')
 else:

  dtTideTimes.append(dt.datetime.now().replace(day=int(j/4+1), hour=int(tidetimes[j][0:2]), minute=int(tidetimes[j][3:5])))

 #make a tuple for each data point and add it to a list
 tupleHolder =(dtTideTimes[j], tideheights[j])
 tideDataList.append(tupleHolder)
 
 #print what we've got so far
# print tideDataList[j]

#find the two closest times in the list to now:

gap1 = abs(tideDataList[0][0] - currentTime)
gap2 = abs(tideDataList[0][0] - currentTime)
nearest1 = tideDataList[0]

#print gap1 

for j in range (0,lastdayofmonth*4):

 if (tideDataList[j][0] !="**"):                      
  gapx = abs(tideDataList[j][0] - currentTime) 

#check if the data point is the first or second nearest to now. 
#Generates the datapoints either side of now

  if (gapx <= gap1):                            
   nearest1 = tideDataList[j]            
   gap1 = gapx
  if (gap1 < gapx and gapx <= gap2): 
   nearest2 = tideDataList[j]                   
   gap2 = gapx             

#print (nearest1, gap1)
#print (nearest2, gap2)
#print (gap1+gap2)    

#and now the maths begins
#print ('tide height 1 = ', nearest1[1])
#print ('tide height 2 = ', nearest2[1])

#need to get them in order of time: (this works)

if nearest1[0] > nearest2[0]:
 nextDataPoint = nearest1
 prevDataPoint = nearest2
 gapToNext = gap1
 gapToPrev = gap2

else:
 nextDataPoint = nearest2
 prevDataPoint = nearest1
 gapToNext = gap2
 gapToPrev = gap1

gapSum = gapToNext + gapToPrev

print('Next: ', nextDataPoint,' is ',gapToNext, ' away. /n Previous: ', prevDataPoint, ' was ', gapToPrev, ' ago.')
print('Sum of both gaps is ', gapSum) #this works

#is the tide rising or falling?
tideDifference = float(nextDataPoint[1])-float(prevDataPoint[1])

if (tideDifference<0 data-blogger-escaped-0="prev" data-blogger-escaped-:="" data-blogger-escaped-all="" data-blogger-escaped-code="" data-blogger-escaped-currently:="" data-blogger-escaped-currenttide="" data-blogger-escaped-data="" data-blogger-escaped-difference=", tideDifference) #this works


lowerTide = (float(nearest1[1]) + float(nearest2[1]) - abs(tideDifference))/2
print (" data-blogger-escaped-doesn="" data-blogger-escaped-else:="" data-blogger-escaped-falling="" data-blogger-escaped-for="" data-blogger-escaped-ide="" data-blogger-escaped-is="" data-blogger-escaped-lower="" data-blogger-escaped-lowertide="" data-blogger-escaped-math.cos="" data-blogger-escaped-math.pi="" data-blogger-escaped-normalisedtime="" data-blogger-escaped-ormalised="" data-blogger-escaped-pi="next" data-blogger-escaped-print="" data-blogger-escaped-scaled="" data-blogger-escaped-t="" data-blogger-escaped-this="" data-blogger-escaped-tide="" data-blogger-escaped-tidedifference="" data-blogger-escaped-tidestate="" data-blogger-escaped-time=", gapToPrev.seconds, gapSum.seconds, normalisedTime)

print (math.cos(normalisedTime))

if tideState == " data-blogger-escaped-to="" data-blogger-escaped-urrent="" data-blogger-escaped-value="" data-blogger-escaped-work="" data-blogger-escaped-works="">

Tide Indicator Pi Project #7 - Finding the two tide data points nearest to the current time.

This project is taking ages! I've done a lot since the last post however, but documented very little, so I'll do my best to recall how I got from there to here. You can see all the posts so far here.

The problem in a nutshell: The program needs to get the two tide data points either side of the current time, to work out what the tide is doing now.

Since the last post, the code has been modified to create a list of tuples, with each tuple having two data points (tide time, tide height)

It then works out the gap between each data point and the current time, and tries to store the two nearest times as 'nearest1' and 'nearest2'. Sometime it works:

Time Now:
2015-01-02 16:33

Output:
(datetime.datetime(2016, 1, 2, 17, 52, 40, 854958), u'4.0'),
(datetime.datetime(2016, 1, 2, 11, 9, 40, 854071), u'8.4')

Sometimes it doesn't and misses a point.

#
import urllib2
from bs4 import BeautifulSoup
from time import sleep
import datetime as dt


#open site and grab html

rawhtml = urllib2.urlopen("http://www.ports.je/Pages/tides.aspx").read(40000)
soup = BeautifulSoup(rawhtml, "html.parser")


#get the tide data (it's all in tags)

rawtidedata = soup.findAll('td')


#parse all data points (date, times, heights) to one big list
#format of the list is [day,tm,ht,tm,ht,tm,lt,tm,lt]

n=0
parsedtidedata=[]
for i in rawtidedata: 
 parsedtidedata.append(rawtidedata[n].get_text())
 n += 1

#extract each class of data (day, time , height) to a separate list (there are 10 data items for each day):

tidetimes=[]
tideheights=[]
tideday=[]
lastdayofmonth=int(parsedtidedata[-10])

for n in range(0,lastdayofmonth*10,10):

 tideday.append(parsedtidedata[n])
 tidetimes.extend([parsedtidedata[n+1],parsedtidedata[n+3],parsedtidedata[n+5],parsedtidedata[n+7]])
 tideheights.extend([parsedtidedata[n+2],parsedtidedata[n+4],parsedtidedata[n+6],parsedtidedata[n+8]])

#get time now:

currentTime = dt.datetime.now()


#create a list of all the tide times as datetime objects:

dtTideTimes=[]
tideDataList=[]

for j in range (0,lastdayofmonth*4):
 #print tidetimes[j][0:2], tidetimes[j][3:6]
 if tidetimes[j]=='**':
  dtTideTimes.append('**')
 else:
  dtTideTimes.append(dt.datetime.now().replace(day=int(j/4+1), hour=int(tidetimes[j][0:2]), minute=int(tidetimes[j][3:5])))


#create a tuple of time and height, and add each tuple to a list



 tupleHolder =(dtTideTimes[j], tideheights[j])
 tideDataList.append(tupleHolder)






#print what we've got so far



for j in range (0,lastdayofmonth*4):
 print tideDataList[j]

#find the two closest data points to now in the list:

gap1 = abs(tideDataList[0][0] - currentTime)
nearest1 = tideDataList[0]
print gap1 

for j in range (0,lastdayofmonth*4):


 if (tideDataList[j][0] !="**"):


  gap2 = abs(tideDataList[j][0] - currentTime)
  print tideDataList[j][0], gap2, nearest1


  if (gap2 < gap1):


   nearest2 = nearest1
   nearest1 = tideDataList[j]
   gap1 = gap2

print (nearest1, nearest2)
    
#this nearly works!!! Gave the two nearest high tides, not nearest high and low.

Portable Power

http://uk.rs-online.com/web/p/lithium-rechargeable-battery-packs/7757504/

Powers a Raspberry Pi with 5V for £8 from RS