Have working code that extracts the code and makes lists from it (with the html tags still attached for now):
#A python program to import tide data from a gov.je website
#tidescrape1.0.py - working fine
#It pulls the data in from the gov.je tide site, wkhich is updated daily
#It looks for the class headers associated with date,time and height information
#and then creates a list of these bits of html
#next step - try to extract just the data from current day and tweet it.
import urllib2
import re
from bs4 import BeautifulSoup
#open site
rawhtml = urllib2.urlopen("http://www.gov.je/Weather/Pages/Tides.aspx").read(20000)
soup = BeautifulSoup(rawhtml)
#from http://stackoverflow.com/questions/14257717/python-beautifulsoup-wildcard-attribute-id-search
#get the dates:
tidedates = soup.findAll('td', {'class': re.compile('TidesDate.*')} )
print (tidedates[0])
#get the times:
tidetimes = soup.findAll('td', {'class': re.compile('TidesTime.*')} )
print (tidetimes[0])
#get the heights:
tideheights = soup.findAll('td', {'class': re.compile('TidesHeight.*')} )
print (tideheights[0])
I have a plan to build a live tide indicator. This will lift tide data from the web, interpret and extrapolate it to work out the current tide height, and then output this data live to the web and a physical indicator. This is the data on the UK Hydrographic Office site:
#python program to import tide data from a website
import urllib2
#open site
rawhtml = urllib2.urlopen("http://www.ukho.gov.uk/easytide/EasyTide/ShowPrediction.aspx?PortID=1605&PredictionLength=7").read(20000)
print (rawhtml)
Which collected the text from the site, but it appeared tricky to extract the meaningful data. I googled and found this:http://docs.python-guide.org/en/latest/scenarios/scrape/ But it wasn't to easy to install the lxml module using pip
but this worked: sudo apt-get install python-lxml Next job is to learn how to extract particular bits of data.
By most measures, teaching in general, and teaching computing in particular, is easier and more fun than ever before. A bold statement perhaps? For me, the way technology facilitates community is key.
For starters, social media allows me to join a great club of fellow professionals. Entry criteria: a twitter account and a willingness to learn and share.
With the advent of resource-sharing sites like TES and primaryresources.co.uk, not to mention our own site, there really is no excuse for poorly planned, boring lessons.
If I get stuck and need to learn something new myself, well, just like Neo in the Matrix you can get on Youtube and learn just about anything (it can take a bit longer than it did for Keanu). For example I learned lots of Kodu tricks from @GeekyNicki and returned the favour with some Kodu and Flowol tutorials of my own.
The thermobot also stores and plots 24 hours of data. It's finished its job now, but was logging live every minute to a webpage. You can see its results from my conservatory for 16th April 2015 here:
Initially, pyplot was producing a wierd colouring-in effect under the graph, but adding
plt.clf()
to reset pyplot after each plot fixed the problem. I discovered this accidentally.
the updated code to include the data logging and plotting is here:
import RPi.GPIO as GPIO
import time
import matplotlib
matplotlib.use('agg')
import os
import matplotlib.pyplot as plt
#pseudo code:
# fill a list with 60 zeros
# set the t1_temp variable to 18 deg c
# every minute:
# get the t2_temperature from the chip
# ext:write the temperature to a list or database
# calc the temperatures difference from previous value
# move the bot an amount relative to the temp difference
#initialise bot settings (stepper motor))
GPIO.setmode(GPIO.BCM)
enable_pin = 18
coil_A_1_pin = 7
coil_A_2_pin = 8
coil_B_1_pin = 23
coil_B_2_pin = 24
GPIO.setup(enable_pin, GPIO.OUT)
GPIO.setup(coil_A_1_pin, GPIO.OUT)
GPIO.setup(coil_A_2_pin, GPIO.OUT)
GPIO.setup(coil_B_1_pin, GPIO.OUT)
GPIO.setup(coil_B_2_pin, GPIO.OUT)
GPIO.output(enable_pin, 1)
delay = 2
def forward(delay, steps):
for i in range(0, steps):
setStep(1, 0, 1, 0)
time.sleep(delay)
setStep(0, 1, 1, 0)
time.sleep(delay)
setStep(0, 1, 0, 1)
time.sleep(delay)
setStep(1, 0, 0, 1)
time.sleep(delay)
def backwards(delay, steps):
for i in range(0, steps):
setStep(1, 0, 0, 1)
time.sleep(delay)
setStep(0, 1, 0, 1)
time.sleep(delay)
setStep(0, 1, 1, 0)
time.sleep(delay)
setStep(1, 0, 1, 0)
time.sleep(delay)
def setStep(w1, w2, w3, w4):
GPIO.output(coil_A_1_pin, w1)
GPIO.output(coil_A_2_pin, w2)
GPIO.output(coil_B_1_pin, w3)
GPIO.output(coil_B_2_pin, w4)
#from http://www.modmypi.com/blog/ds18b20-one-wire-digital-temperature-sensor-and-the-raspberry-pi:
#load drivers for temperature probe
os.system('modprobe w1-gpio')
os.system('modprobe w1-therm')
temp_sensor = '/sys/bus/w1/devices/28-0414607111ff/w1_slave'
#Get the raw data from the sensor:
def temp_raw():
f = open(temp_sensor, 'r')
lines = f.readlines()
f.close()
return lines
def read_temp():
lines = temp_raw()
while lines[0].strip()[-3:] != 'YES':
time.sleep(0.2)
lines = temp_raw()
temp_output = lines[1].find('t=')
if temp_output != -1:
temp_string = lines[1].strip()[temp_output+2:]
temp_c = float(temp_string) / 1000.0
return temp_c
#initialise current temperature
currentTemp = 14.00
#make an empty list for 24 hours worth of data
tempLog = [0] * (60*24)
#run the bot
while True:
oldTemp = currentTemp
currentTemp = read_temp()
tempDiff = currentTemp - oldTemp
if tempDiff >= 0:
steps = tempDiff * 256
forward(int(delay) / 1000.0, int(steps))
else:
steps = tempDiff * -256
backwards(int(delay) / 1000.0, int(steps))
setStep(0,0,0,0)
#now do the graph stuff:
#add the new reading to the end of the list
tempLog.append(currentTemp)
#delete the first item in the list
del tempLog[0]
#plot the graph as an image
plt.plot(tempLog)
plt.ylabel ('Bedroom Temperature (deg C)')
plt.xlabel ('Time (minutes) (Full width = 24hrs)')
plt.savefig('/var/www/images/tempLog.png')
plt.clf()
hrlog=(tempLog[-60:])
plt.plot(hrlog)
plt.ylabel ('Temp (deg C)')
plt.xlabel ('Time (minutes) last hour')
plt.savefig('/var/www/images/tempLog60.png')
plt.clf()
time.sleep(60)
To establish whether, in the summer, I should heat my hot water tank
using electricity via the immersion heater or use gas via the central
heating boiler.
Hypothesis: Electricity is a more efficient way to heat the water in the
tank because when using gas, there is heat loss via the pipes
transporting the heat from the boiler to the tank and also heat lost
from the boiler flue pipe.
Proposed Method:
Use Raspberry Pi and temperature sensor to log the temperature of the
tank by fitting the sensor to the outside of the tank. Record the value every 2 minutes. By looking for when the
temperature is rising, the length of time for which energy is being
supplied to the tank each day can be measured.
For gas heating, if the quantity of gas used during the same 24hr period is known, the energy consumption can be calculated.
For electrical heating, if the power consumption of the the heater is known then the energy consumption can be calculated.
Other factors should be revealed, for example the rate of heating
for gas compared to electricity and how well the thermostats work at
regulating the temperature.
Step 1: Rig up the pi logging the temperature of the tank with the sensor taped to the outside to gather preliminary data.
draft pseudo code:
#import sqlite to write the temperature and time data to a file
#import matplotlib to write the data
to graphs #import OS
to access the temperature sensor
#if it doesn't exist create the database
#open the database
#read the current temperature and time
#write time and temp to the end of db
#get the data from the db (how much?)
#1) last 24 hours
#2) last 6 hours
#3) last 3 hours
#plot each graph to a .png for showing on a database
#data analysis extension:
#import last 24 hours data (midnight to midnight)
#read temperature values
#if temperature is rising start the timer
#if temperature is falling stop the timer
#might need to average these values out to allow for small fluctuations
#add the timer value to total 'on' time
#return a value for total time of rising temperature in 24 hrs
#calculate the theoretical value of energy used vs actual?
#for electrical this is easier done by the program if the power rating of the heater is known
#for gas would need to input the meter reading for the 24hour period and do some calcs.
