Tide Indicator Pi Project #1 - Scraping the tidal data for St Helier - Update

A quick update to post 1 on this project.

This is the data from the Jerseymet site and it looks like it is indexed well and the required data sits in nice blocks.

Tide Indicator Pi Project #1 - Scraping the tidal data for St Helier

I have a plan to build a live tide indicator. This will lift tide data from the web, interpret and extrapolate it to work out the current tide height, and then output this data live to the web and a physical indicator. 

This is the data on the UK Hydrographic Office site:

(http://www.ukho.gov.uk/easytide/EasyTide/ShowPrediction.aspx?PortID=1605&PredictionLength=7)





and here's the html to scrape:


<table border="0" class="HWLWTable first"> <tbody> <tr> <th class="HWLWTableHeaderCell" colspan="4">Sun 26 Jul</th> </tr> <tr> <th class="HWLWTableHWLWCell">HW</th><th class="HWLWTableHWLWCell">LW</th><th class="HWLWTableHWLWCell">HW</th><th class="HWLWTableHWLWCell">LW</th> </tr> <tr> <td class="HWLWTableCell">00:41</td><td class="HWLWTableCell">07:26</td><td class="HWLWTableCell">13:32</td><td class="HWLWTableCell">20:10</td> </tr> <tr> <td class="HWLWTableCell">7.9 m</td><td class="HWLWTableCell">4.2 m</td><td class="HWLWTableCell">8.0 m</td><td class="HWLWTableCell">4.1 m</td> </tr> </tbody></table> <table border="0" class="HWLWTable"> <tbody> <tr> <th class="HWLWTableHeaderCell" colspan="4">Mon 27 Jul</th> </tr> <tr> <th class="HWLWTableHWLWCell">HW</th><th class="HWLWTableHWLWCell">LW</th><th class="HWLWTableHWLWCell">HW</th><th class="HWLWTableHWLWCell">LW</th> </tr> <tr> <td class="HWLWTableCell">02:04</td><td class="HWLWTableCell">08:47</td><td class="HWLWTableCell">14:47</td><td class="HWLWTableCell">21:25</td> </tr> <tr> <td class="HWLWTableCell">8.1 m</td><td class="HWLWTableCell">4.0 m</td><td class="HWLWTableCell">8.4 m</td><td pre=""><span style="font-family: Trebuchet MS, sans-serif;"> </span></td></tr> </tbody></table>  

Tidal info is also avalable here: http://www.portofjersey.je/Pages/tides.aspx

and here*: http://www.gov.je/Weather/Pages/Tides.aspx

*Of these it actually looks like this last one has the neatest, most 'scrapable' formatting and the current days tides will always be at the same position on the page.


To scrape the data I initially tried this: 

#python program to import tide data from a website
import urllib2

#open site
rawhtml = urllib2.urlopen("http://www.ukho.gov.uk/easytide/EasyTide/ShowPrediction.aspx?PortID=1605&PredictionLength=7").read(20000)

print (rawhtml)

Which collected the text from the site, but it appeared tricky to extract the meaningful data.

I googled and found this: http://docs.python-guide.org/en/latest/scenarios/scrape/

But it wasn't to easy to install the lxml module using pip but this worked:

sudo apt-get install python-lxml

Next job is to learn how to extract particular bits of data.