MattCASmith

Last month I introduced my new project for 2019 – building a Raspberry Pi Home Hub with Python. Now it’s time to start coding the modules to generate the information screens, starting with the latest news, weather, and Tube updates.

How the news screen will work

As per the specifications defined in my introductory post, the Raspberry Pi Home Hub’s news and weather screen will fetch and display the following information:

• The latest UK news headlines (via a BBC News RSS feed)
• Weather information for today and tomorrow (via the Met Office’s API)
• London Underground status (via the Transport for London API)

The module’s script will use settings from the config.ini file, including the RSS feed URL, the Met Office API key and location, and the name of the Tube line we want updates on. The file looks like this:

[NEWS]
BBCNewsFeedURL = http://feeds.bbci.co.uk/news/uk/rss.xml
MetOfficeLocationID = 354160
MetOfficeApiKey = XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
TubeLine = northern</pre>

Once it has fetched the necessary data, the script will populate an HTML template file filled with tags that dictate where different variables will be displayed. Here’s an extract to give you a feel for it:

[div class="sidebar_element" style="background-image: url(../imgs/weather/{$weather_today_type})"]
    [h2]{$today_temp}°C[/h2]
    [h3]{$today_day}[/h3]
[/div]

For now we’ll just focus on the Python script that generates the news and weather page, but later we’ll build one that displays it in a web browser and automatically cycles between all the screens we add.

Setting things up

As always, the first step is to import the libraries we’re going to use in our module’s main function. Feedparser, ssl, requests, and json will all help us to fetch and parse the required data from the web. String and datetime will be used to manipulate that data and convert it to the correct format, while the os library will be used to replace any existing HTML file with the latest output.

import feedparser, ssl, requests, json, os, string, datetime

Now let’s populate some basic variables. As I mentioned, the main script fetches certain parameters from a config.ini file and passes them to each module’s function, so we can simply insert these arguments in the right places to build the URLs that the news script will query.

weather_feed_url = "http://datapoint.metoffice.gov.uk/public/data/val/wxfcs/all/xml/" + weather_location_id + "?res=daily&key=" + weather_api_key
tube_feed_url = "https://api.tfl.gov.uk/Line/" + tube_line + "/Status"

For the Met Office’s API, this involves inserting the correct weather location ID for London and the API key I was granted after registering for an account. For updates from Transport for London, it’s as simple as substituting the name of my Tube line into the status URL.

Fetching the news headlines from the BBC

Let’s start with the news. First, we use the ssl and feedparser modules to fetch the BBC News UK RSS feed (again, the URL comes from the config.ini file) and pass the data to the news_data variable.

if hasattr(ssl, '_create_unverified_context'):
    ssl._create_default_https_context = ssl._create_unverified_context
news_data = feedparser.parse(news_feed_url)

Next, we need to fetch the correct parts of each story’s entry in the RSS feed and use them to populate variables. These will eventually be used to fill out our HTML template.

news_story_1_headline = news_data['entries'][0]['title']
news_story_1_thumb = news_data['entries'][0]['media_thumbnail'][0]['url']
news_story_1_summary = news_data['entries'][0]['summary']
news_story_2 = news_data['entries'][1]['title']
...

In the extract above, the number in brackets refers to the story number in the RSS feed (for example, [0] fetches the first story). The various elements referenced (for example, title, media_thumbnail, and summary) are pretty self-explanatory. I’ve repeated this process for six stories, although only the first has a summary and thumbnail URL. You’ll see why when we look at the output HTML file.

Fetching weather information from the Met Office

Now let’s get the latest weather from the Met Office. This is achieved using the requests library and the weather_feed_url variable that we constructed at the beginning of the function.

weather_data = requests.get(weather_feed_url)
weather_data = weather_data.content

The output from this query is kind of awkward, and as you can see below I’ve had to manually extract the required data using the split function. Weather_today_temp is today’s temperature in degrees Celsius and today_weather_type is a number referring to the Met Office’s long list of weather types.

weather_today_temp = weather_data.split('FDm=\"', 1)[1]
weather_today_temp = weather_today_temp.split('\"', 1)[0]
weather_today_type = weather_data.split('W=\"', 1)[1]
weather_today_type = int(weather_today_type.split('\"', 1)[0])

Later, we’ll use weather_today_type to insert an appropriate weather icon. The process above is repeated to populate equivalent variables beginning with weather_tomorrow_. All we need now are the names of the correct days to be displayed alongside this data on the news screen.

today = datetime.date.today()
weather_today_day = (today.strftime("%A"))[0:3]
tomorrow = today + datetime.timedelta(days=1)
weather_tomorrow_day = (tomorrow.strftime("%A"))[0:3]

The code above gets today’s date using datetime, then trims the day’s name to the first three letters (for example, FRI) and stores it in the weather_today_day variable. Again, it repeats the process for weather_tomorrow_day, and we now have all the weather information we need.

Fetching Tube travel updates from Transport for London

Transport for London’s Tube status updates can be retrieved in much the same way as the Met Office weather data, using the requests library and the tube_feed_url we constructed earlier.

transport_data = requests.get(tube_feed_url)
northern_line_status = str(json.loads(transport_data.content))
northern_line_status = northern_line_status.split('u\'statusSeverityDescription',1)[1]
northern_line_status = northern_line_status[5:]
northern_line_status = northern_line_status.split('\'',1)[0]

The data can be manipulated using the split function – statusSeverityDescription is the term we’re interested in here – and trimmed to leave us with a nice string like “Good Service”.

Populating the HTML template

We’ve now fetched all the information we need for the news and weather screen, so it’s time to populate the HTML template. Before that, though, we’ll check the output directory to see if a file called news.htm already exists. If it does, we’ll delete it to make space for our new one.

try:
    os.remove("temp/news.htm")
except:
    pass

Let’s open our template file to read and create a new HTML file, which we’ll open to append to.

template_file = open('html/templates/news.htm', 'r')
html_file = open('temp/news.htm', 'a')

To build the HTML page, we’ll take each line of the template in turn and scan it for dynamic content tags. In the example below, any instance of {$headline_thumb} or {$headline_title} is replaced by the contents of news_story_1_thumb and news_story_1_headline respectively.

for line in template_file:
    if "{$headline_thumb}" in line:
        line = string.replace(line, '{$headline_thumb}', str(news_story_1_thumb))
    if "{$headline_title}" in line:
        line = string.replace(line, '{$headline_title}', str(news_story_1_headline))
...

The only slightly different tag is {$weather_today_type} which populates a CSS background-image URL. In this case, the script also checks the contents of weather_today_type and inserts an appropriate image as the weather icon for that day. For example, a weather_today_type of ``, 1, or 4 (the Met Office’s codes for clear night, sunny day, and not used) will return sun.png from the images folder.

Finally, we write the HTML line – complete with substituted dynamic values – to the output file. When this process has been repeated for every line in the template, the files are closed.

html_file.write(line)
template_file.close()
html_file.close()

There we have it – our news and weather screen HTML file is complete and populated with up-to-date data. We’ll go into how this function is called and how the file is displayed in a future post.

The output

What comes out of the other side? The HTML template, populated by our new Python script and formatted with a CSS stylesheet, displays the latest news, weather, and travel information in an easy-to-read format – and at 800 by 480 pixels, it’s the perfect size for the Home Hub’s screen.

With the news and weather screen sorted, next month I’ll take a look at how to take this information and turn it into a personal email briefing that is automatically sent every morning.

Meanwhile, I’ve just ordered the hardware components to build the Home Hub itself, so I’m looking forward to putting the device together and running my Python on the real thing. Look out for a blog post about that in the near future, and be sure to visit the project page if you didn’t catch the first post.

Photo from rawpixel.com on Pexels