Posts from the ‘Blog’ Category
A quick project to visualise some of the open data available about the UK train network.
First of all, the result, you should be able to see a map of the UK below, with coloured dots appearing to represent trains as they arrive at stations in realtime.
This requires a browser that supports SVG and websockets, I’ve only tested it works in Firefox and Chrome.
Click for larger version
So, whats going on here?
It starts with the network rail operational data feeds one of which provides a stream of events for train movements (excellently documented by the Open Rail Data wiki). The interesting fields of data are the timestamp of the event, and the planned timestamp of the event, as well as the STANOX code. These allow me to locate the event, and to calculate the delay between when the event should have happened, and when it actually happened.
To display the events on a map the STANOX code needs to be translated into something more useful, to do that I grabbed one of the reference data files described here. I then use that to convert STANOX codes to TIPLOC codes (bear with me). Once I have the TIPLOC code I can find the Easting and Northing of stations using the NaPTAN dataset. This doesn’t cover the locations of all events as it only has public stations and doesn’t include depot/switch points that trains also report at, but it turns out it locates about 70% of the events received, which is more than enough for a visualisation.
Unfortunately Eastings and Northings are not a very useful co-ordinate system, what I really want is WGS84 latitude and longitude (GPS co-ordinates), thankfully Chris Veness of movable-type.co.uk has written conversion functions in javascript.
At this point I’m connecting to the National Rail data feed in node, looking up the location and coverting to the useful lat/long, so I have a realtime stream of train locations and how delayed they are in my console. The next step is to get this into a browser.
I don’t really have any experience with drawing maps in the browser, but my first guess that d3 will be useful proves correct, and I quickly find this extremely useful tutorial: Let’s Make a Map. I take the end result of this tutorial, slightly zoom and recentre the map (the national rail data doesn’t cover Northern Ireland, and not much goes on in the very north of Scotland). The result is a nice looking SVG UK map that I can draw points on using the d3 projection to convert the lat/long to x/y points.
The last thing to link together is to get the data from node to the browser, I wanted to use websockets given the streaming nature of the data, so I grab ws for node. I currently use lighttpd as a webserver, which can’t handle websocket proxying, so I also setup HAProxy in front of lighttpd to pass the websocket requests directly to node.
At this point it all works, I have dots appearing on my map in the browser as trains arrive at stations. To make things a little more interesting I use d3 to create a colour gradient for on time to very delayed trains, and I use a radial gradient with some transparency to make trains slightly colour an area of the map.
The result is what you (hopefully) see above, an animated map that shows locations of trains as they arrive, colour coded by how delayed the train is. The source for this is all available on GitHub.
First of all, most applications for smartphones (in this post smartphone mainly means iPhone or Android) are actually just interfaces (often simplified) to websites, i.e. Twitter, Google Maps, etc. However, both the Android and iPhone appear to fail quite badly at allowing websites to directly integrate with the phone, and I have some thoughts on how they could work better. When thinking about these ideas I am mainly thinking of what features I would like to see when developing phone specific features for Ircster (a fairly major project I am working on that I will eventually get round to writing blog post(s) about), but I think the same features would be useful to a large number of developers and lead to a better platform for users and developers alike.
Looking at the iPhone first, it does allow websites to be directly added as “applications” (applications meaning buttons on the iPhones menu). It also allows some limited customisation of the way the website is displayed when it is accessed as an application using html meta tags (i.e. <meta content="yes" name="apple-mobile-web-app-capable" />). This is a good start, however, the iPhone seems to be completely let down here by not allowing background applications, meaning there is no way to run a website constantly and alert the user on specific events, or receive sensor information (i.e. GPS) while the phone is being used for another purpose.
Android on the other hand supports background applications, however it does not allow (as far as I can tell) for websites to be added as applications, and does not have a way for websites to interact with the phones features. What I would like to see is a JavaScript API allowing phone features such as alerts to be triggered by the website (possibly similar to the message passing feature in Chrome extensions), and also for the phone to reply to the app with sensor information such as GPS location. This seems to fit in very well with the ideas behind Chrome OS and applications being simply websites. (While considering this it occurred to me some of this functionality could be hacked into Android using its current Java app platform, something I have started working on and will demonstrate in another blog post.)
Desktop applications definitely seem to be being replaced by websites, I don’t see why the opposite should be true on phones (websites should not need to write specific phone apps). I’d like to see APIs included with smartphone browsers (Google Gears is a potential start, although this appears to be deprecated on Android) that provide more direct interaction with the phones features (alerts, access to sensors, local storage and local serving of content) and allow complete applications to be developed as websites for multiple platforms.
