First of all, the result, you should be able to see a map of the UK below, with coloured dots appearing to represent trains as they arrive at stations in realtime.

This requires a browser that supports SVG and websockets, I’ve only tested it works in Firefox and Chrome.

Click for larger version

So, whats going on here?

It starts with the network rail operational data feeds one of which provides a stream of events for train movements (excellently documented by the Open Rail Data wiki). The interesting fields of data are the timestamp of the event, and the planned timestamp of the event, as well as the STANOX code. These allow me to locate the event, and to calculate the delay between when the event should have happened, and when it actually happened.

To display the events on a map the STANOX code needs to be translated into something more useful, to do that I grabbed one of the reference data files described here. I then use that to convert STANOX codes to TIPLOC codes (bear with me). Once I have the TIPLOC code I can find the Easting and Northing of stations using the NaPTAN dataset. This doesn’t cover the locations of all events as it only has public stations and doesn’t include depot/switch points that trains also report at, but it turns out it locates about 70% of the events received, which is more than enough for a visualisation.

Unfortunately Eastings and Northings are not a very useful co-ordinate system, what I really want is WGS84 latitude and longitude (GPS co-ordinates), thankfully Chris Veness of movable-type.co.uk has written conversion functions in javascript.

At this point I’m connecting to the National Rail data feed in node, looking up the location and coverting to the useful lat/long, so I have a realtime stream of train locations and how delayed they are in my console. The next step is to get this into a browser.

I don’t really have any experience with drawing maps in the browser, but my first guess that d3 will be useful proves correct, and I quickly find this extremely useful tutorial: Let’s Make a Map. I take the end result of this tutorial, slightly zoom and recentre the map (the national rail data doesn’t cover Northern Ireland, and not much goes on in the very north of Scotland). The result is a nice looking SVG UK map that I can draw points on using the d3 projection to convert the lat/long to x/y points.

The last thing to link together is to get the data from node to the browser, I wanted to use websockets given the streaming nature of the data, so I grab ws for node. I currently use lighttpd as a webserver, which can’t handle websocket proxying, so I also setup HAProxy in front of lighttpd to pass the websocket requests directly to node.

At this point it all works, I have dots appearing on my map in the browser as trains arrive at stations. To make things a little more interesting I use d3 to create a colour gradient for on time to very delayed trains, and I use a radial gradient with some transparency to make trains slightly colour an area of the map.

The result is what you (hopefully) see above, an animated map that shows locations of trains as they arrive, colour coded by how delayed the train is. The source for this is all available on GitHub.

To demonstrate here are some screenshots of a sample page using a sample application (if you have an Android phone you can use that link to download it and try it yourself):

The application when you load it:

When clicking go an alert is created:

Viewing the created alert:

The example page is very simple javascript:

function go() {
	window.location = 'android://'+document.getElementById('title').value
		+'/'+document.getElementById('text').value;
}

The source for the app follows and is also fairly simple, most of the code is the rather complex way alerts are generated on Android:

package uk.co.connorhd.android.webapp;
 
import java.net.URLDecoder;
 
import uk.co.connorhd.android.webapp.R;
import uk.co.connorhd.android.webapp.WebApp;
import android.app.Activity;
import android.app.Notification;
import android.app.NotificationManager;
import android.app.PendingIntent;
import android.content.Context;
import android.content.Intent;
import android.os.Bundle;
import android.webkit.WebView;
import android.webkit.WebViewClient;
import android.widget.Toast;
 
public class WebApp extends Activity {
  /** Called when the activity is first created. */
  @Override
  public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    WebView webview = new WebView(this);
 
    // We're testing, clear the cache.
    webview.clearCache(true);
 
    setContentView(webview);
 
    webview.getSettings().setJavaScriptEnabled(true);
 
    final Activity activity = this;
    webview.setWebViewClient(new WebViewClient() {
      int alert = 1;
 
      public void onReceivedError(WebView view, int errorCode,
          String description, String failingUrl) {
        Toast.makeText(activity, "Oh no! " + description,
            Toast.LENGTH_SHORT).show();
      }
 
      public boolean shouldOverrideUrlLoading(WebView view, String url) {
        // Is it a hack?
        if (url.startsWith("android")) {
          String ns = Context.NOTIFICATION_SERVICE;
          NotificationManager mNotificationManager = (NotificationManager) getSystemService(ns);
          int icon = R.drawable.icon;
          CharSequence tickerText = "WebApp Alert!";
          long when = System.currentTimeMillis();
 
          Notification notification = new Notification(icon,
              tickerText, when);
          notification.flags = Notification.FLAG_AUTO_CANCEL;
          Context context = getApplicationContext();
 
          String[] split = url.split("/");
 
          CharSequence contentTitle = URLDecoder.decode(split[2]);
          CharSequence contentText = URLDecoder.decode(split[3]);
 
          Intent notificationIntent = new Intent(activity,
              WebApp.class);
          PendingIntent contentIntent = PendingIntent.getActivity(
              activity, 0, notificationIntent, 0);
 
          notification.setLatestEventInfo(context, contentTitle,
              contentText, contentIntent);
 
          mNotificationManager.notify(alert++, notification);
        } else {
          view.loadUrl(url);
        }
        return true;
      }
    });
 
    webview.loadUrl("http://connorhd.co.uk/files/WebApp.htm");
  }
}

This could easily be extended (and I may try to do this myself for Ircster) to provide all the standard browser functionality (for example the back button doesn’t currently work) and a more complete API to interface with Android.

Looking at the iPhone first, it does allow websites to be directly added as “applications” (applications meaning buttons on the iPhones menu). It also allows some limited customisation of the way the website is displayed when it is accessed as an application using html meta tags (i.e. <meta content="yes" name="apple-mobile-web-app-capable" />). This is a good start, however, the iPhone seems to be completely let down here by not allowing background applications, meaning there is no way to run a website constantly and alert the user on specific events, or receive sensor information (i.e. GPS) while the phone is being used for another purpose.

Android on the other hand supports background applications, however it does not allow (as far as I can tell) for websites to be added as applications, and does not have a way for websites to interact with the phones features. What I would like to see is a JavaScript API allowing phone features such as alerts to be triggered by the website (possibly similar to the message passing feature in Chrome extensions), and also for the phone to reply to the app with sensor information such as GPS location. This seems to fit in very well with the ideas behind Chrome OS and applications being simply websites. (While considering this it occurred to me some of this functionality could be hacked into Android using its current Java app platform, something I have started working on and will demonstrate in another blog post.)

Desktop applications definitely seem to be being replaced by websites, I don’t see why the opposite should be true on phones (websites should not need to write specific phone apps). I’d like to see APIs included with smartphone browsers (Google Gears is a potential start, although this appears to be deprecated on Android) that provide more direct interaction with the phones features (alerts, access to sensors, local storage and local serving of content) and allow complete applications to be developed as websites for multiple platforms.

Before reading this post you should probably know what node.js is.

node_debug allows you to evaluate statements and view their result (much like a REPL) from your favourite browser (assuming your favourite browser isn’t IE). It also allows you to browse objects if the result of your statement is an object, for example the global object in node is process so executing process; will return a browsable tree structure of the global scope.

Additionally it exports a log function that allows you to asynchronously output to any active console, for example setTimeout(function () { debug.log("Hello") }, 5000); will output Hello to any open consoles after 5 seconds, allowing useful debug output when using asynchronous functions.

To get started using node_debug you need to have a server running node, and to grab the latest node_debug from github. There is an example.js included to show how you would include it in your own scripts, the idea being you can simply include this in your own project in order to easily debug issues.

Future features I would like to add include:

• A debug.scope object that allows you to import other objects into the debuggers scope (currently only the global scope is available which can cause confusion due to closures in JavaScript hiding a lot of variables from the debugger).
• debug.error/warn/info functions in line with the browser console object provided by firebug.
• Detect disconnection from node and attempt reconnection.
• Caching of asynchronous messages that can be shown when connecting to the console, in order to log things without having an active browser window all the time.