First of all, the result, you should be able to see a map of the UK below, with coloured dots appearing to represent trains as they arrive at stations in realtime.

This requires a browser that supports SVG and websockets, I’ve only tested it works in Firefox and Chrome.

Click for larger version

So, whats going on here?

It starts with the network rail operational data feeds one of which provides a stream of events for train movements (excellently documented by the Open Rail Data wiki). The interesting fields of data are the timestamp of the event, and the planned timestamp of the event, as well as the STANOX code. These allow me to locate the event, and to calculate the delay between when the event should have happened, and when it actually happened.

To display the events on a map the STANOX code needs to be translated into something more useful, to do that I grabbed one of the reference data files described here. I then use that to convert STANOX codes to TIPLOC codes (bear with me). Once I have the TIPLOC code I can find the Easting and Northing of stations using the NaPTAN dataset. This doesn’t cover the locations of all events as it only has public stations and doesn’t include depot/switch points that trains also report at, but it turns out it locates about 70% of the events received, which is more than enough for a visualisation.

Unfortunately Eastings and Northings are not a very useful co-ordinate system, what I really want is WGS84 latitude and longitude (GPS co-ordinates), thankfully Chris Veness of movable-type.co.uk has written conversion functions in javascript.

At this point I’m connecting to the National Rail data feed in node, looking up the location and coverting to the useful lat/long, so I have a realtime stream of train locations and how delayed they are in my console. The next step is to get this into a browser.

I don’t really have any experience with drawing maps in the browser, but my first guess that d3 will be useful proves correct, and I quickly find this extremely useful tutorial: Let’s Make a Map. I take the end result of this tutorial, slightly zoom and recentre the map (the national rail data doesn’t cover Northern Ireland, and not much goes on in the very north of Scotland). The result is a nice looking SVG UK map that I can draw points on using the d3 projection to convert the lat/long to x/y points.

The last thing to link together is to get the data from node to the browser, I wanted to use websockets given the streaming nature of the data, so I grab ws for node. I currently use lighttpd as a webserver, which can’t handle websocket proxying, so I also setup HAProxy in front of lighttpd to pass the websocket requests directly to node.

At this point it all works, I have dots appearing on my map in the browser as trains arrive at stations. To make things a little more interesting I use d3 to create a colour gradient for on time to very delayed trains, and I use a radial gradient with some transparency to make trains slightly colour an area of the map.

The result is what you (hopefully) see above, an animated map that shows locations of trains as they arrive, colour coded by how delayed the train is. The source for this is all available on GitHub.

Having secure passwords for every site you visit is difficult, in reality many people (including me) use the same password for more than one place. This is bad as it increases the potential problem if your password is compromised, for example, if you share your email account’s password with another website, who then (accidentally or intentionally) give away your password, your email is no longer secure. There are a number of methods of reducing these problems and PassHash uses a technique which generates new passwords based on one secure password which you never share directly with anyone. It is worth noting that PassHash is far from unique, and there are many similar tools, as well as alternatives like LastPass, however, none of the alternatives met my requirements, as I will describe below.

PassHash works by taking your master password (which should be a password which is complicated, long and only known by you), and combines it with a memorable key for each website you want to use it on (using the domain name is the default, i.e. amazon.com or ebay.co.uk). This combination is then transformed in a way that cannot be easily reversed (using SHA-1), and modified to start with a lower case letter, followed by a number, followed by an upper case letter, and finally 9 more characters which can be lowercase, uppercase, or numeric. You can then use this password for the website with the key you gave, and know that you will be able to generate it again in the future, but no one else will.

So, why is PassHash better than any of the available alternatives?

• PassHash has no settings (unlike most of the alternatives), this might seem to be a bad thing, but settings are something else to remember, and something that could be wrong by default. PassHash attempts to have sensible defaults, and generate passwords accepted on as many websites as possible. Passwords generated are 12 characters long, and always include at least one of each of lowercase, uppercase and numeric characters. Longer passwords may not be allowed on all websites, and are harder to type if you need to manually enter them. Symbols are often not allowed in passwords and are also hard to type, especially if being input on a phone or other mobile device.
• PassHash is accessible from anywhere, there is a Chrome extension for use on your own computer, as well as a website you can use anywhere. There is also an Android app which allows you to generate passwords without entering your master password on a computer you may not trust. As nothing is stored PassHash also works even when you have no internet connection.
• The PassHash algorithm is simple and the source code is available, there is no need to trust that it generates passwords in a sensible way, you can check for yourself. As well as this, if I were to withdraw the Chrome extension anyone could release software to generate the same passwords.
• PassHash doesn’t store any of your passwords – including your master password – anywhere, locally or remotely. Services such as LastPass are always at risk of attack and by using them you are trusting the LastPass developers have not made any mistakes which could result in your passwords not being properly encrypted.
• PassHash is completely free, and always will be, with no potential for me to change this, as the source code is publicly available and no part of PassHash is provided as a service.
• The PassHash chrome extension can automatically enter generated passwords into text boxes on websites, this avoids both showing the password on the screen and storing the password in the clipboard at any point.

While this all sounds great, PassHash does have a couple of downsides.

• All of your passwords being based on one master passwords requires you to keep that password completely secret, if you compromise that password, none of your passwords are safe. Unlike services such as LastPass if you compromise your master password you will need to change your password on every site you used PassHash to generate passwords for. This means you must be particularly careful when entering your master password on any computer or device you do not fully trust.
• Remembering passwords generated with PassHash is not really an option, which is not necessarily an issue as long as you have ways of accessing your passwords while not at your computer, which is why the website and Android app exist

Website: http://passhash.connorhd.co.uk/
Source code: GitHub

Having recently acquired a Kindle e-book reader I immediately investigated less obvious ways of using it than just reading books. Amazon themselves offer magazines, newspapers and blogs as well as normal books, however blogs particularly seem like something I shouldn’t need to pay for (especially considering I don’t have the 3G Kindle so won’t be making use of the included Whispernet fee). Of course there are a number of 3rd party services for Kindle out there, calibre is an e-book manager that will also download many popular news/magazines and send them directly to your Kindle. There is also kindlefeeder.com which will send aggregated RSS feeds to your Kindle. Other more generic services such as Instapaper also have some form of Kindle support (Instapaper will send unread saved pages as a digest to your Kindle on a regular basis).

Kindlizer is slightly different to all of these services as it attempts to create an interface to be used directly on the Kindle, rather than sending static files to the Kindle to be read later. Amazon have announced a development kit for the Kindle called the KDK to allow development of applications which run on the Kindle, however it is currently in a closed beta and appears to be quite restrictive in terms of free data usage. This means Kindlizer is restricted to the experimental web browser built into the Kindle.

The Kindle web browser is Webkit based which means it has excellent support for web standards, unfortunately in order for the browser to display all kinds of websites it has a less intuitive interface. The directional pad on the Kindle controls a mouse pointer which jumps around in a grid on the screen and snaps to clickable elements (why spacial navigation wasn’t used I have no idea), as well as this the page back and forward keys on the Kindle become page up/page down in the browser, meaning on each press the page is only scrolled a small portion. On top of this many pages are far too large to fit on the Kindle screen and are first shown zoomed out, requiring the user to first select a region to zoom in on.

Kindlizer attempts to fix some of these problems with the use of some JavaScript to trick the browser into showing the content one page at a time, with the page turn buttons on the Kindle moving forward or backward entire pages at once. On top of this CSS styling makes the documents appear with the same formatting as Kindle books, and large areas around clickable elements allows for far easier movement of the mouse cursor. The JavaScript is written to allow for arbitrary blocks of HTML to be split into pages and displayed on the Kindle, currently this is used in conjunction with the Google Feed API to allow users to view RSS feeds on the Kindle. The results so far are shown in the screenshots below:

Kindlizer homepage as viewed on the Kindle

Viewing this blog's RSS feed on Kindlizer

You can visit the (as of writing unfinished) Kindlizer site on Google App Engine here, alternatively there is a demo which shows this blog’s RSS feed and does not require login here. Neither of which work particularly well on any device other than a Kindle (they may kind of work on your browser, but you won’t be able to change page). The full source code for the App Engine site is open source and available here.

To demonstrate here are some screenshots of a sample page using a sample application (if you have an Android phone you can use that link to download it and try it yourself):

The application when you load it:

When clicking go an alert is created:

Viewing the created alert:

The example page is very simple javascript:

function go() {
	window.location = 'android://'+document.getElementById('title').value
		+'/'+document.getElementById('text').value;
}

The source for the app follows and is also fairly simple, most of the code is the rather complex way alerts are generated on Android:

package uk.co.connorhd.android.webapp;
 
import java.net.URLDecoder;
 
import uk.co.connorhd.android.webapp.R;
import uk.co.connorhd.android.webapp.WebApp;
import android.app.Activity;
import android.app.Notification;
import android.app.NotificationManager;
import android.app.PendingIntent;
import android.content.Context;
import android.content.Intent;
import android.os.Bundle;
import android.webkit.WebView;
import android.webkit.WebViewClient;
import android.widget.Toast;
 
public class WebApp extends Activity {
  /** Called when the activity is first created. */
  @Override
  public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    WebView webview = new WebView(this);
 
    // We're testing, clear the cache.
    webview.clearCache(true);
 
    setContentView(webview);
 
    webview.getSettings().setJavaScriptEnabled(true);
 
    final Activity activity = this;
    webview.setWebViewClient(new WebViewClient() {
      int alert = 1;
 
      public void onReceivedError(WebView view, int errorCode,
          String description, String failingUrl) {
        Toast.makeText(activity, "Oh no! " + description,
            Toast.LENGTH_SHORT).show();
      }
 
      public boolean shouldOverrideUrlLoading(WebView view, String url) {
        // Is it a hack?
        if (url.startsWith("android")) {
          String ns = Context.NOTIFICATION_SERVICE;
          NotificationManager mNotificationManager = (NotificationManager) getSystemService(ns);
          int icon = R.drawable.icon;
          CharSequence tickerText = "WebApp Alert!";
          long when = System.currentTimeMillis();
 
          Notification notification = new Notification(icon,
              tickerText, when);
          notification.flags = Notification.FLAG_AUTO_CANCEL;
          Context context = getApplicationContext();
 
          String[] split = url.split("/");
 
          CharSequence contentTitle = URLDecoder.decode(split[2]);
          CharSequence contentText = URLDecoder.decode(split[3]);
 
          Intent notificationIntent = new Intent(activity,
              WebApp.class);
          PendingIntent contentIntent = PendingIntent.getActivity(
              activity, 0, notificationIntent, 0);
 
          notification.setLatestEventInfo(context, contentTitle,
              contentText, contentIntent);
 
          mNotificationManager.notify(alert++, notification);
        } else {
          view.loadUrl(url);
        }
        return true;
      }
    });
 
    webview.loadUrl("http://connorhd.co.uk/files/WebApp.htm");
  }
}

This could easily be extended (and I may try to do this myself for Ircster) to provide all the standard browser functionality (for example the back button doesn’t currently work) and a more complete API to interface with Android.

Looking at the iPhone first, it does allow websites to be directly added as “applications” (applications meaning buttons on the iPhones menu). It also allows some limited customisation of the way the website is displayed when it is accessed as an application using html meta tags (i.e. <meta content="yes" name="apple-mobile-web-app-capable" />). This is a good start, however, the iPhone seems to be completely let down here by not allowing background applications, meaning there is no way to run a website constantly and alert the user on specific events, or receive sensor information (i.e. GPS) while the phone is being used for another purpose.

Android on the other hand supports background applications, however it does not allow (as far as I can tell) for websites to be added as applications, and does not have a way for websites to interact with the phones features. What I would like to see is a JavaScript API allowing phone features such as alerts to be triggered by the website (possibly similar to the message passing feature in Chrome extensions), and also for the phone to reply to the app with sensor information such as GPS location. This seems to fit in very well with the ideas behind Chrome OS and applications being simply websites. (While considering this it occurred to me some of this functionality could be hacked into Android using its current Java app platform, something I have started working on and will demonstrate in another blog post.)

Desktop applications definitely seem to be being replaced by websites, I don’t see why the opposite should be true on phones (websites should not need to write specific phone apps). I’d like to see APIs included with smartphone browsers (Google Gears is a potential start, although this appears to be deprecated on Android) that provide more direct interaction with the phones features (alerts, access to sensors, local storage and local serving of content) and allow complete applications to be developed as websites for multiple platforms.