Tracking the London Marathon

One of the fun things about being a Googler are the opportunities that come along to work on unofficial ad-hoc projects using a bunch of different Google technologies. It’s even better when you get to do it while helping charity. I got the chance to do just that in the lead-up to the 2010 London Marathon.

About a month ago Phong Luu, Jean-Laurent Wotton and I offered to help Stephen Woodford raise some money for NABS. Stephen volunteered to run the marathon, we pitched in by building an application to let people follow his progress.

Why Build New?

Existing tools such as Latitude and MyTracks didn't offer the real-time, high frequency live updates needed to allow anonymous visitors to track a marathon in real time. The goal was to create a simple web app using Street View and Maps that didn't require a login that got updated as Stephen was running.

What We Used

1. AppEngine was used to create a server to receive position updates during the race.
2. A purpose-built Android app transmitted updates to the server at a regular interval during the race.
3. Google Maps and Street View displayed each updated position along the route.

Practical Considerations

Before Stephen took off there were a few practical considerations. In particular I needed to be wary of two things: battery life and data signal availability.

To maintain battery life we made the frequency and minimum distance of our location update requests user configurable. During testing we found that 5 minute intervals over 1km was a good compromise between real-time updates and battery life.

As any Londoner knows, a 3G data signal isn’t something you can take for granted while running (literally) around the city. To account for this I created an upload Service. Each new location was recorded into a Content Provider along with a timestamp. Using a combination of Alarms and Broadcast Receivers, I could then start the update Service at semi-regular intervals - or when a data connection was available - and upload all the pending updates to the server.

The Future

With the 2010 race now over, the team is looking for ways to improve our custom solution to offer a generic tool for future marathons. To do that we'll need to:

• Update the Android app and AppEngine server to support tracking multiple runners simultaneously and in different races.
• Add compass direction so that Street View shows the direction the runner is facing.

Spring! (or the Return of the Awesome Sunset)

I do love me some Spring, and this Spring I get to enjoy sunsets like these from the comfort of my living room.

Which is the perfect way to end a day spent wandering around the gardens of Cannizaro Park and Wimbledon Common.

Oh, and because it's Spring, Vodafone UK have started selling the Nexus One. Did I mention how much I enjoy Spring?

The Return of the Animal Translator

Earlier this year I was lucky enough to be a part of Google London's "Translate for Animals" April Fool's joke. 

To avoid confusion it got taken down shortly after April Fool's, but in the short time it was live it had tens of thousands of downloads and over 1,000 comments (averaging 5 stars!).

I've gotten some requests for the app to be made available for people to cruelly torment their kids drunk friends, so I've put the app back on the Market as "Animal Translator". Enjoy!

Content Provider Iterator (or Things That Make Me Nervous)

Yesterday Tim Bray, Google's newest Android Developer Advocate, shared his Content Provider Iterator. Before he shared it with you, he showed it to me to check "is this sick and twisted?". Here (with some additional colour) is what I told him.

Let me start with an admission. I have a deep mistrust of metaprogramming tricks in statically-typed systems. I've spent far too much of my working life unpicking frameworks created to simplify complex parts of existing frameworks. Often as not, the class descriptions could be summed up as "I don't understand this way of doing things -- this class makes this framework behave the way I expect."

There's often a good reason the original framework works the way it does. In Android that reason is performance. When developing for servers and PCs simplicity and familiarity is often a good trade-off for a marginal performance hit. In mobile it isn’t.

That said, these sorts of helper classes can improve code readability and cement best practices so I owe it to Tim to justify my unease.

The Problem

First off, let’s look at what we’re trying to solve -- iterating over a Cursor.

Tim’s code lets us do this:

Reader calls = new Reader(Call.class, activity, CallLog.Calls.CONTENT_URI);
for (Call call : calls)
  String number = call.getnumber();

The alternative in “normal” Android is this:

Cursor cursor =   
  getContentResolver().query(CallLog.Calls.CONTENT_URI,
                             null, null, null, null);
while (cursor.moveToNext())
  String number = 
   cursor.getString(cursor.getColumnIndex(CallLog.Calls.NUMBER));
cursor.close();

Certainly Tim’s solution is more familiar to those of us who prefer working with Objects to Cursors, and we save a line of code, but what’s the cost?

A Solution?

The two golden rules when designing for performance in Android are:

• Don't do work that you don't need to do
• Don't allocate memory if you can avoid it

 This pattern looks expensive.

1) Avoid creating objects

Allocating memory (and performing garbage collection in a resource constrained environment is expensive. Every time you extract information from a Cursor Using this class, you're creating a new object for each row, and allocating memory to store every column value too.

2) Avoid Reflection

Wow, that’s a lot of reflection. Reflection is expensive.

For every iteration you’re using reflection to create a new instance of your row object. During the construction of every object you use reflection to find a Setter for each column, then again to invoke the Setter for each column.

That’s a lot of reflection and it won’t take long to feel that hit on your performance.

3) Avoid Method Calls

Method calls are (comparatively) expensive - enough so that it's generally considered good practice in Android to avoid using Getters and Setters within a class. After creating objects you need virtual method calls to allocate each of the row values used, and again to extract them.

4) Minimize frequency of object creation

What's neat about this solution is it's dynamicsm, it creates your new objects each time you to iterate over the list without needing to maintain a separate collection.

But there is a risk here. Because it's dynamic, the generic solution doesn't offer a solution for monitoring changes to objects or the addition or removal of rows. To handle this you'd re-iterate over the list whenever the underlying dataset changes - reinforcing the performance issues described above.

5) Static state and thread safety

As written, this code isn’t thread safe, and maintains the current position of the cursor as a state variable. Using the same Reader to iterate over a cursor in two threads will come to grief very quickly indeed!

Conclusion

The Call Log has a half dozen columns and maybe a hundred rows, so you might get away with this. But what happens when you do it over contacts? Or email? Or Ocado’s entire grocery catalogue? It’s a dangerous pattern because its performance cost is dependent entirely on the shape of your dataset.

Imagine the case for a retailer with a database of several thousand products, each with a dozen String-based columns. Say 2,000 products with 12 string rows. That’s using reflection over 50,000 times and creating nearing 30,000 objects. That’s an awful lot of reflection and object creation to save a line of code.

Different platforms require different thinking. In my experience, becoming comfortable with how these systems are designed to be used is vital to understanding the best ways of implementing solutions. To quote an Android engineer on the subject, “I understand that coming from Ruby, one would want objects generated for you, but there is such a huge performance cost on mobile devices that it simply doesn't work.”