Massive Earthquake Hammers China

A series of massive Earthquakes hammered China this morning, the largest of which registered 7.8 on the Richter scale.

How big is 7.8? The image below from my Earthquake map mashup shows that the 'rumble zone' (the area over which the quake could be felt) covered all of China.


There's been over a dozen after-shocks registering over 5, so the 'impact' zone is harder to view, but if you look at the 'Australia' tab they're only showing the largest quake which gives a better idea of its scale.

For comparisons sake the 'damage zone' (the area susceptible to severe damage) covers an area about the size of Ohio or Austria. In the image below it's represented by the darker red circle about the size of Thailand.


The humanitarian result of a quake of this size is predictable. You can read more about the effect of a quake of magnitude at Google News.

Announcing 'My Travel Map'

I've always wanted a great big map of the world with pieces of red string tracing my trips from city to city. I always figured it would look cool to see the combination of 'hubs' with lots of lines coming out, then seeing the little snail trails as I trekked across western Europe.

But let's face it red string on a poster map is so 1986. Presenting My Travel Map.


Sign in and start adding your trips on the 'My Travel' tab and watch with barely restrained glee as your global jaunts are plotted automagically. And because a trip worth noting probably has photos, your public, geotagged Picasaweb albums will automagically be added as map markers as soon as you sign in.

While you're there you can record the date of each trip, rate it, add a description, and remember a related link.


As you might have come to expect by now, My Travel Map was written entirely using the Google Mashup Editor.

Sign in to My Travel Map and create your own travel map.

Earthquake! With Damage and Rumble Radii

I visited the Natural History Museum in New York earlier this year and was seriously impressed by the real-time earthquake display. Despite my earnest pleas they cruelly refused to let me take it home with me; instead I stole copied created my own version of their concentric circle filled goodness.

Like the exhibit that inspired it, Worldwide Earthquakes shows not just the epicentres, but also an approximated 'damage zone' (inner circle, dark shading) and 'felt zone' (outer circle, thick border) to give an impression of the areas likely to be affected by each earthquake. Zoom in to see which cities and suburbs will feel the tremor, and which are at risk for property damage.


Red circles show quakes within the last 24 hours, yellow are within the last two days, and grey are everything that's left (up to the last 2 months).

As a bonus, unlike the IRIS/USGS display at the museum, I use multiple data sources to get better world-wide coverage. I include feeds from the Geoscience Australia, the US Geological Survey (USGS), and the European-Mediterranean Seismological Centre.

Big quakes (>5) should be picked up by all the agencies -- but in any case, I've mashed up all three feeds to produce the 'World' tab that shows only earthquakes that people can feel (>3).

Now, if you've got a 30" plasma, you can have you own 'at home' real-time earthquake display. How sweet is that?

Note. The Worldwide Earthquake mashup was constructed using Google's Mashup Editor, with additional data processing done using Yahoo Pipes. Read more information on using Pipes to help construct mashups with GME.

UPDATE (10/08/07): Fixed for IE, circles now draw properly in IE (yay!)

Fuel Price Mapplet: Developing Mapplets vs Mashups

About a week ago Google opened Mapplets to everyone, and now, the WA Petrol Map Mashup is available as a Google Maps Mapplet! Find the cheapest petrol stations while you're getting directions and searching for businesses. Bookmark your commute for easy reference and use waypoints to take a detour to the nearest, cheapest petrol.

Mapplets are found in the 'My Maps' tab on Google Maps when you're signed in to Google. You can overlay multiple Mapplets giving you the power to create a customized Google Maps interface with useful information overlaid over your normal map search results, directions, and business listings.

In Part 1 I'll show you how to use the Petrol Price Mapplet to find the best place to refuel on your commute, developers might want to skip straight to Part 2 where I continue from last week's post on creating a mashup with GME and Pipes, and describe the simple process of creating a mapplet using Yahoo Pipes as a geocoded KML data source.

Part 1: Still in Perth? Still Want Cheap Petrol?

What's better than a mashup with the best petrol prices in your usual suburbs? A mapplet! See where the best petrol prices are on your commute -- and get directions to include the cheapest station on the way.

Login to Google Maps, then add the mapplet using the gallery (or 'add by url' with this url). When it's on your list of mapplets check the checkbox to enable it. You should see something like the image below.

Click 'Change Settings' and chose three suburbs where you normally fill up. When you save your changes, you should see the markers move to show you the cheapest 15 locations from those three suburbs. Clicking a marker will show you the price at that station plus the cheapest overall price.

Now click 'Search Results' and choose the 'directions' tab (under the search box). Enter your home address in one box and your work address in the other and search. You should now see you morning commute with the cheapest petrol stations overlaid on top.




Your commute probably isn't quite right (mine isn't). I have to click 'avoid highways' as I never travel the freeway in rush-hour, then I left-click drag my route from Harbourne over to Pearson -- Google Maps recalculates my journey in real time. I want to check this whenever I need to refuel, so I click 'link to this page' and copy the result from the popup box and paste it back into the address bar; then I bookmark it 'My Daily Commute'. Now whenever I need petrol I just open that link, enable the 'WA Petrol Mapplet' and I can see where to fill up.

Today I see that the station on my way is 14c/l more expensive than the cheapest. Knowing the cheapest isn't far out of my way, I right click the map next to the Karrinyup Caltex and choose 'Add to journey' and move it up the list on the left. Google revises the journey (it's less than 10mins out of my way) and I save 14c/l on filling up.

Part 2: How to Make a Mapplet in Under 2 Hours -- and Why?

Mapplets can be even simpler than mashups, but let's start with why you should bother with a mapplet at all, Mapplets:

• give people access to all the Google Maps functionality (traffic view, street view, directions, drag-and-drop re-routing, etc.) out of the box.
• work as part of Google Maps, so people don't have to login to different websites.
• can be 'layered' so your mapplet can be used in conjunction with other people's (traffic congestion + petrol prices + ?)

That's not to say you should abandon your stand-alone mashup. Leverage the mapplet to drive traffic to it:

• The Mapplet interface is limited. The best use case is to 'enable' the mapplet, then switch back to the search results view. Side-effect: Any and all useful information you have, has to be shown in the marker info-windows (or with the marker itself, try and use customised markers if possible). Use the window to say there's more to see at your full site (and provide a handy link!)
• Your ability to save user data in Mapplets is limited. Stand-alone mashups are much better if you're storing more than simple user preferences.
• Keep it simple. Mapplets work best as useful markers that sit on top of a normal map, like a layer in Google Earth. Keep complex interactions and data presentation to standalone mashups. Remember: A mapplet is a side-order to the map, a mashup is a main course.

With keeping it simple in mind, my mapplet source code is about as basic as you could hope for. Mapplets are 90% Javascript (10% XML). There's no editor or built-in hosting (unlike GME -- maybe coming soon?), but there is a 'scratchpad' that you can should use to develop and test your code. When you're done, you can host it in a variety of Google Places -- Groups, Pages, or the GME itself.

Writing the Mapplet

There's three things I want to achieve in my mapplet, I want

• users to be able to enter a few suburbs to look at on the map (arbitrary number -- 3).
• to display the cheapest (arbitrary number -- 15) petrol stations as markers on the map.
• to indicate the overall cheapest petrol available in those selected suburbs.

User Prefs

Mapplets handle user preferences well. Add require feature="setprefs" within the moduleprefs, then add UserPref definitions, one for each preference:

userpref name="SuburbOne" display_name="Suburb 1" default_value="West Perth" datatype="string"

Then you can access these values like this:
var prefs = new _IG_Prefs(__MODULE_ID__);
var s1 = prefs.getString("SuburbOne");

Show the Petrol Stations

I hate doing XML manipulation in JavaScript. Luckily I don't have to. Mapplets let you pass in a GeoRSS or KML file in and it will automatically plot the placemarks on your map. Use the GeoRSS function like this:

var geoXml = new GGeoXml(kmlFileURLString);

map.addOverlay(geoXml);

Now all you need is a pre-manipulated KML link (here's one I prepared earlier). Once again our friend and saviour -- Yahoo Pipes (how is this not a Google product?!), where I do all my XML manipulation.

I create a new pipe, 'combined petrol prices', that takes three user inputs as input to our existing 'Petrol Price by Suburb' pipe, and uses a 'union module' to combine them. The I sort the result by fuel price and truncate it to 15 items. Then I add a 'Location Extractor'* module (which allows you to export your pipe as a KML file) and I'm done. Three inputs (my user prefs), sorted and truncated to 15 cheapest stations, outputted as KML.

Back in the mapplet I write a bit of JavaScript to construct the URL with the user prefs as parameters for the pipe url. Then I pass this URL into the constructor for a GeoRSS overlay and add the overlay to the map. Unfortunately Pipes doesn't give you the ability to hand tinker with the KML output, which means you can't specify a custom marker / placemark icon in the feed. Rather than forgo the simple / handy GeoRSS functionality I'm going to live with the plain blue markers -- mainly because I'm lazy.

*A note on the Location Extractor Pipe module. It's a little… finicky. The best way I've found to have it reliably parse geolocations is to ensure your feed has 'geo:lat' and 'geo:long' fields. You'll know it works when the output from the Location module includes a populated 'y:location' field.

You can test that the Pipes KML feed is working by running it within Pipes itself, you should see your stations on a Yahoo Map. To double check, just paste the Pipe's KML output address into your Google Maps search box and it will magically render it on the map for you.

Always Show the Cheapest Petrol

I still want to append the cheapest available petrol to each station marker's description.

I create another new Pipe and drop the 'combined' Pipe from above into it (hooking up appropriate user inputs so it takes the same parameters). Then I add a 'foreach annotate' module, and use a new instance of the combined pipe as the source. On the dropdown choose 'first only' -- this will add a new node to each item that contains the first item in the source feed (Ie. The cheapest station). Bring out the regex hammer to nail that info onto the bottom of the description field and I'm done. I Don't even have to touch the mapplet code.

That's It?

Upload your Mapplet XML somewhere, then click 'Add Content' and choose 'add by URL' (next to the search button). Paste in your address and you're done. Once you're happy that everything works, you can submit your Mapplet to the gallery for the world to enjoy.

Here's a few tips:

• Track your mapplet use with Google Analytics (here's how).
• Show your user prefs in you mapplet output panel. If you state what the current settings are, there's a better chance that people will change them to something more suitable.
• Like a mashup, make sure the 'default' result on installation shows something useful / interesting. Most people will add --> judge --> remove, well before they start playing with user preferences.
• Keep the overhead low. Don't add 10,000 markers at a time -- people will turn it off very quickly indeed. If you have a lot of data use the Marker Manager, the Panaramio mapplet is a good example of how and when to do this.
• Host it for free. If you don't have a server handy, you can host your mapplet on a number of Google properties.
• All the data fetching (XML/GeoRSS/HTML) routines within mapplets are cached every 30mins. If you need data fresher than that you'll need to increase that refresh rate. Don't if you don't have to, caching is a Good Thing.

Criticisms, Suggestions, and Conclusions

My Maps / Mapplets in Google Maps is seems based on the idea of layers and 'custom placemarks' in Google Earth, and in a lot of cases it makes a lot more sense to show this layered information in the context of your usual Google Maps use, rather than a collection of standalone mashups.

Things like weather, petrol, road congestion, speed camera locations, and panaramio pictures are excellent examples of data I'd like to see on my Google Map all the time, and together -- once we get full Google Maps as an in-car GPS, the world will truly be a brighter place.

That said, here's some things missing from Mapplets that would be nice to see in future releases:

• No hosting or built in web editor. Let us use the Google Mashup Editor to write, host, and test our mapplets.
• You can't assign custom markers to a KML / GeoRSS feed. Makes sense, as you should be able to put the marker definitions into the feed itself -- but what if you can't? Let us specify a default marker setting.
• Selected Mapplets don't persist. Each time I load Google Maps I need to activate the mapplet / my maps I want to overlay. Lets us persist them between sessions (or at least create a link with them included. I *always* want to have certain mapplets turned on, making people turn them on each time makes them less likely to be used.
• Adsense revenue. Please.
• User settings saving is patchy. Using the 'back' button on your browser almost guarantees the user settings are lost.
• …and one for Yahoo Pipes: Let us tinker with the KML output please!

A Fuel Price Mashup: Web Development Goes Entirely Online

I put this mashup together to play with two new online tools -- the Google Mashup Editor (GME) and Yahoo Pipes. If you're interested in the process (and the effort involved), read-on in part 2 below. If you live in Perth, use the mashup!

Part 1: For Those in WA Who Want Cheap Petrol…

I'm not one for driving 50km out of my way to save 5c/L on petrol, but with unleaded approaching $1.50/L it's probably worth scouting out who's offering it cheapest around where you usually fill up anyway.

Fuel Watch has published fuel prices for years, you've probably seen the updates after the news alerting you to the cheapest petrol 'North of the river' (as though Subiaco was next to Mindarie).



Sign in and create a list of suburbs where you usually 'fillerup', then check which of the stations in your local areas have the best prices on the handy Google Map.

Part 2: ...Piping Chocolate into Your Mashup Peanut Butter

Lunchtime last Monday I got my invite to the Google Mashups Editor (GME) beta (Check out Philipp's introduction), that evening I decided to try it out.

In 48 hours this technology test project completely changed my view of web development. I've written mashups before (Live Cricket, Cricket Venues) but they're pretty basic -- static and read-only, with no personalisation. Worse, each took over a week to finish. This time, before I went to bed on Monday I was done. A couple more hours on Tuesday for the 20% stuff and I'm finished. In less than a day. Done.

The Technology
Overview

Development, debugging, hosting and deployment are done entirely online -- no local environment, editors, or builds. I moved between three different computers during the development and needed only to login to Google and Yahoo to have my development environment ready to go. For me this is a revelation. Gone, the time-wasting installation and configuration of a development environment, no finding, changing, and configuring paths and dependencies.

If you're a coder, chances are you remember the first moment that a change in a line of text affected what happened on the computer in front of you. That shift from passive viewer to active participant is what's driven a lot of us into the world of software development. These tools have given me that same feeling of empowerment over technology for the first time in years.

The Peanut Butter: Google Mashups Editor (GME)

The GME is an online development environment that lets you build Web 2.0 style apps based on RSS feeds. Crucially, it hosts your app and lets you define data structures and hosts read / write data sources on an application wide and a per user basis.

In real-life I'm a C# desktop developer, though I've tinkered in web development and I've used the Google Web Toolkit. The WA Petrol Price Map had a development time, from scratch to working prototype, of about 5 hours, probably a couple more hours for the polish.

A Step-By-Step to Creating a Mashup in GME

(Here’s the source code so you can follow along at home)
Google provide excellent sample projects that you can use as templates. Each editor page is essentially HTML, you use 'tags' to add all the goodies. So I start by laying out my page -- a header and footer, and a table to hold the map and lists.

You use gm:template tags to define data structures and how to display them. You also set the editability (if and how to display the add/edit/delete buttons). So next I create a template for the data to store - a list of suburbs w/ descriptions.

With the data structure defined, I add an instance of this to my UI with a gm:list tag, specifying the template and the data source. In this case I want it stored per user, so I use the ${user} reference. Then I add the petrol price list by adding another gm:list, but this time I don't specify a template, and I point the data source to the address of my RSS feed.

Then I add a gm:map tag for my Google map, and tie its data source directly to the petrol price list data -- ${petrolPriceList}. The map has built in support for goecoded fields, so I just need to specify the feed tags for lat / long.

Now I've got a list of user editable suburbs, a list of the cheapest petrol stations for the suburb selected, and a map of all the stations. The only thing missing is interactivity. GME uses gm:handleEvent tags for this. I tie the map and petrol station lists together by adding a 'select' event handler to both (they share the same data source, a select on one now selects the same item on the other.)

Finally, I make selecting a suburb update the RSS query I use to access the petrol prices. This needs JavaScript, so I add a select handleEvent tag to the suburb list and instruct it to execute a JavaScript function that updates the data source based on my suburb selection.

AND THAT'S IT. Done and dusted. I've seen the future of web development, and this is the way forward.

Some Quick GME Tips & Tricks:

• Use Pipes for RSS feed manipulation, especially anything involving regex. Pipe’s visual interface is a thousand times better than writing JavaScript functions.
• Get usage stats with Google Analytics.
• Make sure users can make some use your mashup without signing in. People are reluctant to sign in without knowing what they're getting first.
• Use a sensible title when you create your new project as that's what will appear in the Google powered login page.
• You can copy a project in its entirety. Useful for using a sample project as a template, or renaming an existing project.
• Host your data in Google Spreadsheets and use the Spreadsheets API for a feed to use in your mashups.
• GME caches requests so you don't hammer your feed providers. This can result in stale data. Add a &frequency= parameter to your feed requests to ensure you get timely data. Concatenate part of the date/time for the update rate you need (Monthly? Daily? Hourly?)
• You can store data either per user (using the ${user} data store) or per application (using ${app}). You can set access levels to application wide stored data to public, read-only, or members’ access only. Create a private mashup for family vacations by adding them all as members and cutting off public access.
• The GME creates a Google Code project for each GME project, including a version controlled SVN repository.
• Use JavaScript for access to the Map functions not exposed in the gm:map tag. (like turning on Traffic View, etc).

…and some existing problems:

• The GME really didn't like either IE6 or IE7. I haven't experimented in FF yet.
• Code formatting was wonky at best, and it was very easy to 'accidentally' delete text by using seemingly innocuous key sequences (shift-home deletes a line!).
• This is definitely still in Beta. The generated code works flawlessly, but the editor itself is buggy as hell (which may be why they recently allowed you to edit offline). Luckily, the team is very receptive to comments and bug reports via the support forums.
• Functionality is lacking, but more (by way of new tags) is on the way. With luck they’ll expand it to provide all the GWT functionality, and eventually make this the backend for Google Page Maker.

The Chocolate: Yahoo Pipes

Yahoo Pipes was a revelation. Manipulating feed items to modify and annotate fields, looking up and incorporating information based on feed item fields is the 'hard work' behind much of mashup development -- really any data driven development. Pipes makes this the easy part.

Pipes lets you make any RSS feed your bitch. Join feeds together, kick their contents with a regex boot, annotate them with lookups from other web services (like Google's geocoder), sort, filter, take user inputs -- whatever. This is real power, and it's implemented simply, visually and hosted entirely by Yahoo. No more spending hours writing and hosting a web service that does the same thing, say goodbye caching and bandwidth issues.

Here's how it worked for me.

The FuelWatch feed takes a suburb name as a parameter and returns an RSS feed in the form, [price]: [Station Name] - [Station Address].

Bring on the Pipes. I specify an input parameter (suburb) and join that with the FuelWatch feed address to generate my input feed.

Everything from here-on acts on every item of the feed, so I start by copying the title field into a new address field. Whack it with a regex stick to trim the address from the title field and clean up the address field. The foreach transaction lets me pass this address to the Google Geocoder, adding the result as a 'geocode' section on each feed item. A little more copying and a little more regex -- voila -- I've got glat and glong fields at the root level of my feed, perfect for a mashup.

Check out the completed pipe. Total creation time including registering with Yahoo and figuring out how to use pipes (and re-learning regex)? 2 hours.

Pipe Tips and Tricks

• Use the 'copy' filter to duplicate fields, then you can perform different actions on each copy.
• Use regex to modify each field until it contains exactly what you want.
• The regex control works sequentially, so you can perform multiple changes on the same field one after the other.
• Use the 'foreach' transform to add the results of a web-based lookup based on a field into each feed item (Eg. Add geocoordinates by passing an address to the Google Geocoder. Add photos by passing tags to Flickr).
• Pipes lets you take CSV, RSS, and ATOM data as feed sources, then outputs it as either RSS or JSON.

The Future of Development?

This time last week I'd started a draft on how the move towards Internet applications was raising barriers for entry into the world of software development. On Monday afternoon my perspective changed.

I develop desktop apps in Windows; have done for the last decade. I can see the world moving to web based applications, but before Monday I was staying in the shallows, debating the value of investing the time and infrastructure necessary to develop a real Web 2.0 app.

See, if you want to write a desktop app all you need is a computer and a compiler. Distribution requires little more than an email account. To write a web app (and widgets != applications), you also need a server, database, and bandwidth. Without any server infrastructure, doing things like saving user data is non-trivial. Even if you host it yourself, you need an Internet connection with suitable bandwidth and a 24/7 server. That's a lot of overhead for a curious 12 year old, an experimenting student living on campus, or a consultant developer working on a laptop.

No longer. Google's Mashup Editor provides a framework for developers to define and save 'per user' and 'per application' data structures and hosts whatever application you produce. It is a Good Thing. Used with a data manipulation framework as powerful as Pipes and you don’t need anything but a computer connected to the Internet to develop and deploy a fully interactive web app without even owning a computer, let alone installing a compiler.

Now your development environment moves with you. If you're computer access consists of a computer you share with your family and the workstation at the school library, you can still write fully featured web apps just by logging in.

I'm still only dipping my foot in the deep end of web development, but if Google and Yahoo continue to develop tools like these, it won't be long before anyone with the desire can dive into full scale web development without a second thought.

Update (18/07/07) Read this new article on how to use Pipes to turn this mashup into an even more useful Mapplet.