mikejcorey.com

Tutorial part 2: Create beautiful hillshade maps from digital elevation models with GDAL and Mapnik

mike — Fri, 25 Feb 2011 21:31:44 +0000

In part 1 of our GDAL and Mapnik hillshade map tutorial, we used GDAL to convert tiled USGS digital elevation models to a merged GeoTIFF. When also reprojected the map to Mercator and used a California border shapefile to cut out just the state of California.

In this installment, we’ll use the result as a Mapnik layer, and check out some different options for using raster data in your Mapnik maps.

Before we dive in, a note about versions. In this tutorial I’m using Mapnik2, which is now in beta release. I’m doing this both because Mapnik2 adds some exciting new capabilities, and so this tutorial is worth anything for more than a few weeks. Even though Mapnik2 is still beta, I’ve been running it for several weeks now and so far haven’t run into any serious difficulties, so especially if you’re on Mac OS X, I’d recommend giving version 2 a shot.

Even if you do run into problems, the Mapnik listserv is very active, and the developers themselves are very quick to respond with help and to fix bugs. If only every open-source project was so well-maintained!

If you’re not running version 2, a lot of this will still work, though you might need to tweak syntax a little bit. I’ll note anything you just can’t do in Mapnik before version 2.

Data mis en place

Here’s the data we’ll use for this part of the tutorial:

A California border shapefile in Mercator projection (SRID 3395)
A California county/shorelines shapefile (SRID 4326)
A California water shapefile (SRID 4269)
The final GeoTIFF cutout from part 1 (This is an 88 MB file, so don’t download if you don’t have to!)

Now on to the fun!

First, let’s do a basic Mapnik map with our GeoTIFF layer. For many Mapnik maps, Cascadenik makes styling much simpler for anyone familiar with CSS syntax, but currently (February 2011) Cascadenik doesn’t support raster layers. So we’ll start with this XML:

hillshade style gdal /Users/yourname/Documents/gdaltutorial/ca-dem-combined-merc-cutout.tif

Let’s break this down in stages. First, the overall wrapper:

This is where we set the projection of the overall map. Don’t worry, you don’t need to understand what all of it means (although it’s fun to find out!). As you no doubt noticed, this is the same srs data we used to reproject our GeoTIFF in part 1.

One of the great features of Mapnik is that even if the data for separate layers are in different projections, in most cases Mapnik can simply reproject that data on the fly. You can’t reproject raster data within Mapnik, however, so you’ll always want to be sure that you get your GeoTIFF into the projection you want your final map to use before you get to Mapnik (as we did in part 1).

Now let’s drop down a bit to the layer data:

hillshade style gdal /Users/yourname/Documents/gdaltutorial/ca-dem-combined-merc-cutout.tif

This is where we tell Mapnik what type of data we’re using for this layer. Inside the tags, we specify the data type (gdal), and point to the location of our GeoTIFF file. It’s good practice to use the absolute path to your files so if you move your Mapnik files you won’t break the script.

The remaining line inside our tags is the . This tells Mapnik which style definition to use for this layer. So let’s take a look at that style definition.

There’s a lot you can do with raster data, especially in Mapnik2. This style definiton is pretty basic, though. We’re saying that we want the GeoTIFF to be rendered at 100% opacity. Scaling controls how Mapnik renders the image when scaling it down from its original size, and the mode controls how the raster layer is blended with layers below it. Since there’s nothing currently behind the layer, mode is somewhat moot at the moment. Feel free to play with different values for both scaling and mode.

That’s about it. Just make sure your style name matches the tag in your layer.

That’s it for our first, pretty basic GeoTIFF Mapnik map. Save this file as ca-map.xml.

Now we’ll use Python to compile this XML into an image. I highly recommend that once you get the basics down, you simplify your life and download Cascadenik and nik2img to do a little of this coding for you, but at this point you’ve installed enough for one tutorial. So we’ll stick to a basic Python script. Even if you’ve never used Python before, now’s the time to start. Trust me, if you have any programming experience this will make sense.

Open a new text file, and save the following code as ca-stuff-compile.py into the same directory as your XML document:

#!/usr/bin/env python import mapnik2 mapfile = 'ca-stuff.xml' map_output = 'ca-map.png' m = mapnik2.Map(1200, 800) mapnik2.load_map(m, mapfile) bbox = mapnik2.Box2d(mapnik2.Coord(-14000000, 3700000), mapnik2.Coord(-12500000, 5200000)) m.zoom_to_box(bbox) mapnik2.render_to_file(m, map_output)

Right off the bat, if you’re not using Mapnik2 for this, you’ll need to change everthing that says mapnik2 to simply mapnik.

Next we assign some variables. We specify our XML file for Mapnik to chew on, and tell Mapnik what filename to use for the final image.

mapfile = 'ca-stuff.xml' map_output = 'ca-map.png'

Next we instantiate a new mapnik map, and pass our XML to Mapnik’s load_map() function. We also specify the pixel dimensions we want to give our final map, 800 pixels wide by 600 pixels deep.

m = mapnik2.Map(800, 600) mapnik2.load_map(m, mapfile)

Now we tell Mapnik what bounding box we want the map to use. We’ll use this to control how much space will show up around our map content, but you could also specify only a section of the map or any other arbitrary bounding box. But how do we know what to use?

Remember back in part 1 when we used ogrinfo to get the bounding box of our California shapefile that we used to cut out the GeoTIFF? Here’s what that returned:

Extent: (-13849389.898804, 3810165.061203) - (-12704836.275367, 5133847.169980)

So if you want to clip your map at California’s exact boundaries, you can use that extent exactly. But generally we want our maps to have a little padding, so we’ll round up and down a bit:

bbox = mapnik2.Box2d(mapnik2.Coord(-14000000, 3700000), mapnik2.Coord(-12500000, 5200000)) m.zoom_to_box(bbox)

Note that, mercifully, Mapnik uses the same format for its bounding box as ogrinfo. The points represent the lower left and upper right corners of our image.

Finally, the script renders the map to our file:

mapnik2.render_to_file(m, map_output)

If you haven’t already save this file into the same directory as your XML document as ca-stuff-compile.py.

Now, in a terminal window, change into this directory, and let’s build that map:

$ python ca-stuff-compile.py

If you don’t get any error messages, check your directory. You should see a new file called ca-map.png, and hopefully your hillshade map is in the middle.

Neato! Well, sort of – it’s pretty boring, right? Not much different from our original GeoTIFF. Let’s change that now by adding more styles and more map layers to our XML. We’ll add a lot at once here, but not so much conceptually:

polygon style /Users/youruser/Documents/gdaltutorial/ca-mercator/ca-mercator.shp shape hillshade style gdal /Users/youruser/Documents/gdaltutorial/ca-dem-combined-merc-cutout.tif water style /Users/youruser/Documents/gdaltutorial/california_water/california_water.shp shape county style /Users/youruser/Documents/gdaltutorial/california_county_shoreline/california_county_shoreline.shp shape

We’ve added 3 shapefile layers: one in Mercator projection that you may recognize as the state border shapefile we used to cutout the GeoTIFF in part 1:

polygon style /Users/youruser/Documents/gdaltutorial/ca-mercator/ca-mercator.shp shape

You’ll also notice that we’ve put this layer before the GeoTIFF in our XML. This is because we want the state border to be behind the GeoTIFF in the rendered image.

We’ve specified that this layer uses polygon style, so let’s take a look at the polygon style style, defined above the layers:

This one’s pretty simple. We set a fill color for features in this layer using standard Web RGB syntax and Mapnik’s PolygonSymbolizer, and specify that we want 100% opacity for this layer.

Next let’s turn to the style for the GeoTIFF, which we’ve now changed so it will blend with our state border background layer:

We’ve changed the opacity of the GeoTIFF layer to 80%, to keep the level of gray down, and we’ve changed the blend mode to multiply. This tells Mapnik how to blend the GeoTIFF with the underlying layers. There are several other modes you can experiment with, but in most cases you’ll probably want multiply.

Now we’ll add a water layer so we can see some of California’s major lakes, and then county borders. Here’s another great thing about Mapnik: You can use many types of data as layers: shapefiles, PostGIS tables and many other ogr types.

Here’s the top 2 layers:

water style /Users/youruser/Documents/gdaltutorial/california_water/california_water.shp shape county style /Users/youruser/Documents/gdaltutorial/california_county_shoreline/california_county_shoreline.shp shape

The main thing to notice about these layers is that we’ve specified an additional parameter in the tag: srs. We’re doing this because each of these layers is in a different spatial reference system than our main map. The water layer is in SRID 4326, or standard latitude/longitude pairs. The county layer is in SRID 4269, otherwise known as NAD83. As long as you specify the correct srs, Mapnik will do any reprojection needed on the fly (though not on raster layers, remember), so there’s no need to reproject everything into the same projection on your own.

All that’s left is the style definitions for those two layers:

Nothing too fancy here. The county style is using a LineSymbolizer instead of the PolygonSymbolizer we saw earlier. Look – we can use hexadecimal color codes too! And we set the stroke width to 0.4 pixels. The water style colors the water blue — not much more to say than that.

OK, time to render this map. In your Terminal:

$ python ca-stuff-compile.py

If you did everything right, you should have a simple but pretty nice multi-layer Mapnik map.

Since we used a fairly high-resolution GeoTIFF, you should be able to change the image dimensions in ca-stuff-compile.py to make the image quite large for print, HD video, or a poster for your bedroom. But even I don’t do that with my maps!

Unless you want to send me this one.

Tutorial: Create beautiful hillshade maps from digital elevation models with GDAL and Mapnik

mike — Sat, 05 Feb 2011 06:59:37 +0000

Mapnik is your best friend if you want to create map tiles for a slippy map or just want an open-source way to output high-quality printed maps for a wide variety of uses.

Though for your first project I might recommend a somewhat lower-priority project, I like to think I’m the only person to ever use Mapnik to make wedding invitations.

(If I’m wrong, I’d love to see more examples!)

One of Mapnik’s greatest strengths is its ability to combine multiple types of data into a single map: shapefiles, PostGIS data and raster data are all easy to combine.

When I began working with GIS data I started with exclusively vector data, because vector data are so easily scaled to any size and don’t take up much storage space. Vector layers are also easy to color and style, and are great at keeping your thematic maps clean and easy to read.

But sooner or later you’ll want to add some texture to your maps, and it’s time to add elevation models with raster data.

Luckily, there’s lots of raster data available to get started. The USGS publishes digital elevation models (.DEM) of most of the United States (everywhere except Alaska, I believe). This is definitely high-enough resolution for adding texture to a state-level visualization.

Less luckily, the digital elevation models are sliced up into small pieces. This can be nice because it cuts down on file sizes, but if you want a statewide elevation map, you’ll have to put many pieces together.

A statewide elevation model of California, for example, requires merging 70 .dem files at the 2 arc-second resolution.

However, never fear! GDAL is here to help.

GDAL is a powerful, open-source library of translation scripts for raster geographic data.

So with GDAL, a shapefile of California’s border and a set of digital elevation models and Mapnik, it’s straightforward to make a seamless high-resolution .PNG map of California with nice elevation hillshades.

This tutorial isn’t for people without any GIS or programming experience, but if you have a little of both you’ll be fine. This tutorial is for Mac OS X Snow Leopard users, but will probably work for Linux users with a few OS-related tweaks.

Using GDAL requires knowledge of basic GIS concepts and basic command-line programming. I’ve had good luck installing the GDAL Complete package from kyngchaos.com. Scroll down to the “GDAL 1.8 Complete” .dmg installer.

To get started with raster data, you mostly just need to know what a TIF image is.

To do any Mapnik work, you should feel comfortable with:

GIS concepts like map projections
Shapefiles and/or PostGIS
XML
Basic command-line programming

The Mapnik website has detailed installation guides and tutorials on how to use it. There’s an excellent OS X-specific guide to installing Mapnik and dependencies (including GDAL and many other useful spacial libraries) here. There’s many ways to do it and I won’t get into that here. Just make sure you have Mapnik running with GDAL support.

The tutorial is divided into two parts. Part 1, this post, will focus on merging, converting, projecting and clipping the digital elevation models so they can be used with Mapnik. We’ll handle the Mapnik process in part 2.

Data mis en place

Here’s the data we’ll use for this tutorial:

Fun with GDAL

Download and upzip the shapefile and the .dem folder. Then open a terminal window and navigate to wherever you downloaded the shapefile and dem folders.

Then, in the terminal:

$ cd raw-ca-dems

First we’ll merge the digital elevation model tiles into a single .dem file using gdal_merge. We need to specify that we want any areas without data to be white by specifying -init “255″. Then we specify our desired output filename with -o (filename) and then list the files we want to merge together. Like so:

$ gdal_merge.py -init "255" -o ca-dem-combined.dem alturas-e.dem alturas-w.dem bakersfield-e.dem bakersfield-w.dem chico-e.dem chico-w.dem crescent_city-e.dem death_valley-e.dem death_valley-w.dem el_centro-e.dem el_centro-w.dem eureka-e.dem fresno-e.dem fresno-w.dem goldfield-w.dem kingman-e.dem kingman-w.dem klamath_falls-w.dem las_vegas-w.dem long_beach-e.dem long_beach-w.dem los_angeles-e.dem los_angeles-w.dem lovelock-w.dem mariposa-e.dem mariposa-w.dem medford-e.dem medford-w.dem monterey-e.dem monterey-w.dem needles-e.dem needles-w.dem noyo_canyon-e.dem redding-e.dem redding-w.dem reno-e.dem reno-w.dem sacramento-e.dem sacramento-w.dem salton_sea-e.dem salton_sea-w.dem san_bernardino-e.dem san_bernardino-w.dem san_clemente_island-e.dem san_diego-e.dem san_diego-w.dem san_francisco-e.dem san_francisco-w.dem san_jose-e.dem san_jose-w.dem san_luis_obispo-e.dem san_luis_obispo-w.dem santa_ana-e.dem santa_ana-w.dem santa_cruz-e.dem santa_maria-e.dem santa_rosa_island-e.dem santa_rosa-e.dem santa_rosa-w.dem susanville-e.dem susanville-w.dem trona-e.dem trona-w.dem ukiah-e.dem ukiah-w.dem vya-w.dem walker_lake-e.dem walker_lake-w.dem weed-e.dem weed-w.dem

As long as you don’t get any errors, you should now have a new file in your folder called ca-dem-combined.dem. If you have GIS software like Quantum GIS you should be able to add this as a raster layer. If you do open it up, you shouldn’t see any seams between the former tiles — that’s what we want for a good-looking Mapnik layer.

But for this to work in Mapnik, we need a GeoTIFF. To convert the merged .dem to a GeoTIFF, we’ll use gdaldem. The gdaldem command has several modes; we’ll use hillshade to create a shaded relief map, the most common way to visualize texture. We also need to specify the ratio of vertical units to horizontal. Since right now the .dem is in WGS84 projection (standard latitude/longitude), we’ll use degrees, so -s 111120. Then we specify the input file and the output file. Here’s the full command:

$ gdaldem hillshade -s 111120 ca-dem-combined.dem ca-dem-combined.tif

Even if you don’t have a GIS viewer, you should be able to open the .tif file in any image browser like Preview or Photoshop. (Be careful you don’t resave the file, though — most image editors will remove necessary geometric data from the file if you re-save it.) You’ll see a few thin black lines around the outside edge of the old tile data, but don’t worry — we’ll cut this out later.

So now we have a combined .tif, but most of the time we want to display our finished project in Mercator projection — most non-GIS types will think non-Mercator maps look a bit strange. We need to reproject our .tif to Mercator with gdalwarp. We’ll use the -t_srs option to set the target projection, and use the projection specification for the version of Mercator that’s also used by Google Maps and OpenLayers.

$ gdalwarp -t_srs '+proj=merc +lon_0=0 +k=1 +x_0=0 +y_0=0 +ellps=WGS84 +datum=WGS84 +units=m +no_defs' ca-dem-combined.tif ca-dem-combined-merc.tif

If you open up the resulting file, you should see that the projection has changed.

Now we have our final hillshade data, but we want to only show California — not all the extra stuff outside the border. We’ll use our California border shapefile as a mask to cut out just the hillshade data inside the boundary. This is a three-step process.

First, we determine the bounding box, or extent, of our clipping mask — the California border shapefile in this case. We use ogrinfo for this, which comes along with GDAL Complete.

$ ogrinfo -al ../ca-mercator/ca-mercator.shp

This will return a lot of details about the shapefile. But if you scroll back to the top of the output you’ll see:

Extent: (-13849389.898804, 3810165.061203) - (-12704836.275367, 5133847.169980)

These represent the longitude and latitude points (in Mercator projection) of the southwest and northeast corners of our shapefile data.

Next, we trim our merged Mercator .tif to those exact boundaries with gdal_translate.

$ gdal_translate -projwin -13849389.898804 5133847.169980 -12704836.275367 3810165.061203 ca-dem-combined-merc.tif ca-dem-combined-merc-box.tif

Warning: Notice that in the above command we reversed the order of the two y-axis values (5133847.169980 and 3810165.061203) from how they appeared in our ogrinfo query. This is because gdal_translate’s -projwin option is looking for the upper left and lower right coordinates — exactly the opposite of the extent we got from ogrinfo. Annoying, but that’s just the way it is.

Finally, we’ll use the shapefile’s polygon boundaries as a clipping mask on the merged, projected and trimmed .tif, which is now trimmed to show the same area as the shapefile. We’ll use gdalwarp for this, using some different options than before. The -co COMPRESS=DEFLATE option is a generic “creation option” that can have many values. We specify -dstalpha to create a “nodata” band in our resulting .tif, so we are left with a transparent background. And finally we use -cutline to specify a file to use as the clipping mask. As before we then specify our input file and desired output filename.

$ gdalwarp -co COMPRESS=DEFLATE -dstalpha -cutline ../ca-mercator/ca-mercator.shp ca-dem-combined-merc-box.tif ca-dem-combined-merc-cutout.tif

And there’s our textured California border. If you open ca-dem-combined-merc-cutout.tif in an image viewer, you should see a beautifully clipped grayscale image.

In the second half of our tutorial, we’ll use this cutout .tif as a layer in Mapnik. We’re more than halfway there!

Google Maps API hands-on training, part 2

mike — Wed, 10 Mar 2010 04:39:58 +0000

Go back to part one of the training.
Assumed knowledge:
HTML
Moderate to advanced Javascript understanding
CSS
XML
Google Maps API basic skills:

Initialization of map
Adding map controls
Basic adding of points, click event listeners
Display of map points from flat XML

Supporting files:

Starter file, with a basic map, for you to follow along: googlemapstartfile2.html
XML file, which you will want to have locally so you can modify it: basicmapxml.xml
Completed file, if you’re stuck or just want to skip ahead: googlemapspart2-complete.html

Dynamic geocoding

Users often will want to figure out how your data relates to their own geographic location. They want to search against an address or by clicking on a point on the map. To do this kind of functionality you’ll need to use Google’s gecoder function.
1. Create an instance of the geocoder to use for your searches. This should be placed in the code outside of the function that you use to create your map so you can use it anywhere on the page.
```
var objGeocoder = new GClientGeocoder();
```
2. Now let’s put the geocoder to use. Whereas most of our actions are called against the map instance (objMap), in this case we use the geocoder. Lets do a really basic manual address query that in any real situation you’d probably collect from an HTML form. You’ll see that Google takes the address and returns a point — formatted as a lat,long pair — if it finds something, and returns as undefined if it can’t geocode the address. In this case, if it does find an address, we’ll center the map on that point at zoom level 10. This code goes INSIDE the loadMap function, after the map has been instantiated.
```
var strTestAddress = "Your address here";
objGeocoder.getLatLng(strTestAddress, function(objPoint) {
	if (!objPoint) {
		alert(strTestAddress + " not found");
	} else {
		objMap.panTo(objPoint,10);
	}
});
```
3. Another way to react to user activity is to ask Google for a lat-long pair based on a user’s click. This allows a user who doesn’t know the address for their desired location to still use a proximity feature. This involves applying an event listener to the entire map. When a user clicks on the map, we first make sure they’re not clicking on a marker. If they are, we let that marker’s earlier set click handler do its work and do nothing in our function. If it’s not a marker, we re-center the map and do whatever else we might want to. Often we’ll want to pass the geocoded point to another function.
```
GEvent.addListener(objMap, 'click', function(objMarker,objPoint) {
	if (objMarker) {
	}
	else {
		objMap.panTo(objPoint,10);
		var objNewMarker = new GMarker(objPoint);
		objMap.addOverlay(objNewMarker);
		objNewMarker.openInfoWindowHtml("Pssst, pass it on: " + objPoint);
	}
});
```

Polylines and polygons

Markers are the most commonly used overlays in the Google Maps API, but polylines and polygons are next. Polylines are familar to most Google Maps users from the driving directions feature, but you can use them for other applications. Polygons are useful for showing regions and other non-point data.

I know what you’re thinking: STOP! Please don’t think you can throw up a Google map showing all the counties in your state. It will be slow, slow slow — this is a major failing of the Javascript API to me, and is one of the main reasons we use Google Maps for Flash for complex polygon data.

That said, polygons used one at a time or a few at a time work well. Let’s start by drawing a basic line:
1. We’ll start by putting some points into an array:
```
var arrLinePoints = new Array();
arrLinePoints.push(new GLatLng(33.5081, -112.2047));
arrLinePoints.push(new GLatLng(33.5642, -112.1154));
arrLinePoints.push(new GLatLng(33.5608, -112.0042));
arrLinePoints.push(new GLatLng(33.5116, -111.9211));
```
2. Then we’ll spit those points back out as a line. We specify an array that’s the source of the points, the color of the line, the line width, and the opacity of the line (0.1 to 1):
```
var objMyLine = new GPolyline(arrLinePoints, "#446891", 3, 1);
objMap.addOverlay(objMyLine);
```
3. Polygons work much the same, except that the last point in the array should be the same as the first one to close the polygon, and you specify a color and opacity for the fill color in addition to the stroke color and opacity:
```
var arrShapePoints = new Array();
arrShapePoints.push(new GLatLng(33.4623, -112.1086));
arrShapePoints.push(new GLatLng(33.4632, -112.0382));
arrShapePoints.push(new GLatLng(33.4280, -112.0375));
arrShapePoints.push(new GLatLng(33.4285, -112.1075));
arrShapePoints.push(new GLatLng(33.4623, -112.1086));
var objMyPolygon = new GPolygon(arrShapePoints, "#446891", 3, 1, "#99C1D8", 0.2);
objMap.addOverlay(objMyPolygon);
```
Clustering markers

Many times you’ll have a lot of points that at wide zoom levels are too close together to distinguish. Using marker clustering is generally the best way to get around this. Google has built-in clustering functions, but we like one called ClusterMarker better because it has more flexibility. The approach works so well that we use ClusterMarker even when we’re not clustering icons because it makes it easy to trigger clicks to a particular point.

You can download the plugin here:

http://googlemapsapi.martinpearman.co.uk/downloads.php?cat_id=1

And read more about using it here:

http://googlemapsapi.martinpearman.co.uk/articles.php?cat_id=1
1. For today’s example, in your HTML document’s , import the Javascript file that contains the clustering functions.
2. Create a custom icon to be used as your clustering icon. Since this icon has a different size and shape from the other icons from part one, we won’t base this on the baseIcon we created earlier. This should be defined with your other markers, outside the loadMap function
```
var objGroupIcon = new GIcon();
objGroupIcon.image = "http://www.mikejcorey.com/nicar2010/images/severalicons-brown.png";
objGroupIcon.shadow = "http://www.mikejcorey.com/nicar2010/images/severalicons-brown-shadow.png";
objGroupIcon.iconSize = new GSize(41.0, 36.0);
objGroupIcon.shadowSize = new GSize(57.0, 34.0);
objGroupIcon.iconAnchor = new GPoint(20.5, 18.0);
objGroupIcon.infoWindowAnchor = new GPoint(20.5, 18.0);
```
3. Create a new ClusterMarker instance for your map. We’ll also tell ClusterMarker what icon to use to mark a cluster. We can also easily turn the clustering on and off by setting the second line to true or false. Note: For some reason my code display tags are freaking out after this point, so delete any backslashes in the code
```
var objMyCluster;
var arrMarkerStorage = new Array();

objMyCluster = new ClusterMarker(objMap, { clusterMarkerIcon: objGroupIcon });
objMyCluster.clusteringEnabled = true;
```
4. Instead of manually adding a marker using map.addOverlay, when our marker function runs each time we’ll push each marker into the array we just created.
```
function createMarker(objPoint,strHTML,strIconName) {
	var objMarker = new GMarker(objPoint,{ icon:strIconName });
	GEvent.addListener(objMarker, 'click', function() {
		objMarker.openInfoWindowHtml(strHTML);
	});
	arrMarkerStorage.push(objMarker);
	return objMarker;
```
5. Since the ClusterMarker is now going to handle the placement of markers, we can delete or comment out the line inside loadMyLocations that actually places the marker.
```
//objMap.addOverlay(objMarker);
```
6. Once our XML loop has finished running, we’ll tell cluster marker to go to work on the array of points we’ve stored up. Make sure this is outside and after your loop but still inside the loadLocations function.
```
objMyCluster.addMarkers(arrMarkerStorage);
objMyCluster.refresh();
```

Location-aware potholes map lives!

mike — Wed, 10 Mar 2010 02:07:51 +0000

We launched a beta today of our mobile potholes map, which features extensive use of geolocation using the user’s Web browser. This has been on my and James Wilkerson’s plate for a long time, and it feels really good to get at least this version out in the public.

The mobile site is at http://dmreg.com/potholes.

You can see the desktop browser version at DesMoinesRegister.com/potholes, and there’s more info on that project in a previous post.

This is our first extensively location-aware site, and it’s our first really extensive specialty mobile site, so there’s a lot of figuring out still left on this one. But we’re pretty happy with the result.

We used quite a few methods that were different or modified from our other mapping projects.

1. For geolocation, we’re using the W3C geolocation API built in to many phones’ browsers. Safari for iPhone and most Google phones have it, but BlackBerries don’t. Theoretically we’re also supporting Google Gears geolocation, but we haven’t really tested it on mobile devices.

2. We used the Google Static Maps API instead of the more-familiar Javascript API. The Javascript API actually works fairly well, technically speaking, on iPhones and Google phones, but the user experience can quickly become confusing. Dragging things on the map is hard on a mobile device, and zooming in and out on the map often messes with the browser zoom. The static maps API is a lot lighter load for a mobile connection to bear. Almost as a bonus, because I was about ready to write off the BlackBerry browser, using the static API also enables BlackBerry users to use the site (the lack of browser GPS still limits BlackBerry users, but they can at least submit a pothole using an address).

3. We’re using Prototype’s Ajax and JSON to load points and send new pothole data. We’ve done this a few times now, and it’s becoming our standard. JSON is lighter than XML on the server, and you can send data back and forth as Javascript objects, which makes sending complex data much easier (and much easier to modify later).

4. We did this as a website instead of an app in order to work with more platforms at once, and because we’re much better at Web design than objective C (which is to say I know not a bit, and learning will take a while). I’m curious to hear others’ thoughts on this, because I think most journalism shops are going to have a hard time making the economics of iPhone, BlackBerry and Google apps a pretty tough thing to make work. As long as mobile browsers keep developing quickly, I think this is probably the way to go for now, unless you can afford to build your own apps, have a lot more time to devote to one project than we do, or can afford to pay someone else to build them for you.

Check it out, let me know what works and what doesn’t.

Google Maps API hands-on training, part 1

mike — Mon, 08 Mar 2010 03:01:23 +0000

Go to part two of the training
Presumed knowledge:
HTML
Moderate Javascript understanding
CSS
Basic XML
What you’ll need:

A Google Maps API key (one per server, make an include)
A text editor
Some addresses or Lat-Long coordinates
Where to geocode points:
- geocoder.us
- batchgeocode.com (feed it a tab-delimited file of addresses, get one back)
A browser
Firebug is very helpful

Supporting files:

Starter file, with a basic map, for you to follow along: googlemapstartfile.html
Completed file, if you’re stuck or just want to skip ahead: googlemapspart1-complete.html

Getting started:

Make sure you’re using the API appropriately. The good news is that Google has very open terms, but be sure you understand the advertising caveat

Create a basic HTML page

Keep a tab open for the API reference at http://code.google.com/apis/maps/documentation/reference.html

Load your API key inside the of the HTML document

Inside your HTML body, create a target
for your map

Inside your document’s tag, write Javascript to initialize your map after the entire body of the document has loaded

var objMap; //A variable we declare early so our map is accessible outside the function that creates it.
//A function that loads when the HTML page is finished loading, which creates the map
function loadMap() {
	if (GBrowserIsCompatible()) {
		objMap = new GMap2(document.getElementById("map"));
	} else {
		document.getElementById("map").innerHTML = "There's supposed to be a Google Map here, but your browser isn't capable of showing it.";
	}
}

Set a center point (lat-long pair) and zoom level for the map
```
objMap.setCenter(new GLatLng(33.453814, -112.073239), 12);
```

Add this code to the tag of your HTML document to initialize the map once the body has finished loading, and to prevent memory leaks

Add a different map type, set the map to use that type

objMap.addMapType(G_PHYSICAL_MAP);
objMap.setMapType(G_PHYSICAL_MAP);

Add some controls to the map for zooming and changing the map type

objMap.addControl(new GLargeMapControl());
objMap.addControl(new GMapTypeControl());

A marker is one type of overlay. Other overlays include lines, polygons, and ground overlays. There’s several ways to add markers to the map. Let’s start with the simplest method, and add a marker to the map using the default icon.
```
//The simplest way to add a marker
var objBasicPoint = new GLatLng(33.453814, -112.073239);
var objBasicMarker = new GMarker(objBasicPoint);
objMap.addOverlay(objBasicMarker);
```

That’s fine if we don’t want too many markers and we don’t want them to do much, but as soon as you get over about 5 markers, this approach becomes impractical. Writing a function that can be called over and over is a good way to handle multiple markers. Inside the function we’ll also add an event listener to pop up a label (called an InfoWindow) when the user clicks on the icon.

//Our function for creating markers with event listeners
function createMarker(objPoint,strHTML,strIconName) {
	var objMarker = new GMarker(objPoint,{ icon:strIconName });
	GEvent.addListener(objMarker, 'click', function() {
		objMarker.openInfoWindowHtml(strHTML);
	});
	return objMarker;
}

//Adding a marker the more useful way, using a function
var objPoint = new GLatLng(33.479035, -112.047882);
var strWindowContent = "Barrio Cafe\
Azcentral.com raves: Superb Mexican food in a funky setting.\";
var objTestMarker = createMarker(objPoint,strWindowContent);
objMap.addOverlay(objTestMarker);

If you have more than one type of data you want to show on the map, you’ll want to use a different color or shape of icon. You can make custom markers with any .PNG image file with a transparent background. Photoshop will work fine, but Adobe Fireworks is even better for this. We won’t cover creating the icon image here, so we’re starting with an image that we know the pixel dimensions of. For this map we’re going to use two icons that are the same shape, but are different sizes. First we’ll tell Google some key information about both icons. We’ll declare these variables before our other functions and outside of those functions, so we can access the markers from anywhere in the script.
```
//marker definitions
var objBaseIcon = new GIcon();
objBaseIcon.shadow = "http://www.mikejcorey.com/nicar2010/images/shadow-stop.png";
objBaseIcon.iconSize = new GSize(19, 25);
objBaseIcon.shadowSize = new GSize(32, 25);
objBaseIcon.iconAnchor = new GPoint(9, 25);
objBaseIcon.infoWindowAnchor = new GPoint(9, 1);
```

Now we’ll define the different icons, which will be based on our baseicon, and put the icons into an array for easy reference later.

var objRedIcon = new GIcon(objBaseIcon);
objRedIcon.image = "http://www.mikejcorey.com/nicar2010/images/stop.png";

var objGreenIcon = new GIcon(objBaseIcon);
objGreenIcon.image = "http://www.mikejcorey.com/nicar2010/images/start.png";

var arrMapIcons = [objRedIcon,objGreenIcon];

If you want to load marker data from a database, or if you’d like to be able to easily change the markers, you’ll want to load markers dynamically from XML or JSON. In this example we’ll load multiple points from a flat XML file. This process uses a custom asynchronous request from Google called GDownloadUrl.
```
//function for retrieving list of points from XML
function loadMyLocations(strWhichXML) {
	GDownloadUrl(strWhichXML, function(data) {
		var objXML = GXml.parse(data);
		var objLocations = objXML.documentElement.getElementsByTagName("location");

	});
}
```

Set up a basic Javascript loop to get each XML row

for (var numI = 0; numI < objLocations.length; numI++) {
}

For each row, get the attributes you need and run our marker function as before

var objLocationItem = objLocations[numI];
var numMarkerLabel = objLocationItem.getAttribute("label");
var numMarkerIcon = objLocationItem.getAttribute("markertype");
var numMarkerLat = parseFloat(objLocationItem.getAttribute("lat"));
var numMarkerLng = parseFloat(objLocationItem.getAttribute("lng"));
var objPoint = new GLatLng(numMarkerLat, numMarkerLng);
var strWindowContent = "" + numMarkerLabel + "\";
var objMarker = createMarker(objPoint,strWindowContent,arrMapIcons[numMarkerIcon]);
objMap.addOverlay(objMarker);

That’s all for part one. Here’s the code for the entire page:



	
	My awesome first Google map

Go to part two of the training

Frustrated with all things BlackBerry

mike — Wed, 03 Mar 2010 04:01:46 +0000

This week I’m getting to sink my teeth into one of my big priorities for the year: building mobile websites.

Yes, websites and not apps. I’d love to be building phone apps as well, but for now in the interest of speed we’re going with Web pages before apps. (Not that we’re not working on apps either, that’s just a separate topic.)

The good news for most Web designers is that building basic mobile websites is easy: Just write really simple HTML. You can use all the PHP you want since it’s server-side (insert also ASP, Django, your server-side secret weapon of choice). Ems are your friend in mobile Web design: Yes, you finally have to actually use them.

So far this year we’ve knocked out really simple mobile pages for state high school wrestling tournament results and for state high school basketball tournament scores.

As you can see, nothing fancy, but they look pretty, and they work on pretty much every mobile browser we tried them on, smartphone or no.

These were also a bit of R and D for the one I really want to build, which is in progress now: a mobile, location-aware version of our potholes map.

This one is a lot more technically complex, as you might imagine. And as soon as I started researching and testing, I hit a big snag: BlackBerries.

We want to use basic Google Maps API features, so we have to use Javascript and Ajax. Right there is a big problem: Even the newest BlackBerry browsers don’t work with the Google Maps Javascript API.

We also want to include Facebook Connect integration, but the Javascript method won’t work for that on BlackBerries either (to be fair, it’s pretty buggy on iPhones and Android as well, but the FBML all renders, and you can log in with Javascript).

Location awareness in the browser? No problem on iPhones and Android, as long as the phone has GPS. Not so on most BlackBerries.

So that makes three dead-ends today. I figured I could at least download the BlackBerry simulator and figure out what type of site I COULD make work on a BlackBerry with the Google Static Maps API. (This would certainly work, but no moving the point once it’s geocoded, which would be an issue for our purposes)

Oops, the simulator is for Windows only. Does RIM really think there’s only a few Web developers using Macs? Is it even a majority using Windows?

So I’ll just download the simulator on my Virtual Box and run it from there, right? Only if I want to spend 5 minutes waiting for the simulator to actually run, only to give me a connection error and not show any sites when I DO get the browser running.

I. Give. Up.

So can Web developers afford to ignore BlackBerry? Well, not probably not completely, but luckily the number of BlackBerry browser users is heading the in the right direction.

Comscore reported in December that from September 2009 to Dec. 2009 RIM browser market share fell by one percentage point, while mobile Safari was up 1.2 percentage points. Google mobile browser share was up 2.7 percentage points. Yes, RIM still had 41.6% of the smartphone marketshare, but that’s not so much more than iPhones and Google phones combined: 30.5% in December, and certainly still going up.

And my (admittedly unscientific) research with BlackBerry users in our newsroom seemed to indicate that people with BlackBerries aren’t as interested in Web usage as their Google- and iPhone-toting colleagues. They don’t seem to expect the same degree of neato interface/experience.

In any case, our options for mobile potholes:

1) Build two mobile sites, one for iPhones and Google, and another for BlackBerries. Do you want to build 2 sites for everything? Me neither.

2) Build for the browsers that work (OK, not fair, but it’s not like the Google Maps Javascrip API is new), on the theory that you’re offering the people who DO care the best experience you can, and that the number of people with access to the best features is rising all the time. And maybe BlackBerry will get its act together.

Neither are great, but for now I think I have to opt for 2. Sorry, BlackBerry friends! But I’d love to hear your thoughts. Am I way off base on the “BlackBerry users aren’t after great Web experiences” theory?

Am I missing the magical secret to BlackBerry Javascript development?

New project: Killing potholes with user input, local government support

mike — Sat, 27 Feb 2010 04:22:18 +0000

Project link: DesMoinesRegister.com/potholes

HTML/Javascript/Facebook Connect Development: Michael Corey
Database backend/Local government admin: James E. Wilkerson

UPDATE: Mobile, location-aware potholes beta map now launched!

In case you haven’t noticed, potholes are everywhere. We hate ‘em, and we’re out to kill them.

We did a similar map two years ago, but there’s a few big twists that make us pretty proud of this one.

Working with local governments: We consulted extensively with Des Moines Public Works to see how we could build a system for reporting potholes that worked for our users AND for the city. We sent them data last time as well, but it didn’t work well for them. We didn’t have reverse-geocoding figured out two years ago, and someone had to re-enter our data into the city’s system to make anything happen.

Neither side wanted a repeat of that, but we really wanted to build a system Des Moines would actually use. The result: We send them daily e-mails with Excel attachments showing new potholes with exact locations. They’ve told use they’re going to hand those spreadsheets directly to their road crews and use that as a manifest for the day alongside their existing system. We’ve gotten word from several suburbs that they’ll participate as well.

As far as I know this is the first time we’ve ever pushed data TO a government agency. Usually we’re trying our best to pry it loose from them. And we think this is a pretty big win-win. We have a larger megaphone than the city Web site, and they have the power to make the evil potholes go away.

Facebook Connect support: This is our first Facebook Connect app. We’re giving users the option of logging in with their Facebook ID or their DesMoinesRegister.com account. We wanted to add registration of some kind this time around to cut down on spam and to see how willing people will be to use their real ID on our site.

We’re only scratching the surface of what we can do with Facebook Connect, but hey, we launched this thing today. Give us a bit. Lots more to come.

Geo-awareness: We’ve enabled browser-geolocation if the user chooses, so she can mark a pothole near her current location. This isn’t incredibly useful for desktop users, but we’re adding a mobile version soon, and we all know that’s where the real geolocation action is.

The early buzz on Twitter in response to the launch has really been good. And this is the first time I’ve been happy about the worst winter in my recent memory: there won’t be a shortage of potholes to fill this spring.

Denmark photos!

mike — Sat, 28 Nov 2009 02:55:25 +0000

Lots of photos from my trip to Denmark are now posted!

Highlights include Koldinghus:

Tivoli at Jul, with København city hall in the background:

Smørrebrød in Amagerbro:

Danmark, farvel

mike — Wed, 25 Nov 2009 08:32:12 +0000

I’m waiting groggily (not the Tivoli kind of grog) and a bit grumpily for my flight out of København. I’ve had a great time in Denmark and will have a lot more to say in the next few days, but for now anything I post is going to be more unintelligible than me trying to say “rød grød med fløde.”

So an early thank you to everyone who made me feel so welcome here. I miss home, but it’s very hard to leave. Hopefully we’ll all meet again soon.

Danmark, her kommer jeg!

mike — Wed, 11 Nov 2009 01:56:52 +0000

Hello, Denmark visitors! I can see some traffic coming in from the GeoForum site, and rest assured, I’m plugging away and putting the finishing touches on my presentation for Kortdage 2009 in Kolding next week.

I’ll be talking about the use of Google Maps in journalism, and it should be a fun talk. In any case it has been fun for me to go back over the last few years of work and see how far we’ve come.

And of course I’m extremely excited to be visiting Denmark. Many thanks in advance to GeoForum Danmark and especially to Jesper Ishøj, Hans Ravnkjær Larsen, Lars Brodersen and more for helping to bring me across the Atlantic. I’m excited to learn more about Danish history and culture, and I’m especially looking forward to meeting some of the innovators in Danish neocartography and journalism.

Those who know me well also know how excited I am to have the opportunity to see a European football match while I’m there between Bronby and OB, two of Denmark’s top teams. Someone be sure to tell me what colors not to wear before we go!