tirsdag den 7. april 2009

Parallel processing with ArcGIS Server: A case study with geocoding

Tænk hvis alle maskiner rundt omkring i verden som kørte ESRI software blev bundet sammen i det samme net ... ville vi så alle få en bedre oplevelse ud af det eller ville det blive det samme eller værre?

Image all servers around the world running ESRI software put together in one big great net work ... would we all experience a better performance or would it just be the same or worse?


You're probably already familiar with creating Web applications and services with ArcGIS Server, but in this blog post we're going to talk about how you can use ArcGIS Server to run very large back-end jobs. We'll also do it in the context of geocoding, which is an aspect of ArcGIS Server that we haven't covered much in this blog yet. We hope this will get you excited about some new possibilities available through ArcGIS Server.

ArcGIS Server is an exceptional framework for running large jobs because it allows you to break them down into smaller chunks of work that can be run simultaneously. This way you can take full advantage of your hardware resources. A good example that you may already be familiar with is map caching. A map caching job is transparently split across all your Server Object Containers. By creatively using ArcGIS Server services, you can work in a similar manner with more specific jobs such as batch geocoding, generation of high quality map series, demographic analysis, simulation, and so on.

Today we'll illustrate this with geocoding, which is the process by which you associate X/Y coordinates with addresses or place names. Imagine that you have a list of 25 million addresses to geocode. How do you effectively run this job?

One idea is to create a geoprocessing service that takes a small subset of these addresses and puts the outputs in a common repository. Once the service is up, you can simultaneously send small chunks of addresses to the service, allowing the geocoding process to occur in parallel until you are done. You can use a simple Python script or executable to span the jobs on the server. A typical locator handles around 2 million addresses per hour per core on a 3Ghz server, so on an eight core server our 25 million addresses would be geocoded in a bit less than a couple of hours.

All right, that is way too much of a high level description! We put together a small document that will disclose how you do this with a practical example. The document is called ArcGIS Server in Practice Series: Large Batch Geocoding and it's available from the ArcGIS Server brochures and white papers page.

Read more: http://blogs.esri.com/Dev/blogs/arcgisserver/archive/2009/04/06/Parallel-processing-with-ArcGIS-Server_3A00_-A-case-study-with-geocoding.aspx

Ingen kommentarer:

Send en kommentar