Improving GIS performance with AppDynamics

Twenty years ago I started my professional career working in a niche field of IT called Geographical Information Systems (GIS) – basically mapping systems. This is an area where Big Data was prevalent, long before the term was officially coined, GIS covered a range of concepts from satellite imagery and aerial photography processing to geo-demographic data analysis, address cleansing and geocoding to routing analysis, spatial algorithms and finally to publishing maps & associated data.

Google and Microsoft Bing have recently commoditised a lot of the publishing aspects of background mapping data, but the need for analysing and publishing an organisation’s own geographic data is as important today is it has ever been.

Historically, Application Performance Management (APM) and GIS have never made good bedfellows mainly due to the effort required in deploying APM.  At a recent conference I spoke to a GIS developer who stated they had no interest in APM because “they work in GIS.” Upon further questioning, this GIS developer acknowledged processing and accessing these large datasets is complicated and it’s slow and that’s “just how it is.”. So I challenged this and decided to demonstrate AppDynamics on GeoServer,  an open source mapping server written in Java that allows users to share and edit geospatial data.

Within a few minutes of installation, AppDynamics automatically detected key transactions such as publishing a map layer.

01B-WMS_Service

Taking this a step further, we looked at how AppDynamics can help a system administrator understand which parts of their service is accessed most and which components are the slowest (in this case average response times for publishing mapping datasets).

02-Layer_Performance

In the race to prove innocence, the challenge is to be able to determine whether any slow service calls were due to the end user or consumer of a service rather than the service itself — to understand if users are doing something you did not expect. For example, they could be pulling back too much data from the service (this applies to any system dealing with data).

In the case of mapping systems, AppDynamics can capture the bounding box representing the coordinates used to outline and return an area of interest, if the area of interest retrieved is too large and that leads to a slow transaction, then it’s likely a system administrator is allowing users to pull back too much data in one go:

04-HTTP_Params

If an administrator does see a slow down in the service they can drill into the service to isolate where performance problems occur. Problems in this type of application can manifest themselves in coordinate conversion, vector data reading (roads, rivers & buildings), database queries, grid handlers (reading image files such as satellite and aerial images), writing to an image file: 

03-WMS_at_Method_Level

System administrators and owners can improve their application or data processing times by identifying bottlenecks. The following example shows how an address lookup service was sped up from processing 60 calls per minute taking a second each, to more than 700 calls per minute taking less than 50ms.

Geocode Response Time

Geocode SQL

AppDynamics for Databases identified the cause of the slowness as the spatial query used to query the postcode, without an index it went from querying 1,684,930 rows…

MySQL 1a

To just 5,976: –

MySQL 2a

Tuning this query led to a tenfold performance improvement!

“Software is eating the world” [Marc Andreessen] and ensuring your application or service is performing is key. This message applies across all sectors, not just banking and ecommerce, “typical” APM customers, but everyone who offers a service or process. Our customers & consumers expect Google-like response times for all online services and we need to deliver.

Using AppDynamics, you can halve or better your data processing times, free up resources, achieve a faster time to market or execution and ensure your consumers in the wider world get that up to date map when they need it.

Take five minutes to get complete visibility into the performance of your production applications with AppDynamics Pro today.

Gartner’s APM Magic Quadrant: Application Mapping and Transaction Profiling Explained

Gartner recently released their latest magic quadrant for Application Performance Monitoring (APM) and in this report mentioned five key dimensions, two of which were Application Mapping and Transaction Profiling. These two dimensions are critical for users to identify performance bottlenecks in distributed applications, whose architecture design is typically based around SOA or Cloud concepts.

The point we’d like to emphasize in this post is this: To quickly find bottlenecks in distributed or SOA architectures, these two dimensions must be visible simultaneously to the user (troubleshooter).  Ok, get ready, we’re going to say something very “unvendor” – these two dimensions should actually be “features” and not separate “products”.   The only two APM products that combine these views in a single product today are AppDynamics and dynaTrace.

Unfortunately, the rest of the APM solutions don’t work that way. Most of the APM vendors who claimed to support these two dimensions for the MQ require customers to buy two or more distinct products – one of them usually a re-branded CMDB tool. The downside is that it is nowhere near as efficient, especially if the troubleshooter has to log into 2-3 different products and has to try to stitch together this view in their own mind.