Holographic Haystacks — Capturing Business Transactions

Why Archive Every Business Transaction?

The other day I found myself in a discussion with a prospect regarding the merits of capturing complete stack traces for every transaction that flowed through their production system. You see AppDynamics doesn’t do that; we capture problematic business transactions, and this bloke seemed concerned someone would one day ask him for the details of a specific transaction that our method would leave him underprepared to fulfill. As the conversation played on, we explored the cost – both literal, in the sense of hardware required for data warehousing, and virtual, like the compounding impact additional polling during bad times has on performance. Ultimately we arrived at the conclusion that understanding your haystack is better than having the haystack. This point can be subtle, so let’s explore how we got there…

Fullscreen capture 782014 31924 PMSince AppDynamics automatically baselines business transactions, we can very quickly tell the difference between a normal transaction and an outlier, or in the context of this analogy: hay vs. needles. When we encounter a piece of hay we record the metadata of the transaction. You know: how long the strand is, where it resides, who put it in the stack, etc. In fact, the part of me that humbly struggles with homonyms likes to think of this as a piece of “hey.” I imagine someone (a business transaction) flying past me on the highway and waiving “heeeyy!” as I feverishly jot down some key details about their ride. For the most part I already know enough about them, they’re speeding along unabated, just like the plethora of friends to follow, no need to dig too deep, just note the event so we can better understand our “heystack.”

In contrast, when a particular transaction performs several standard deviations outside of the norm, or perhaps worse than a given threshold, we need to know more than “it’s slow.” This is where the context before content idea really starts to come into light. Because we know what a good piece of hay looks like, and because we understand how our virtual haystack looks at any given moment, we have the context to both appreciate and spot the needles – the need for content. Needles are special. They’re shiny. And you probably want to trade them for hay. So we need to know all their specs. We need content.

 

No worries, that’s why AppDynamics captures complete transaction flows for these items. But what if – returning to the highway analogy for a moment – a funeral procession (a bout of bad performance) drives by? The act of gathering information during this kind of needle storm could exacerbate the issue and result in exaggerated wait times, inaccurate information, and a unnecessarily pitiful user experience. Well we have the context to understand when that’s happening so AppDynamics will automatically throttle back needle gathering to ensure our application never negatively impacts yours. Remember the ultimate goal here is to ensure you have all the tools you need to deliver an exceptional user experience, in the short and long term.

Thinking about the long term for a moment … we should probably use this information to improve the performance of the application. So let’s pass the data about the problems along, shall we? But … if all you ever gave your team were needles, they might forget what hay looks like. This lack of context could hinder their ability to remediate issues. To mitigate these risks, AppDynamics will snag the occasional piece of hay… just for comparison’s sake and ensure your team, your developers, have all the context and content they need to improve the performance of your applications.

So as you can see, you don’t need to house an entire haystack to effectively identify and remediate problems. With a clear understanding of what the haystack would look like – a holographic haystack, if you will – a few pieces of hay for reference, and a spotlight on all your needles, you’ll have everything you need to see, act, and know more quickly than ever before.

Also check out how Citrix uses AppDynamics to capture business transactions, increase visibility, and lower their MTTR for their complex applications.

 

Take five minutes to get complete visibility into the performance, and start finding those needles, of your production applications with AppDynamics today.

How Garmin attacks IT and business problems with Application Performance Management (APM)

If you’ve used a GPS product before you’re probably familiar with Garmin. They are the leading worldwide provider of navigation devices and technologies. In addition to their popular consumer GPS products, Garmin also offers devices for planes, boats, outdoor activities, fitness, and cars. Because end user experience is so important to Garmin and its customers, ensuring the speed of its web and mobile applications is one of its highest priorities.

This blog post summarizes the content of a case study that can be found by clicking here.

The reality in today’s IT driven world is that there are many different issues that need to be solved and therefore many different use cases for the products designed to help with these issues. Garmin has shown it’s industry leadership by taking advantage of AppDynamics to address many use cases with a single product. This shows a high level of maturity that most organizations strive to achieve.

“We never had anything that would give us a good deep dive into what was going on in the transactions inside our application – we had no historical data, and we had no insight into database calls, threads, etc.,” – Doug Strick

Garmin-FlowMap

AppDynamics flow map showing part of Garmin’s production environment.

Real-Time and Historic Visibility in Production

You just can’t test everything before your application goes to production. Real users will find ways to break your application and/or make it slow to a crawl. When this happens you need historical data to compare against as well as real-time information to quickly remediate problems.

Garmin found that AppDynamics, an application performance management solution designed for high-volume production environments, best suited its needs. Garmin’s operations team feels comfortable leaving AppDynamics on in the production environment, allowing them to respond more quickly and collect historical data.

The following use cases are examples of how Garmin is attacking IT and business problems to better serve their company and their customers.

Use Case 1: Memory Monitoring

Before they started using AppDynamics, Garmin was unable to track application memory usage over time. Using AppDynamics they were able to identify a major memory issue impacting customers and ensure a fast and stable user experience. Click here to read the details in the case study.

Garmin-Heap

Chart of heap usage over time.

Use Case 2: Automated Remediation Using Application Run Book Automation

AppDynamics’ Application Run Book Automation feature allows organizations to trigger certain tasks, such as restarting a JVM or running a script, with thresholds in AppDynamics. Garmin was able to use this feature very effectively one weekend while waiting for a fix from a third party. Click here to read the details in the case study.

Use Case 3: Code-level Diagnostics

“We knew we needed a tool that could constantly monitor our production environment, allowing us to collect historical data and trend performance over time,” said Strick. “Also, we needed something that would give us a better view of what was going on inside the application at the code level.” – Doug Strick

With AppDynamics, Garmin’s application teams have been able to rapidly resolve several issues originating in the application code.  Click here to read the details in the case study.

Use Case 4: Bad Configuration in Production

Garmin also uses AppDynamics to verify that new code releases perform well. Because Garmin has an agile release schedule, new code releases are very frequent (every 2 weeks) and it’s important to ensure that each release goes smoothly. One common problem that occurs when deploying new code is that configurations for test are not updated for the production environment, resulting in performance issues. Click here to read the details in the case study.

Garmin-Verification

AppDynamics flow map showing production nodes making services calls to test environments.

Use Case 5: Real-time Business Metrics (RTBM)

IT metrics alone don’t tell the whole story. There can be business problems due to pricing issues our issues with external service providers that impact the business and will never show up as an IT metric. That’s why AppDynamics created RTBM and why Garmin adopted it so early on in their deployment of AppDynamics. With Real-Time Business Metrics, organizations like Garmin can collect, monitor, and visualize transaction payload data from their applications. Strick used this capability to measure the revenue flow in Garmin’s eCommerce application.  Click here to read the details in the case study.

Garmin-RTBM

Real-time order totals for web orders.

Conclusion

“AppDynamics has given us visibility into areas we never had visibility into before… Some issues that used to take several hours now take under 30 minutes to resolve, or a matter of seconds if we’re using Run Book Automation.” – Doug Strick

Garmin has shown how to get real value out of its APM tool of choice and you can do the same. Click here to try AppDynamics for free in your environment today.

Deploying APM in the Enterprise Part 7: Dashboards and Reports – Get the Business on Your Side

Welcome back to my series on Deploying APM in the Enterprise. In Part 6: Spread the APM Love we talked about ways to increase user adoption of your APM tool and thereby make your organization more successful.

This week we focus on Dashboards and Reports. I’m a huge believer in unlocking the information and intelligence contained within your monitoring tools, and turning that data into actionable information. Over time your tooling ecosystem will collect a wealth of data that can be used for capacity planning, business intelligence, development planning, and many other business and IT activities that require information about usage patterns and operational statistics. By unlocking this value, and surfacing it to the business and IT organizations in a meaningful way, you take another step up the maturity ladder and solidify your rockstar status within your organization. A huge benefit of providing useful information to the business is that they will fight for your tools and projects if they are ever in question!

Dashboards

Dashboards should be used to provide real-time insight into critical business and IT indicators. A failed server doesn’t always mean that there is impact to the business.

Several of our customers are using AppDynamics dashboards to better understand business activity. For example, we have multiple e-commerce customers that are tracking revenue and order volumes in real time.

Let’s take a look at some different perspectives within the organization and explore what type of dashboard each role requires.

Management

Most managers don’t need to know the sordid details about the infrastructure that supports the business applications. For each application within the managers realm of responsibility they need to understand key business indicators and have an overall indication of their application performance and health. Take a look at the dashboard below…

 MgmtDashThmb

This is the type of dashboard that managers want to keep an eye on throughout the workday. Any impact to the business will be easily identified on this dashboard so that the manager can make sure the impact is being addressed. There should be alerts associated with these metrics so that the operations center is notified of business impact but it’s not always an IT problem when business metrics deviate from normal behavior. It’s possible that sales volume is impacted by a competitor offering lower prices on the same product. This will show up as business impact and there is nothing that the IT staff can do to fix it. This is a business problem that needs to be addressed by the appropriate department.

Of course any IT infrastructure problems that impact the business will also show up on the managers dashboard but only by way of business metrics that have deviated from normal behavior. This fosters communication between IT and the business which should lead to improved cooperation and coordination over time.

Application Support

Application support rides the fence between the business and IT. If something goes wrong with their application, the business makes sure that app support hears about it and gets the problem resolved in a timely manner. Application support teams need a view of key business indicators (similar to what the managers are looking at) as well as key IT indicators so they know when there are problems with the application or the infrastructure they depend upon.

Operations

The operations teams need to know when infrastructure components are nearing capacity, throwing errors, or failing completely. It is their responsibility to ensure the proper functionality of the infrastructure and as such need an infrastructure centric view of the components they are responsible for. This is a very classic enterprise monitoring view that has been long established and still has its place in the modern monitoring world. With that said, its beneficial to show the operations teams a few key business indicators so they know how urgent it is to replace that failed piece of hardware.

Reports

Given that dashboards are best utilized to provide real-time status information, reports are the go-to solution for information that drives action in the future. Reports can be about the application, infrastructure, business, or anything else that makes sense given the data you ave to work with. What I want to focus on in this blog are the insights I have picked up over the years when developing and utilizing reports at my past companies.

  • Reports should contain information that is actionable. There is little value in receiving a report that you cannot use to decide if action is required.
  • Reports need to be concise. Very few people are interested in reading a 50 page report. Try to keep them to a few pages or at least summarized in the first couple of pages with supporting details in the rest of the report.
  • Don’t send an avalanche of reports. People usually don’t have the time or desire to read multiple reports per day. Ideally you should send reports only when there is something in the report that needs attention.
  • If you can, include the source of the report and a description of what they report is used for. I’ve received reports in the past where I didn’t know what system sent them and why I needed to see them.
  • Know your audience. Make sure the business gets business related information and IT gets technology related information.
  • Don’t blast reports to email groups (usually). Most reports need to be seen by only a few people in your organization. Email distribution groups tend to contain way more people than are truly interested in your report. No need to clog up the email system with a 50 page PDF sent to 2000 addresses.

Reports are important and can help your organization avoid issues down the road but you need to implement them carefully or they become just another piece of internal spam. The best report that I can ever receive is the one that only shows up when there is a problem brewing within the next few weeks, clearly identifies the problem, and I am the right person to make sure it gets addressed. That would be reporting done right!

Dashboards and reports need to be implemented properly to get the most out of your monitoring investments. You can amplify the value of all of your monitoring investments by combining, analyzing, and displaying the data contained within each disparate repository. The most mature organizations will build out their monitoring ecosystem and then invest in analytics to derive business and technology value and to create the best dashboards and reports possible.

Thanks for taking the time to read this series. The final post will be next week and will focus on how to keep up the momentum and stay relevant when it comes to monitoring within your organization.

Click here to read Deploying APM in the Enterprise Part 8: Stay Thirsty (and Relevant) My Friends

Orbitz Deploys AppDynamics Across 2,700 Servers In Just 15 Days

Here at AppDynamics we’re very proud of how easy our solution is to deploy and maintain. We also tout the fact that in many cases there is no configuration required to gain the insight needed to solve complex application problems. All of this is absolutely true, but does it mean that AppDynamics doesn’t have enough functionality to support complex deployments that make little to no use of common frameworks? Absolutely not! But don’t take our word for it, see for yourself in this presentation by Orbitz (Geoff Kramer and Nick Jasieniecki) at one of our Chicago User Group meetings. In case you don’t know what Orbitz does I think this quote from Geoff Kramer sums it up quite well… “Orbitz unlocks the joy of travel for our customers. You can’t do that if you are having site problems.”

If you don’t have 50 minutes to spare right now I will summarize the videos key points in this blog post.