AppDynamics Transaction Analytics Raises the Bar for Business Metrics

Last year, we announced our Real-Time Business Metrics (RTBM) capabilities to the world. The response from our existing and new customers has been amazing. We’ve seen business and IT dashboards created and used by business and technology executives alike.

We’ve seen major hotel chains collect metrics like searches and bookings per brand, incoming partner search and booking volumes, and detailed revenue reports by property location and booking channel. We’ve seen some of the largest car rental agencies in the world track revenue per region in real time to proactively identify business impact. We’ve also seen major footwear retailers track revenue per shoe type, checkout total and discount percentages to make better business decisions. These are just a small sample of the industries and use cases where RTBM has been successfully used.

When we introduced RTBM, it was a meaningful first step on the path to create a real-time analytics platform to serve both business and technology stakeholders equally well. After all, the most successful businesses today are transforming themselves into digital enterprises to deliver products and services through the channels consumers use most often. Mobile and internet-based consumption have taken center stage and most enterprises are investing heavily to meet that consumption model.

However, RTBM isn’t the only way we allow customers to better understand  their business problems.  AppDynamics’ overall analytics capabilities encompass three main focus areas:

Business Analytics harnesses the power of RTBM to provide meaningful business information and alerts.

analytics page - real-time business metrics graphic2

Business Analytics is used to answer the following types of questions:

  • What is our total revenue for the day?
  • How many checkout transactions failed today?
  • How many trades completed successfully today?
  • How many car rental reservations were made today?

The second type of analytics we offer is Operational Analytics, which collects all of the infrastructure data (OS metrics, JMX metrics, WMI or perfmon metrics, log data, etc.) from your devices and application components.

scalability analysis

Operational Analytics is used to answer the following types of questions:

  • What is the breakdown of failed transactions between 3G and 4G mobile networks?
  • How many connection pool errors led to failed transactions today?
  • Which application components are throwing errors that caused failed transactions?

Today, we announced the future availability of AppDynamics’ third type of analytics, Transaction Analytics, an integral component of the AppDynamics Application Intelligence Platform, which was also announced today.

Transaction Analytics captures raw business data from all layers of an application and across every single node without altering any code.  Gathering this data, correlating it with operational intelligence, and presenting it in a way that makes it easy for customers to answer important business questions is a capability we are extremely excited about.

analytics

For example, capturing the specific details of each transaction enables businesses to answer the following questions:

  • Which Android users experienced failed checkout transactions in the last 24 hours and what was in their cart at the time?
  • How many customers experienced a slow ‘submit reservation’ transaction in the last hour from a Chrome browser in New York and what was the total dollar amount of those reservations?
  • What was the revenue impact by product category associated with the two marketing campaigns we ran on Black Friday?
  • How many transactions originated from Tier 1 partners over the last 90 days and what was the resource utilization and revenue associated with those requests?
  • How many car rental reservations were made for each category of vehicle and what is the associated breakdown of revenue by region?
  • What are the total bookings from California broken down between iOS and Android, and what is the associated breakdown between wireless carriers.

The answers to the questions above help make better business decisions like:

  • Immediately fixing a business mistake that is negatively impacting revenue
  • Investing in a “mobile first” strategy
  • Choosing what marketing campaigns to run again and which to stop
  • Selecting what product types to buy more of for increased revenue growth and determining which regions to market them
  • What mobile platform should we invest more time and effort into?
  • Do we need to make our mobile application more resilient to poor network performance?
  • Do we have technology problems related to specific products, regions, or consumption models?

One of the unique differentiators of all of our analytics capabilities is that it does not require any change to existing code. AppDynamics will gather all of the requisite data from your servers without the use of special compiled API calls or extra logging statements.

If you’d like to get started using AppDynamics to try business & operational analytics out for yourself, you can create a free account and start monitoring and analyzing your applications today.  If you are interested in being a part of our upcoming Transaction Analytics beta program, you can find more information here.

Announcing AppDynamics Application Intelligence Platform

AppDynamics is excited to unveil details behind our new Application Intelligence Platform which is the technology foundation for the AppDynamics portfolio of performance monitoring, automation and analytics solutions (transaction analytics was also announced today).

Today’s software defined businesses need the ability to see everything in their application environment, act with certainty, and know faster. See, Act, & Know, the three core pillars of Application Intelligence.

See Act Know

Making sense of all the data, transactions, events and user interactions in these environments is increasingly difficult as the volume, variety and velocity of data grows. Traditional approaches to solving this problem are typically silo’d in their view providing limited business correlation and only present static data. In order to keep up with todays complexity, what’s needed is a platform that can collect, understand, and act on the data and translate that into actionable information about your business.

platform page - application intelligence platform graphic2

The AppDynamics Application Intelligence Platform is uniquely able to deliver rich performance data, learning, and analytics, combined with the flexibility to adapt to virtually any infrastructure or software environment. The platform was designed for the modern day application, including SOA architectures, frequent agile releases, cloud deployments, big data technologies and mobile delivery – and maintains a unique low-overhead architecture. This gives you the ability to instantly see the impact of production changes as you work, and adapt to optimize performance and minimize bottlenecks while ensuring the revenues are maximized.

Today’s consumer-driven world moves faster than ever. Everything from your customer experience to revenue depends on your business-critical applications performing at their highest level. AppDynamics delivers real-time access to every aspect of your performance, so you can anticipate problems, resolve them automatically, and make smarter, more certain business decisions.  This includes detailed understanding of the user experience, business data such as real time revenue metrics, deep application performance and infrastructure data. All collected within context of business & operational transactions.

Platform Design Principles:

The Application Intelligence Platform is the underlying architecture that we use to deliver all of the capabilities a modern business needs to keep their business running at peak performance and use application intelligence as a competitive advantage.

The core design principles behind the AppDynamics Application Intelligence Platform include:

Scalability:

The interactions customers have with the business are highly dynamic, environments are larger and more hybrid and organizations are demanding IT transform the business faster than ever. AppDynamics monitors many of the largest, most complex enterprise application environments in the world, supporting environments up to 10,000 nodes on a single management server. This is critical for customers who do not want to manage and maintain a farm of servers.

Lightweight:

Our platform was designed to be a production tool from the very beginning, giving it a unique low-overhead architecture. Intelligent data collection to ensure the lowest production overhead in the market. AppDynamics correlates end user business transaction details with completion status (success, error, exception), response times, and all other data points measured at any given time. It automatically analyzes the entire data set to provide information from which to draw conclusions and take the appropriate action.

Self-learning:

Auto-instrumentation, and minimal configuration enables simple and intuitive operation for any size business. AppDynamics has a unique ability to instrument your application, learning its transactions, code execution behavior, and normal performance patterns. It can adapt and adjust instrumentation automatically when you change your application-no manual configuration needed.

Open and Extensible:

The AppDynamics platform is open, extensible, and interoperable to fit any business need. Through the AppDynamics Exchange our community comes together to share knowledge and contribute back over 100 extensions that provide deep integrations to the tools you already use like Splunk, Apica, PagerDuty, and Amazon Web Services. From monitoring Amazon Web Services costs to MEAS mainframes our extensions allow you to leverage the tools you already have in place. We believe you own your data and it should be easy to consume and analyze which is why we provide REST APIs and SDKs available on GitHub that make it as easy as possible to get started.

Flexible:

The platform can be deployed in any operating model, including on-premise, SaaS, private cloud, or a hybrid combination. It’s always the exact same platform and there are no cost implications associated regardless of deployment model.

Secure:

Provides enterprise grade security with Role-Based Access Control (RBAC), LDAP, and SaaS certifications that guarantee security and compliance.

What services are delivered on the Application Intelligence Platform?

The AppDynamics Application Intelligence Platform gives companies the foundation they need to monitor performance and extract meaningful analytics from their business applications. Software-defined businesses must be certain that their most complex, business-critical applications are performing at the highest level; and be certain that the data and information generated by these applications can be harnessed for ongoing business advantage and impact. AppDynamics’ Application Intelligence platform enables customers to monitor, respond, and analyze their application environment.

Monitor

Application Performance Management

Mobile Application Monitoring

End User Experience Management

Database/NoSQL/BigData Monitoring

Infrastructure Monitoring

Respond

Run Book Automation

Cloud Auto-Scaling

Alerting

Analyze

Transaction Analytics (Beta)

analytics

Real-Time Business Metrics

analytics page - real-time business metrics graphic2

Operational Analytics

scalability analysis

The Application Intelligence Platform delivers this functionality by collecting data from across the application environment, processing & correlating it, and converting that data into knowledge for customers.

Collecting Data:

  • Instrumentation allows us to watch every line of code.
  • Distributed transaction tracing follows the business transaction across all tiers of the application, not just limited to a single node.

  • Transaction auto-learning engine inspects execution code, payload, libraries and methods so you are not left in the dark.

  • Real-time service discovery renders architecture topology to clearly and automatically map critical relationships and dependencies..

  • The smart agent filters and transfers data to the management server securely and in real-time.

Processing & Correlating Data

  • Real-time stream processing allows for complex event processing of metric and event streams.
  • Time series clustering and analytics provides the ability to index and manage time series by auto rolling up, purging or clustering data sets by time increments.

  • Behavioral learning is an engine that continuously adjusts dynamic baselines for an automated manageable data set.

  • Unstructured and structured big data indexing creates a data warehouse for different types of application data which is seamlessly correlated and processed.

  • Event correlation shows the relationship of performance, change, and business events.

Converting Data Into Knowledge

  • Intuitive user interface (UI) allows for a simple and easy-to-use platform for business users, developers and operations teams.

  • Dynamic flow maps create a clear and understandable visual representation of the application topology, so organizations understand what is happening in the big picture.

  • Cross-correlated drill down from anywhere in the UI, enabling pin-point accuracy across multiple nodes and machines.

  • Real-time correlation of performance metrics and business metrics create a business understanding of data never before delivered.

  • Compare and analyze the side-by side performance of different versions of an application.

  • Custom drag and drop HTML5 dashboards can be configured to any specification to deliver clear actionable results.

  • Query language allows easy, targeted data discovery.

In today’s competitive and demanding environment, organizations need to leverage Application Intelligence for ongoing business advantage. The AppDynamics Application Intelligence Platform enables customers to see everything across all front and back-end interactions: business transaction and code-level visibility with no blind spots.

Maximize digital business performance through an open, integrated solution that delivers real-time monitoring, actionable business and operational insights, and automated issue resolution across applications, infrastructure, and user experience. Take five minutes to get complete visibility and control into the applications that power your software-defined business. Get started with the AppDynamics Application Intelligence Platform today.

A New Breed of Actionable Analytics

Data is dumb but we can’t seem to get enough of it these days. An entire industry has evolved to make sense of the massive amounts of data being generated every day. Massive data collection by itself does not guarantee the context required to solve business and IT problems… If we are smart about data, however, it will lead us in the right direction.

Today’s businesses are defined by the software they run on, enabling them to innovate and create new services faster. Apple re-invented the music industry through software; Netflix changed the way we consume movies through software; grocery shopping, taxi-hire, parcel-logistics, car rental, travel booking… these are all industries that have been completely disrupted by software.

As more and more organisations depend upon the software they run on, the consumer demand for superior services, faster innovation, and improved performance continues to accelerate. As a result the software required to compete in this unforgiving market only continues to increase in its complexity.

It was only a few short years ago that when I wanted to book my family vacation, we all made the trip down to the local travel agent and sat around a desk while the agent tapped away on her green-screen searching for available flights and hotels. The process of booking our hotels has obviously changed dramatically, and so have the applications that support the process.

For example, below is a screen shot of the Business Transaction flow through a global travel site when a user makes a simple search for a hotel. All the user does is select a destination, some dates and maybe one or two preferences, but the resultant transaction kicks off more than 200 service requests before the results are returned to the user they can make their selection of preferred hotel.

Hotel Search

This rise in application complexity often means we generate increasingly more structured and unstructured data from our applications such as log files, email, alerts, infrastructure stats, network stats etc. Close to 200 billion emails will be sent in 2014, I wonder how many of those emails are false alerts from monitoring tools?

 

Keep nothing?

I have been helping companies build monitoring strategies for nearly a decade, and the problem I used to regularly face was dealing with organisations that just didn’t have enough data. IT departments didn’t have the required information to be able to quickly diagnose and remediate problems when they occurred.

Keep everything?

As organisations continue to have a greater dependency on the software they run on and application complexity continues to increase, there is a danger we swing the other way by collecting and storing too much data to be able to make sense of it in a timely manner. My colleague Jim Hirschauer often refers to this as the “home hoarders effect” drawing parallels with people who simply never throw anything out from their homes. The bigger the piles get the harder it is to find what you need, and at some point that pile is going to topple over.

 

Keep what you need

There are lots of good reasons to keep data, but “just in-case we need it” is not a good reason. It is important to think about why the data is being captured and stored and how it will help solve problems in the future. By keeping the relevant data and throwing away the clutter, you become more efficient and effective in troubleshooting and resolving problems.

But data alone isn’t good enough to solve your business and IT problems. Data in and of itself is one-dimensional and dumb. Data doesn’t tell you when there are problems, it doesn’t tell you when business is going well, it doesn’t tell you anything meaningful without some help and a lot more context. Let’s explore an example to illustrate my point.

Let’s say we have a couple of data points about a person. The data points are…

Heart Rate = 150 bpm

Blood Pressure = 200 over 100

Now tell me, is this person performing well? With these data points we have no idea. We need more data. The table below shows a list of some possible data points we can collect.

Screen Shot 2014-03-24 at 10.42.29 PM.png

Notice the last attribute in the table. The activity provides us with the context we need to focus on the proper data points to figure out if the person is performing well or not. Here are some more relevant data points…

Distance Run = 100 meters

Time = 9.58s

Now can we determine if the person is performing well or not? I’m not much of a track and field aficionado so I have no idea if this is a good performance or not. I need point of comparison to determine if 9.58 seconds in the 100-meter dash is good. So here is our baseline for comparison sake…

100-Meter World Record Time = 9.69s

Well it looks like the person was performing really well. They set a world record in the 100-meter dash. All of the data points individually didn’t tell us anything. We required correlation (context) and analytics (comparison to baseline) in order to turn data into information. I like to refer to this concept as creating Smart Data.

 

Smart Data Defined

Smart Data is actionable, intelligent, information.

 Smart Data is created by performing correlation and analytics on data sets. AppDynamics correlates end user business transaction details with completion status (success, error, exception), response times, and all other data points measured at any given time. It automatically analyzes the entire data set to provide information from which to draw conclusions and take the appropriate action. This information is called Smart Data.

Being smart about your monitoring data collection allows you to isolate and resolve problems much faster, and with a much lower cost of ownership and overhead.

I have recently been working with a company to replace their legacy monitoring tool with AppDynamics. The environment they are monitoring is a good size, consisting of about 1,200 servers and their main application processes approximately 300,000 transactions per minute. They invested in a monitoring tool to help manage the performance of their applications which captured and stored all the data they could “just in-case” it was needed. Unfortunately this approach required an additional 92 servers to be provisioned for the monitoring tool itself, which consumed approximately 80TB of data storage per year. The increasing investments this customer needed to make in hardware, storage, people, and maintenance was too much to manage for them and they decided to look for a different approach.

 AppDynamics “smart data” approach to analytics means this particular customer now only requires two reporting servers and the storage requirements were reduced to just 1TB per year. Collecting only the data required to make smart decisions gave them both a 98% reduction in hardware costs and  more effective analytics in the process.

Adding business context

Smart data is not just about resolving problems faster though. In October last year we introduced Real-time Business Metrics and described how AppDynamics customers can use Real-time Business Metrics to extract and present business metrics directly from within their applications. These business metrics provide business context enabling customers to turn smart data into actionable information. Our customers, for example, can see the exact revenue impact of performance problems, the end user experience during an app upgrade, or the real-time impact of marketing campaigns. Smart analytics not only clearly show the business benefits of making immediate improvements, they help direct where limited resources should be invested for further business and application improvement going forward.

AppDynamics is focused on delivering actionable intelligence to solve problems for IT operations and development teams as well as business owners. To learn more about Real-time Business Metrics read here

 

Proactive Monitoring and Alerting with AppDynamics

As an application owner, monitoring performance of your application is already second nature. However, are you reactively monitoring your application? In other words, do you review performance metrics after an application issue or outage occurs in hopes to identify problem areas? Or are you also proactively monitoring your application? In this case, you actively receive alerts or notifications about performance degradation before an actual issue or outage happens. In this article, we will discuss how to use AppDynamics to proactively monitor your application.

In AppDynamics, you can define a set of Health Rules where each contains a set of conditions that maps to key performance indicators measuring the health status of your application.

By default, AppDynamics already has a few Health Rules enabled. For example, the “CPU utilization is too high” Health Rule will trigger a “Health Rule Violation Started – Warning” event if CPU utilization on a node is above 75% and a “Health Rule Violation Started – Critical” event if it is above 90%.

You can define your own Health Rules based on the template types that AppDynamics have defined. Each template type provides a set of entities, such as Business Transaction, Node, Tier, or JMX Object, and “health rule” metrics tied to those entities.

Once you have decided a Health Rule type, you have to pick the entity that the Health Rule applies for. This is required because a Health Rule needs an object to evaluate against.

After choosing the entities to evaluate against, you will have to build the conditions that reflect either a warning or critical status of your application. For each status; warning or critical, you can define one or more conditions where each compares a key performance metric against a static value or dynamic baseline. For example, you can build a warning condition that triggers when a node’s Average Response Time is greater than 10 ms and Calls Per Minute is less than 2 standard devision of the default baseline. Careful consideration should be taken when defining these condition sets such that it triggers as a forewarning of imminent issues.

For detailed information on Health Rules, please refer to this AppDynamics document

Now that you have a Health Rule defined, you will get an event triggered off of your defined condition. However, you still need to look for the event after an issue or outage to know of its occurrence. To have AppDynamics proactively alert you of an event occurrence, you still need to define a Policy and associated Actions corresponding to your Health Rule event.

In AppDynamics, a Policy monitors the events logged and when an event that matches the policy’s triggering condition appears, then the policy fires a set of Actions associated to it. The Policy triggering condition consists of matching events and matching entities such that an event must be a specific type, for example Health Rule Violation Started – Warning, and occurs on a specific entity, for example Node 2.

 

For detailed information on Policies, please refer to this AppDynamics document

Once you define the triggering conditions for the Policy based on the Health Rules you created, it is time to define what set of Actions you want AppDynamics to perform when the policy fires. There are many types of actions available to you in AppDynamics. From simple alerting actions via email or SMS, to diagnostics actions such as starting Diagnostic Sessions or taking thread dumps, to advanced custom actions to integrate with existing alerting systems such as JIRA or PagerDuty. AppDynamics has it all.

 

Check out AppDynamics Exchange to find Alerting Extensions that can make your policies even more powerful. Alerting extensions include custom actions that sends alerts to PagerDuty, ServiceNow, and more. For more information,  check on AppDynamics Exchange here. For detailed information on Actions, please read this AppDynamics document

Voila! With Health Rules, Policies and Actions in place, AppDynamics is ready to proactively alert you when your application begins to experience performance degradation. The key to successful proactive alerting is to define that set of Health Rules and triggering Policies that will fire off when your application is entering a state of concern. This will undoubtedly require intimate knowledge of the application as well as the application’s operations. Here is a good example: degraded average response time when call volumes are high might be normal for your application, hence, define the health rule to include the condition of calls per minute lower than average operations (using a baseline).

Now go enjoy the praise while you resolve issues before they impact business operations or result in outages!

Take five minutes to get complete visibility into the performance of your production applications with AppDynamics today.

Why BSM Fails to Provide Timely Business Insight

Business Service Management (BSM) projects have always had a reputation for over promising and under delivering. Most people know BSM as the alleged “manager of managers,” or “single source of truth.” According to the latest ITIL definition, BSM is described as:  “The management of business services delivered to business customers.” Like much of ITIL this description is rather ambiguous.

Wikipedia however, currently describes BSM’s purpose as facilitating a “cultural change from one which is very technology-focused to a position which understands and focuses on business objectives and benefits.” Nearly every organization I talk to highlights being technology-focussed as one of their biggest challenges, as well as having a desire for greater alignment to business goals. BSM should therefore be the answer everyone is looking for… it’s just a shame BSM has always been such a challenge to deliver.

Some years ago I worked as a consultant for a small startup which provided BSM software and services. I got to work with many large organizations who all had one common goal: “to make sense of how well IT was supporting their business.” It was a tremendous learning experience for me where I frequently witnessed just how little most organizations really understand the impact major IT events had on their business. For example, I remember working with a major European telco who would have an exec review meeting on the 15th calendar day in the month, to review the previous months’ IT performance. The meeting was held on this date because it was taking 4 people 2 weeks to collate all the information and crunch them into a “mega-spreadsheet.” That’s 40 man days effort to report on the previous 30 day period!

As organizations continue to collect an increasing amount of data from a growing list of data sources, more and more organizations I talk to are looking for solutions to manage this type of “information-fogginess,” but are skeptical about undertaking large scale BSM projects due to the implementation timescale and overall cost.

Implementing BSM:

I’m sure the person who first coined the term “scope creep” must have been involved in implementing BSM, as most BSM projects have a nasty habit of growing arms and legs during the implementation phase. I dread to think how many BSM projects have actually provided a return on their substantial investments.

BSM has always been a heavily services-led undertaking as it is attempting to uniquely model and report on an organization. No two organisations are structured in quite the same way; each having its own unique IT architecture, operating model, tools, challenges and business goals. This is why BSM projects almost always begin with a team of consultants conducting lots of interviews.

Let’s look at cost of implementation for a typical deployment such as the European Telco example I described earlier. This type of project could easily expect 100 – 200 man days of professional services in order to deliver. Factoring in software license costs, training, support & maintenance costs, the project needs to deliver a pretty substantial return in order to justify the spend:

Cost of BSM implementation:

Professional services

(100-200 days @ $1800 per day)

$180,000 – $360,000

Software license

$200,000 -$500,000

Annual support and maintenance

$40,000  – $100,000

Training

$25,000

TOTAL

$445k – $985k

Now if we compare to the pre-existing cost of manually producing the monthly analysis:

Existing manual process costs:

Days per month creating reports

10

Number of people

4

Total man days effort per year

480 days

Average annual salary

$45,000

Total working days per year

225

Annual cost to generate reports

$96,000

Even with our most conservative estimates it would take almost 5 years before this organization would see a return on their investment by which time things will probably have changed sufficiently to require a bunch of additional service days in order to update the BSM implementation. This high cost of implementation is one reason why there is such a reluctance to take the leap of faith needed to implement such technologies.

The most successful BSM implementations I am aware of have typically been the smaller projects, which are primarily focused around data visualization; but with powerful open-source reporting tools such as graphite, graphiti or plotly available for free, I wonder if BSM still has a place even with these small projects today?

What does success look like?

Fundamentally, BSM is about mapping business services to their supporting IT components. However, modern IT environments have become highly distributed, with SOA architectures that have data dispersed across numerous cloud environments and it is just not feasible to map basic 1:1 relationships between business and IT functions any more. This growing complexity can only increase the amount of time and money it takes to complete a traditional BSM implementation. A simplified, more achievable approach is needed in order to fulfil the need to provide meaningful business insight from today’s complex IT environments.

In 2011 Netscape founder Mark Andreessen famously described how today’s businesses depend so heavily on applications when he wrote “software is eating the world”. These applications are built with the purpose of supporting whatever the individual business goals are. It seems logical then that organizations should look into the heart of these applications to get a true understanding of how well the business is functioning.

In a previous post I described how this can be achieved using AppDynamics Real-time Business Metrics (RtBM) to enable multiple parts of an IT organisations to access business metrics from within these applications. By instrumenting the key information points within your application code and gathering business metrics in real time such as the number of orders being placed, the amount of revenue per transaction, and more, AppDynamics can enable everyone in your organization to focus on the success or failure of the most important business metrics.

These are very similar goals to those of a traditional BSM project, however in stark contrast to every BSM project I have ever heard of; AppDynamics can be deployed in under an hour, without the need for any costly services as detailed in a previous blog post introducing Real-time Business Metrics.

Instead of interviewing dozens of people, architecting and building complex dependency models, gathering data and analyzing it all to make sense of what is happening, AppDynamics Real time Business Metrics focuses on the key metrics which matter to your business, providing focus and a common measurement for success across IT and The Business.

So before you embark on a long and costly BSM project to understand what is happening in your business, why not download a free trial of AppDynamics and see for yourself; there is an easier way!

IT holds more business influence than they realise

A ‘well oiled’ organization is one where IT and the rest of the business are working together and on the same page. In order to achieve this there needs to be good communication, and for good communication there needs to be a common language.

In most organizations, while IT are striving to achieve their goal of 99.999% availability, the rest of the business is looking to drive additional revenue, increase user satisfaction, and reduce customer churn.

Ultimately everyone should be working towards a common goal: SUCCESS. Unfortunately different teams define their success in different ways and this lack of alignment often results in a mistrust between IT departments and the rest of the business.

Measuring success

Let’s look at how various teams within a typical organization define success today:

Operations:
IT ops teams are responsible for reducing risk, ensuring the application is available and the ‘lights are green’. The number ‘99.9’ can either be IT Ops best friend or its worst enemy. Availability targets such as these are often the only measure of ops success or failure, meaning many of the other things you are doing often go unnoticed.

Availability targets don’t show business insight, or the positive impact you’re having on the business. For instance, how much did performance improve after you implemented that change last week? Has the average order size increased? How many additional orders can the application process since re-platforming? Is anyone measuring what the performance improvement gains were for that change you implemented last week?

Development:
Dev teams are focussed on change. The Business demands they release more frequently, with more features, less defects, less resources and often less sleep! Dev teams are often targeted according to the number of updates and changes they can release. But nobody is measuring the success of these changes. Can anyone in your dev team demonstrate what the impact of your last code release was? Did revenues climb? Were users more satisfied? Were there an increased number of orders placed?

‘The Business’:
The business is focussed on targets; last month’s achievements and end of year goals. This means they concentrate on the past and the future, but have little or no idea what impact IT is having on the business in the present. Consulting a data warehouse to gather ‘Business Intelligence’ at the end of the month does not allow you to keep your finger on the pulse of the business.

With everyone focussing on different targets there is no real alignment to the overall business goals between different parts of an organization. One reason for this disconnect is due to the lack of meaningful shared metrics. More specifically, it’s access to these metrics in real-time that is the missing link.

If I asked how much revenue has passed through your application since reading this blogpost, or what impact your last code release had on customer adoption, how quickly could you find the answers? How quickly could anyone in your organization find the answers?

What if answers to these questions only took seconds?

Monitoring the Business in Real-time

In a previous post, I introduced AppDynamics Real-time Business Metrics which enables you to easily collect, auto-baseline, alert, and report on the Business data that is flowing through your applications… as it’s really happening.

This post demonstrates how to configure AppDynamics to extract all checkout revenue values from every business transaction and make this available as a new metric “Checkout Revenue” which can be reported in real-time just like any other AppDynamics metric.

With IT Ops, Dev and Business Owners all supporting business critical applications that are responsible for generating revenue, it is a great example of a business metric that could be used by every team to measure success.

Let’s look at a few examples of how this could change the way you do business, if everyone was jointly focussed on the same business metric.

Outage cost
The below example shows the revenue per minute vs. the response time per minute of an application. This application has obviously suffered an outage that lasted approximately 50 mins and it’s clear to see the impact it has had on the business in lost revenue. The short spike/increase in revenue seen after the outage indicates users who returned to complete their transaction, but this is not enough to recover the lost revenue for the period.

RtBM - outage

Impact of agile releases
This example shows the result of a performance improvement program that has taken place. The overall response time has improved by over a second across three code releases and you can clearly see the additional revenue that has been generated as a result of the code releases.

RtBM - agile releases

Here a 1 second improvement in response time has increased the revenue being generated by the online booking system by more than 30%. The value a development team is delivering back to the business is clearly visible with each new release, allowing developers to focus on the areas that drive the most return and quantify the value they are delivering.

Marketing campaign
This example is a little more complex. At midday there is a massive increase in the number of people visiting this eCommerce website due to an expensive TV advertising campaign. The increased load on the system has resulted in a small increase in the overall response time but nothing too significant. However, despite the increased traffic to the site, the revenue has not improved. If we take a look at the Number of Checkouts, which is a second Business Metric that has been configured, it’s clear the advertising campaign has driven additional users to the site, but these users have not generated additional revenue.

RtBM - marketing

Common metrics for common success

With traditional methods of measuring success in different ways it’s impossible to to align towards a common goal. This creates silo’d working environments that make it impossible for teams to collaborate and prioritise.

By enabling all parts of the business to focus on the business metrics that really matter, organizations benefit from being able to proactively prioritise and resolve issues when they occur. It helps IT truly align with the priorities and needs of the business, allowing them to speak the same language and manage the bottom line. For example, after implementing AppDynamics Real-time Business Metrics Doug Strick, who is the Internet Application Admin at Garmin, said the following:

“We can now understand how the application is growing over time. This data will prove invaluable in guiding future decisions in IT.”
-Doug Strick, Internet Application Admin

AppDynamics Real-time Business Metrics enable you to identify business challenges and react to them immediately, instead of waiting hours, days or even weeks for answers. Correlating performance, user experience, and Business metrics together in real-time and in one place.

If you want to capture the business performance and measure your success against it in real-time , you can get started today with Real-time Business Metrics by signing up and taking a free trial of AppDynamics Pro here.