Unified Monitoring: This Is What The Future Looks Like

Seven years ago, when we founded this company, we had a vision of “one solution to rule them all” for software monitoring and application intelligence — one comprehensive platform that would capture user experience on any device, anywhere; trace business transaction performance across an entire application ecosystem; equip teams to quickly identify and resolve issues; correlate application data with business outcomes (at last!); and do it all through a single intuitive, unified interface that would help run and grow the business.

With AppDynamics Unified Monitoring, introduced with our Summer ’15 Release, that vision comes to life, now.

Now, one single solution traces business transactions from the end user all the way through the backend, including everything in between, whether it’s servers, databases, third-party API calls, Docker containers, whatever, and pinpoints the source of application issues in real time, at the code level. So teams can act quickly to resolve issues before they impact the user and the business.

“One solution to rule them all”

Recent research from EMA says that 65 percent of enterprises employ 10 or more individual monitoring tools to keep tabs on the health of their applications and associated infrastructure. With so many tools, how do you know where to look first? How do you connect the dots to know which issues are impacting application performance, and which are not? Few things perform at 100 percent, 100 percent of the time. So how do you know if it’s that minor degradation in database performance that’s slowing down your checkout process and spiking user abandonment? Or is it a server hiccup? Or a slow SQL query? (This might be why it takes 77 percent of enterprises more than five people-hours to fix the average performance issue, according to the same EMA study.)

Some enterprise IT departments try to duct-tape their siloed tools together to create some kind of broader vision, but it’s impossible to avoid the blind spots and limitations of each, and hard to get a single, accurate picture of everything that’s impacting your application. We call this “Frankenmonitoring,” because it’s a stitched-together ‘bottoms up’ monster of a monitoring solution. A certain private equity merger in the APM market announced last week comes to mind. That approach of cobbling together disparate tools and technologies is the exact opposite of our elegant top down Unified Monitoring solution which leverages a single integrated code base to tag and follow the business transaction to ensure and optimize customer experience.

At AppDynamics, we recognized from the beginning that the only way to create a truly unified monitoring solution was to build it from the ground up. And that’s exactly what we did, to provide comprehensive, end-to-end, application-centric monitoring in context of business transactions.

Why the business transaction matters

If you’re at all familiar with AppDynamics, you know we spend a lot of time talking, writing and generally waxing poetic about business transactions. We do that with good reason, because we believe it’s the heart of the most effective monitoring strategies, and it’s central to our Unified Monitoring solution.

Think about it. You’re not really monitoring code, or infrastructure, or even users. What would that tell you? Monitoring these elements in isolation will not provide you with the insights that you need to know  you’re successfully serving the user and generating positive results for your bottom line.

The user is only interested in the experience and performance of the the transaction they’re trying to complete, whether it’s a simple status update or a multi-step e-commerce interaction. Therefore what IT operations should focus on is the how the application and its supporting infrastructure — servers, databases, containers, microservices, etc. — handles the transaction. Because what matters most to the enterprise is to provide great customer experience and enable customers to successfully execute transactions. Because successful transactions are money in the bank.

And from a practical standpoint, transactions are the one thing that encompass the entire infrastructure, and so are the bellwethers for performance of everything. So when your monitoring focuses on business transactions, you can quickly identify and resolve the issues that impact users and the business, wherever they occur.

Welcome to the future of monitoring

AppDynamics Unified Monitoring is the industry’s first application-centric, integrated monitoring platform that traces every transaction, both synchronous and asynchronous, from end user through the entire application environment. It’s built to work as one solution, with one UI, one data platform, one installation.

Screen Shot 2015-06-22 at 12.36.45 PM

What exactly does it monitor? Pretty much everything that impacts business transactions: Applications and browsers, code, databases, servers, VMs, containers, storage, the network, third-party services/APIs, logs. As part of our Unified Monitoring solution, we’ve added Browser Synthetic Monitoring (beta); introduced enhanced and more powerful Server Monitoring (beta); and added support for Python applications, Apache Web Servers, and single-page Angular.js applications. Amazon Web Services and Docker containers are supported. And of course, we’ve tweaked and enhanced various functionality throughout the platform.

Role-relevant views.

While the great strength of Unified Monitoring is its comprehensive view of applications and their transactions, no matter how widely distributed or complex, it also provides situationally aware views and reports. Application support teams can get a consolidated view of potential problems, which they can use to forward the issue to the right experts. And specialists can see and get detailed data on whatever links in the chain are their responsibility — the database team can drill into detailed database stats, for example, in addition to having the big-picture ecosystem view. And finally, teams can come together in a Virtual War Room when needed for coordinated crisis management or ongoing performance optimization.

And it’s easy!

You have flexible deployment options with Unified Monitoring: on-premises, SaaS, or hybrid. There’s just one installation, with minimal manual configuration. Unified Monitoring auto-discovers and maps complex transaction flows, and automatically updates when there are changes. It automatically, dynamically determines performance baselines, and uses them as a basis for reporting. And it consumes less than two percent overhead, so it’s production-ready out of the box.

The days of toolkits heavy with single-purpose, siloed tools are fast coming to an end. AppDynamics Unified Monitoring is here to accelerate their demise, with a comprehensive solution that provides a single, unified, end-to-end view of what matters most: the success of business transactions, resulting in satisfied users and positive business outcomes.

Dispatch from AppSphere #2: Transforming the Future

The AppSphere general sessions kicked off Monday afternoon with a full house in the beautiful Chelsea Theater at the Cosmopolitan. Joe Sexton, AppDynamics president of worldwide field operations, opened the afternoon sessions, followed by Founder and CEO Jyoti Bansal, who presented the Application Intelligence address.
unnamed (5)
I couldn’t help but be struck by what a powerful group of individuals filled the auditorium. To the average onlooker, they may have appeared to be a bunch of brainy computer types. But in reality, these are the people who keep the country’s and the world’s money and goods flowing, the internet working, the news and entertainment coming. Without these people whose focus is how well the software works, our world would be a poorer place in so many ways. It’s such an honor to have them all come together to explore the present and future of application intelligence with us. And I’m glad they’re not all flying on the same plane!

In his Application Intelligence address, Jyoti Bansal described the “generational architectural shift” that’s been evolving with increasing rapidity over the past five years, as applications become more complex and distributed, and mobile, cloud, big data, and the internet of things shape the creation and consumption of digital applications. To thrive in this new world order will require greater communication and cooperation than ever between development (who talks about code), operations (who talks about fixing things), and business (who talks about revenue and conversion). The AppDynamics vision is that all the stakeholders can rally around the business transaction as the common unit of management.
unnamed (2)
Jyoti went on to describe how the AppDynamics Application Intelligence platform addresses the challenges of this generational shift, and used that as a launch pad to describe the exciting advances in the new, soon-to-be-available AppDynamics Fall 2014 Release, specifically:

Next-Generation Platform Architecture that increases capacity exponentially, offering big data scale, easier manageability, and broad extensibility.

Unified Monitoring with one platform for all monitoring needs, and now the full integration of database monitoring into the main AppDynamics platform, as well as C/C++ monitoring.

Mobile First, with Mobile Real-User Monitoring that delivers behavioral analytics, performance monitoring and troubleshooting, and crash analytics.

DevOps Collaboration facilitated by the new Virtual War Room, which gives teams a common space for diverse and dispersed teams to communicate and troubleshoot together.

Application Analytics, which, through automated data collection in real time with business context, facilitates a host of powerful analysis and insights.

AppDynamics CTO Bhaskar Sunkara provided live demos of the Virtual War Room and Application Analytics for an audience that appeared delighted by the new functionality.

“If you can’t monitor it, you can’t manage it.”

Bill Platt, general manager of Amazon Web Services, delivered the day’s keynote around a “monitor it” mantra. The scope and scale of Amazon Web Services as Bill described it is simply mind-numbing, and the task of monitoring it is truly monumental. Some key insights Bill put forth about monitoring:

1. Expect lots of peaks and cliffs
2. Rates of change matter
3. Anomalies matter — they’re almost always the start of something
4. 1 second is the new 1 minute
5. Source of most monitoring data: Logs

(For the record, AppDynamics Application Analytics aims to change #5.)
unnamed (1)

Words of success for DevOps, BizDevOps

Spencer Schwab, staff application engineer for Citrix Systems — whose GoToMeeting product we use almost every day — delivered an inspirational session on his experiences evangelizing for a DevOps, and ultimately a BizDevOps model, for Citrix Systems. First, he described the process as he sees it for getting things done as five P’s: Passion, partnerships, positivity, persistence, and patience.

He went on to describe how these principles apply to effecting the cultural change and collaborative attitude required for devops, and the ultimate desirability to incorporate business into the equation as “bizdevops,” for greater alignment from the top down, more effective resolution of conflicting priorities, and greater information transparency.

He finished with another set of words he considers foundational for success and strategy: Failure, focus, fervor, faith, and forgiveness.
unnamed

A big finish for day one

The last session of the day was our cloud panel, which I’ll cover in a separate post. The day ended with a large number of attendees clustered around the demo stations in the Developer Lounge — a beautiful sight to see! — before making their way to dinner and networking in the Sponsor Pavillion.

By any measure, the first day of the first AppSphere was a huge success. Day two promises to be just as exciting.

Stay up to date with AppDynamics AppSphere on Twitter using the hashtag #AppSphere14

The new battleground in Analytics Part 2: Transforming APM into Application Intelligence

In Part 1 of this series, I talked about my transition from the world of Analytics into the APM space and my assertion that APM is simply another form of Analytics. It’s turning large amounts of data into information you can use to take quick action. Like other forms of Analytics it can help you drive revenue, reduce costs and mitigate risks. The impact to the business can be truly compelling — but that’s only the beginning of the story.

As powerful as traditional APM is, it’s very quickly evolving into what we call Application Intelligence. Our Application Intelligence Platform allows you to do three key things:  See, Act and Know.

  • See what’s happening in real-time:  We enable an integrated view of real-time user experience, application performance and infrastructure capacity.
  • Act fast to resolve issues:  We provide the ability to automate resolution of application or infrastructure bottlenecks – in production.
  • Know that you’re running at your best:  We enable deep, real-time analytics to help businesses make better decisions and create bigger impact – all with certainty and confidence.

It gives IT the operational visibility and control they need. It gives end users the great experience they demand. It gives the business real-time insight into business performance that most have never had.

In this post I’m going to focus on real-time analytics, but before I dive into how it works let me start with the context of traditional analytics. In the past people have tried to directly query the underlying databases of an application.  Unfortunately these databases are primarily designed for inserting small amounts of data. When complex queries run against it there can be a significant performance hit to the application, which is a very bad thing. The workaround was an Operational Data Store (ODS) that would replicate the data in its current structure so you could run queries without impacting the application’s performance. However, since the structure was designed to support transactional applications with short reads and writes, query performance and the ability to ask complicated questions easily weren’t good enough. This meant extracting the data from the ODS, transforming it and loading it into a data mart or data warehouse in a structure that was much easier to query. Given ever-growing data volumes query performance was still an issue, so the next step was adding indexes and aggregates. These workarounds put the end business user further and further away from real-time information. Often the delay in usable information means you’re looking at information that’s hours, days, or even weeks old. In the last few years there have been some disruptive technologies introduced that can simplify this. Though they’re actually quite effective, they come with an exorbitant price tag.

The Application Intelligence approach is considerably different. To get the benefits of Application Intelligence, the complete path the transaction takes through the end user device, the application server tiers, middleware, third party API calls, databases etc. is fully instrumented and the transaction context is propagated in real time. That’s how it sees everything and identifies performance problems and bottlenecks, often before end users are impacted. It also provides the opportunity to extract payload information such as the quantity of items in a shopping cart, revenue in executed transactions, dollar amount of funds transferred and so much more. AppDynamics introduced this concept with Real-Time Business Metrics. It’s allowed customers access to metrics like revenue, number of new accounts, support ticket volume, items shipped and campaign effectiveness in real-time. You can even correlate them with the performance of the application to see how changes in response times for end users impact business performance. It plucks all of this information out as it passes through the application and stores it in a data platform that’s easy to query. AppDynamics is taking this concept to the next level in the recently announced Transaction Analytics, which will extend the scale of the underlying data platform, enabling larger data volumes, more complex queries and impressive visualizations. In real time you can answer questions like:

  • What was the revenue impact by product category associated with the two marketing campaigns we ran on Black Friday?
  • Which Android users experienced failed checkout transactions in the last 24 hours, and what was in their cart at the time?
  • How many customers experienced a slow ‘submit reservation’ transaction in the last hour from a Chrome browser in New York and what was the total dollar amount of those reservations?
  • How many transactions originated from Tier 1 partners over the last 90 days and what was the resource utilization and revenue associated with those requests?

This is the transformation from APM to Application Intelligence. IT has leveraged APM to drive business success through application performance and efficiency. Application Intelligence extends beyond those benefits, allowing the business to use information to make better decisions and differentiate against the competition. In an era where IT and the CIO are expected to lead the business into the future, Application Intelligence is an incredible opportunity for leaders to drive success.

If you’d like to get started using AppDynamics to try business & operational analytics out for yourself, you can create a free account and start monitoring and analyzing your applications today.  If you are interested in being a part of our upcoming Transaction Analytics beta program, you can find more information here.

 

Announcing AppDynamics Application Intelligence Platform

AppDynamics is excited to unveil details behind our new Application Intelligence Platform which is the technology foundation for the AppDynamics portfolio of performance monitoring, automation and analytics solutions (transaction analytics was also announced today).

Today’s software defined businesses need the ability to see everything in their application environment, act with certainty, and know faster. See, Act, & Know, the three core pillars of Application Intelligence.

See Act Know

Making sense of all the data, transactions, events and user interactions in these environments is increasingly difficult as the volume, variety and velocity of data grows. Traditional approaches to solving this problem are typically silo’d in their view providing limited business correlation and only present static data. In order to keep up with todays complexity, what’s needed is a platform that can collect, understand, and act on the data and translate that into actionable information about your business.

platform page - application intelligence platform graphic2

The AppDynamics Application Intelligence Platform is uniquely able to deliver rich performance data, learning, and analytics, combined with the flexibility to adapt to virtually any infrastructure or software environment. The platform was designed for the modern day application, including SOA architectures, frequent agile releases, cloud deployments, big data technologies and mobile delivery – and maintains a unique low-overhead architecture. This gives you the ability to instantly see the impact of production changes as you work, and adapt to optimize performance and minimize bottlenecks while ensuring the revenues are maximized.

Today’s consumer-driven world moves faster than ever. Everything from your customer experience to revenue depends on your business-critical applications performing at their highest level. AppDynamics delivers real-time access to every aspect of your performance, so you can anticipate problems, resolve them automatically, and make smarter, more certain business decisions.  This includes detailed understanding of the user experience, business data such as real time revenue metrics, deep application performance and infrastructure data. All collected within context of business & operational transactions.

Platform Design Principles:

The Application Intelligence Platform is the underlying architecture that we use to deliver all of the capabilities a modern business needs to keep their business running at peak performance and use application intelligence as a competitive advantage.

The core design principles behind the AppDynamics Application Intelligence Platform include:

Scalability:

The interactions customers have with the business are highly dynamic, environments are larger and more hybrid and organizations are demanding IT transform the business faster than ever. AppDynamics monitors many of the largest, most complex enterprise application environments in the world, supporting environments up to 10,000 nodes on a single management server. This is critical for customers who do not want to manage and maintain a farm of servers.

Lightweight:

Our platform was designed to be a production tool from the very beginning, giving it a unique low-overhead architecture. Intelligent data collection to ensure the lowest production overhead in the market. AppDynamics correlates end user business transaction details with completion status (success, error, exception), response times, and all other data points measured at any given time. It automatically analyzes the entire data set to provide information from which to draw conclusions and take the appropriate action.

Self-learning:

Auto-instrumentation, and minimal configuration enables simple and intuitive operation for any size business. AppDynamics has a unique ability to instrument your application, learning its transactions, code execution behavior, and normal performance patterns. It can adapt and adjust instrumentation automatically when you change your application-no manual configuration needed.

Open and Extensible:

The AppDynamics platform is open, extensible, and interoperable to fit any business need. Through the AppDynamics Exchange our community comes together to share knowledge and contribute back over 100 extensions that provide deep integrations to the tools you already use like Splunk, Apica, PagerDuty, and Amazon Web Services. From monitoring Amazon Web Services costs to MEAS mainframes our extensions allow you to leverage the tools you already have in place. We believe you own your data and it should be easy to consume and analyze which is why we provide REST APIs and SDKs available on GitHub that make it as easy as possible to get started.

Flexible:

The platform can be deployed in any operating model, including on-premise, SaaS, private cloud, or a hybrid combination. It’s always the exact same platform and there are no cost implications associated regardless of deployment model.

Secure:

Provides enterprise grade security with Role-Based Access Control (RBAC), LDAP, and SaaS certifications that guarantee security and compliance.

What services are delivered on the Application Intelligence Platform?

The AppDynamics Application Intelligence Platform gives companies the foundation they need to monitor performance and extract meaningful analytics from their business applications. Software-defined businesses must be certain that their most complex, business-critical applications are performing at the highest level; and be certain that the data and information generated by these applications can be harnessed for ongoing business advantage and impact. AppDynamics’ Application Intelligence platform enables customers to monitor, respond, and analyze their application environment.

Monitor

Application Performance Management

Mobile Application Monitoring

End User Experience Management

Database/NoSQL/BigData Monitoring

Infrastructure Monitoring

Respond

Run Book Automation

Cloud Auto-Scaling

Alerting

Analyze

Transaction Analytics (Beta)

analytics

Real-Time Business Metrics

analytics page - real-time business metrics graphic2

Operational Analytics

scalability analysis

The Application Intelligence Platform delivers this functionality by collecting data from across the application environment, processing & correlating it, and converting that data into knowledge for customers.

Collecting Data:

  • Instrumentation allows us to watch every line of code.
  • Distributed transaction tracing follows the business transaction across all tiers of the application, not just limited to a single node.

  • Transaction auto-learning engine inspects execution code, payload, libraries and methods so you are not left in the dark.

  • Real-time service discovery renders architecture topology to clearly and automatically map critical relationships and dependencies..

  • The smart agent filters and transfers data to the management server securely and in real-time.

Processing & Correlating Data

  • Real-time stream processing allows for complex event processing of metric and event streams.
  • Time series clustering and analytics provides the ability to index and manage time series by auto rolling up, purging or clustering data sets by time increments.

  • Behavioral learning is an engine that continuously adjusts dynamic baselines for an automated manageable data set.

  • Unstructured and structured big data indexing creates a data warehouse for different types of application data which is seamlessly correlated and processed.

  • Event correlation shows the relationship of performance, change, and business events.

Converting Data Into Knowledge

  • Intuitive user interface (UI) allows for a simple and easy-to-use platform for business users, developers and operations teams.

  • Dynamic flow maps create a clear and understandable visual representation of the application topology, so organizations understand what is happening in the big picture.

  • Cross-correlated drill down from anywhere in the UI, enabling pin-point accuracy across multiple nodes and machines.

  • Real-time correlation of performance metrics and business metrics create a business understanding of data never before delivered.

  • Compare and analyze the side-by side performance of different versions of an application.

  • Custom drag and drop HTML5 dashboards can be configured to any specification to deliver clear actionable results.

  • Query language allows easy, targeted data discovery.

In today’s competitive and demanding environment, organizations need to leverage Application Intelligence for ongoing business advantage. The AppDynamics Application Intelligence Platform enables customers to see everything across all front and back-end interactions: business transaction and code-level visibility with no blind spots.

Maximize digital business performance through an open, integrated solution that delivers real-time monitoring, actionable business and operational insights, and automated issue resolution across applications, infrastructure, and user experience. Take five minutes to get complete visibility and control into the applications that power your software-defined business. Get started with the AppDynamics Application Intelligence Platform today.