The future of Ops, part 1

The disruption of industries through software

Marc Andreessen famously stated in 2011 that “software is eating the world”. The world runs on software defined businesses. These businesses realize that in order to be efficient and stay ahead of the competition they must innovate or they will die. Technology is no longer secondary to your business, but is now the actual business.

Nowadays there is an app for nearly everything and consumers have the expectation most processes are automated. Access to these apps is ubiquitously available — from the web and mobile. Every disruptive billion dollar company in the last decade has innovated through applications by fundamentally changing the market and user experience. Companies like Netflix, Uber, Square, Tesla, Nest, Instacart, and many others have capitalized on this new user experience and catering to their elevated expectations. The disruption stems from an improved user experience, and enabled through technology.

The evolution of application complexity

Gone are the days where applications were this simple:

Screen Shot 2014-07-30 at 6.34.55 PM

The reality nowadays is applications are extremely complex and distributed using several platforms. Most application architecture we come across utilize several languages such as Java, .NET, PHP, and Node.js. Operations becomes even more complex with virtualization and cloud environments, deploying to containers, and managing application made up of many microservices.

Screen Shot 2014-07-17 at 3.50.39 PM (2)

It is not DevOps, it’s the next generation of Ops

Most people and companies abuse the term DevOps to no end. It is a bit embarrassing, but buzzwords flow rampant on an expo floor of a technology convention. The reality is quite simply that the operations tools engineers use to build and manage complex applications have evolved to match the complexity. I believe the operations complexity breaks down into a few main categories: infrastructure automation, configuration management, deployment automation, log management, performance management, and monitoring.

The evolution of the Ops problem

The modern operations reality is that the cloud is the standard platform, operations are automated through code, testing and quality assurance are automated through code, deployments are automated through code, and monitoring and instrumentation is critical to success.

iStock_000023396874_Small

The DevOps Report from Puppet Labs surveyed the DevOps community and found some interesting results, most notably: “companies with high-performing IT organizations are twice as likely to exceed their profitability, market share and productivity goals.”

The report also found successful DevOps teams tended to share these characteristics:

  • use continuous delivery to ensure consistent and stable deployments
  • leverage version control not just for code, but infrastructure and configuration to track and manage all environments states
  • automate testing to have confidence about the quality of every release
  • invest in monitoring and logging to be proactive about problems
  • correlate IT performance with organizational performance

Download the entire report from Puppet Labs

The enterprise catch up game

Most enterprises are not able to adopt cutting edge technology at a rapid pace so they are in a constant state of migration and catching up. Furthermore, their challenges are exacerbated when dealing with hybrid environments consisting of on-premise legacy systems combined with new public and private cloud environments. Larger, less flexible, legacy companies are just starting to invest in the latest generation of programming languages such as Scala, Node.js, and Go and nosql datastores like Cassandra and Redis.

Though enterprises may experience the challenges adapting to the latest operations trends, there are several tools out there which will help ease the transition. A good APM solution helps foster DevOps best practices and increase collaboration between the traditionally separated Dev and Ops teams.

Don’t believe me? Try AppDynamics for FREE today!

The answer for government applications migrating to the cloud: visibility

Recently, I’ve had several conversations with US Federal Government Agencies about monitoring applications moving to FedRAMP (The Federal Risk and Authorization Management Program) data centers. Because of the Government’s Cloud First policy, which mandates that agencies take full advantage of cloud computing benefits, agencies are increasingly forced to move application outside of their own data centers. With less control on the infrastructure, the new focus is now on the performance and availability of their applications running in the cloud. Agencies want an assurance their applications are running at the same level of performance (or better) once they make the move. This is where I believe an APM solution like AppDynamics is a perfect fit to mitigate risk by providing agencies 100% visibility into their application performance.

With cloud environments, I’ve found traditional approaches for monitoring simply don’t work. This is because the agencies have limited access to the underlying IT infrastructure in the cloud. Federal agencies need the help of companies such as AppDynamics to provide them visibility into application performance from the end user down to the infrastructure to truly understand the health of their critical applications.

Before the cloud, when agencies ran applications on premise, they had the physical access to the underlying IT infrastructure. Which meant they could deploy element-monitoring tools and gain access to the network to try to infer the health of the applications. At AppDynamics, we take a modern approach to APM by monitoring performance from the top down through the concept of Business Transactions. The Business Transaction is the mechanism by which AppDynamics orders and aligns load traffic (response time, throughput, and so on) with the business perspective (For example, Login, Search, etc.).

Screen Shot 2014-07-28 at 3.11.44 PM

I’ve found AppDynamics is flexible to help customers monitor applications both on premises and in cloud environments. AppDynamics was designed from the beginning to be cloud portable by working within the constraints of cloud environments. The three main reasons why I believe AppDynamics is perfect for federal agencies to monitor critical cloud applications are:

Firstly, AppDynamics is an all software agent-based solution that doesn’t require a high network bandwidth connection. The agents can report across the Internet using a one-way HTTP(s) connection back to the controller software. This means agency applications that span multiple FedRAMP clouds can be can be monitored with a single AppDynamics controller. The controller has the intelligence to stitch the transactions that flow between clouds into one view (think – highly layered Service Oriented Architectures). The self-discovering flow map and single pane of glass view is vital to obtain the necessary visibility of your application.Screen Shot 2014-07-17 at 3.50.39 PM (2)

Secondly, AppDynamics doesn’t require privileged network access to components such as a SPAN port or a network TAP. The traditional approach for End User Monitoring was to receive a copy of the traffic to and from the application and decrypt the packets to make sense of the end user experience. This approach fails in cloud environments since the cloud providers typically will not provide access to the underlying physical infrastructure. AppDynamics captures end user experience in a cloud-friendly way through JavaScript on the browser. Through this approach, AppDynamics not only captures and correlates all end user activity with the application code execution, but also measures the page render time in the end user’s browser.

EUM-Analyze (1)

By not requiring privileged network access and high bandwidth management network, AppDynamics can follow the workload as applications are migrated to the cloud. Agencies will have visibility into the before and after state of their applications performance.

Thirdly, AppDynamics can help Agencies with scaling their applications automatically by utilizing our cloud auto-scaling capabilities. Cloud auto-scaling decisions are typically made based on infrastructure metrics such as CPU Utilization. However, I believe the better and more accurate way to auto-scale in Cloud environments is to make decisions based on application metrics such as requests per minute. For more information about cloud auto-scaling please read: http://blog.appdynamics.com/cloud/cloud-auto-scaling-using-appdynamics/

Other reasons why AppDynamics is a perfect solution for modern applications moving to the cloud are:

  1. It’s easy to deploy
  2. It requires minimal configurations (Instrumentation works out of the box)
  3. It requires minimal care and feeding on an ongoing basis (Supports rapid change with Agile development)
  4. It has a built-in Dynamic Baselining Engine to proactively alert teams of performance issues

For agencies to be successful running applications in the cloud they need end-to-end visibility into their application performance. With AppDynamics, federal agencies can finally migrate to the cloud without impacting or worrying about their applications. As all critical software applications become more complex, the visibility AppDynamics provides isn’t just a luxury feature, it’s a necessity.

Take five minutes to get complete visibility into the performance of your cloud applications with AppDynamics today.

Gartner, IBM, + AWS keynotes announced for AppSphere 2014!

We are excited to announce that AppSphere 2014 will feature keynotes from Jonah Kowall, Research Vice President, IT Operations at Gartner; Lance Crosby, Chief Executive Officer of SoftLayer, an IBM Company; and Bill Platt, General Manager of Amazon Web Services.

Screen Shot 2014-07-30 at 9.53.33 AM

 

Secure your seat for these must-see sessions at AppDynamics AppSphere 2014 — November 3-5, 2014 at The Cosmopolitan of Las Vegas.

The conference will also include a panel of cloud experts from Microsoft, Google, Red Hat, and AWS discussing the future of the Cloud. Including breakout sessions featuring industry thought leaders from American Express, MongoDB, CoreLogic, Lithium, and more. See the full list of speakers at AppSphere 2014.

Jonah Kowall’s bio:

Jonah Kowall is a Research VP in Gartner’s IT Operations Management group. He focuses on application performance monitoring (APM), event correlation and analysis (ECA), network fault monitoring (NFM), network performance monitoring (NPM), network configuration and change management (NCCM), and general system and infrastructure monitoring technologies (storage, virtualization, public cloud and private cloud). He also covers SLA monitoring of services and applications. Increasing SaaS delivery options of monitoring technologies include the monitoring of mobile device and mobile application performance. These technologies are the foundation of operations, and they exist to make incident, problem and change management possible for these teams.

Lance Crosby’s bio:

Lance Crosby is the founder and visionary behind SoftLayer, an IBM Company. He is responsible for managing the overall success and growth of the company. Prior to SoftLayer, Mr. Crosby spent five years with The Planet as chief operations officer where he managed all operations including sales, provisioning, product selection, managed services, application development, engineering, information security, business development, and legal departments. Serving in prior executive management roles at Catalog.com and Dallas Internet, he brings extensive experience in leadership and management skills coupled with extensive technical skills in software and hardware environments. Mr. Crosby holds bachelors’ degrees in finance and management, a Master of Science in finance and Master of Business Administration from Tarleton State University, a part of the Texas A&M University System. Mr. Crosby’s legal education includes a Juris Doctor from Texas Wesleyan School of Law. He is a licensed attorney in the state of Texas and is an active member of the State Bar of Texas.

Bill Platt’s bio:

Bill Platt is a General Manager for Amazon Web Services. Bill came to Amazon after being the Executive Vice President of Products at Engine Yard, a Platform as a Service startup which was a large early adopter of Amazon Web Services. Prior, he had senior leadership roles as a product executive at NComputing and Wyse Technology in the desktop virtualization market and a fifteen year career at Sun Microsystems where he led businesses including North American and Asia Pacific Services as well as Solaris and the Java Enterprise System. He has a degree in Systems Engineering from the University of Pennsylvania and a business degree from the Wharton School of Business.

Interested in speaking this year? Please join our Call for Papers.

We look forward to seeing you in November!

AppDynamics Closes $120 million in Growth Funding — The Best is Yet to Come

We’ve come a long way since 2008, when I was writing code in my home at night and pitching venture capitalists during the day. Six years later, we have more than 450 employees in offices across the world: San Francisco, London, Australia, Singapore, Tokyo, and India. Today we are proud to announce we closed $120 million in growth financing to accelerate our technical innovation and sales expansion.

AppDynamics has raised more than $200 million in total capital to-date. While growing AppDynamics, we always remained laser-focused on creating the world’s next great software company. Since our founding, we have kept a keen eye on the ways the world is changing, and we work tirelessly to anticipate the needs of businesses.

The Software-Defined Businessess have now emerged and many companies are embarking on a journey of digital transformation.  Software is now the primary source of differentiation for the modern enterprise. Companies have migrated from “systems of record” to “systems of engagement” in order to get closer to customers and compete to win. The anticipation of this fundamental shift was the genesis and the vision for the founding of AppDynamics and the technical thesis for the Application Intelligence Platform.

We have a big and bold technology mission – to “instrument” each and every line of application code and use that data to provide insights to Developers, Operations and Business users to optimize their business and deliver a great consumer experience. Our portfolio is uniquely built to optimize software operations in the most complex environments. As a result, we have secured more than 1,300 of the most demanding enterprise customers. This latest round of funding validates our position as the market leader in application intelligence, and recognition for our vision, strategy and execution.

This funding comes at a pivotal time in the growth of AppDynamics. While we will always maintain the zeitgeist of a startup company, we are no longer the new kids on the block and our customer centric culture is at the heart of everything we do.  We are immensely proud of our customer satisfaction and work relentlessly every day to earn and keep our customers’ trust. With an impressive list of customers, a stellar team comprised of the brightest minds in the industry, and an inspired and valued set of offerings, we are well on our way to becoming the world’s next great software company. While our upward trajectory is already the talk of the software industry, we are really just getting started — the best is yet to come.

AppDynamics Responds to CA’s Meritless Claims

We issued a press release today in response to a press release earlier today by CA who alleged that it has ownership over a small amount of AppDynamics source code written by AppDynamics’ founders. We strongly believe that these claims are untruthful and intentionally misleading. All AppDynamics code was written independently by its founders and is clearly protected under California law and public policy, including California Labor Code Section 2870.

It is telling that CA first attempted to add these claims to its patent infringement lawsuit in New York, and that this attempt failed miserably when its motion was soundly thrown out by the court. We then promptly filed a lawsuit in California to determine once and for all that CA’s claims are without merit. Last Friday, CA filed its answer to AppDynamics’ complaint, which prompted CA’s latest press release.

CA has an extensive track record of harassing its customers and competitors through meritless lawsuits. It is unfortunate, though not surprising, that CA continues to resort to legal tactics, rather than competing in good faith in the market through innovation and customer success. These tactics are a clear sign that CA is out of touch with its customers and they are not working, as we continues to displace CA on a massive scale. We look forward to continuing to beat CA in the market as well as in court.

One bad App can ruin your health

How do users react when their mobile applications aren’t performing up to expectations? How frustrated do they become? These were answers we sought out to find when we started the App Attention Span Research with the University of London. We wanted to find quantitative proof to see how poor performing apps affect our emotions, and discovered some exciting results. We all know how it makes us feel but we needed to take a more scientific approach to understanding the emotions poor performing apps drive.

So last month we conducted a joint study with the University of London to discover how users react when the performance of mobile applications aren’t up to expectations. We took actual app users as case studies and put them in a scientific lab – with some really lousy apps for company. No need to have a clandestine “mood” study without users knowing (cough, cough, Facebook, cough), our participants were willing and provided some compelling results.

The study gave us great insight into how glitchy, slow, and unstable apps affect users’ app preference, and how terrible performance impacts their emotions. Sure, we expected the lab research to reveal a certain level of user frustration. However, frustration levels were much higher than we thought. And, as we later found out, this directly leads to a sharp spike in users deleting and uninstalling apps that don’t meet their expectations.

App makers, take note.

The University of London’s Director of innovation Chris Brauer – our research lead – observed, “Users experience a lot of negative emotions and frustrations when trying to complete some digital tasks and apps or Web pages are slow to load.”

It is also clear that our patience wears thin in today’s app-enabled world. As Chris Brauer puts it; “Our attention span demands have adapted dramatically to the available technologies.” In short, if we’re not getting the performance from one app we’ll simply move onto another. Despite this lack of patience, results from the study demonstrated that in many cases, people are now becoming more loyal to an app than a brand.

“Consumer expectations regarding apps are really high, so when people’s experience is not satisfying, they are going to go elsewhere and look for an alternative. It’s therefore very important for app developers and service providers to test and optimize.”

Jonathan Freeman, Professor of Psychology, University of London, and Managing Director of i2 media research at Goldsmiths

Screen Shot 2014-07-17 at 9.26.18 AM

Don’t stress me out.

One thing we found especially surprising was the level emotional distress that inferior mobile app performance caused. Despite how melodramatic this may sound, the impact on our guinea pigs was substantial. The three most common reactions were frustration, stress, and disappointment. Yes: stress.

As one of our UK case studies, Will Marshall, put it: “It is very frustrating and can make you anxious that apps often don’t work just when you need them to be there, like on a train. Disruption of digital routine can also be very stressful.”

In short, frustrations stem from the high expectations we have for the apps that are never far from our fingertips. We expect them to respond instantaneously and perform without any stalls or crashes; which means the margin of error that app developers have to work with is quickly diminishing. As a result, mobile application performance has never been more critical.

We’ve also created this infographic showcasing some of the interesting data points from the study:


To check out the full report and detailed data, click here.

Relieve your users’ frustration by gaining visibility into the performance of your mobile applications with AppDynamics Mobile RUM today.

For a introduction to AppDynamics Mobile Real-User Monitoring, watch our On-Demand Webinar now.

Improving the quality of mobile applications with Mobile APM

Mobile applications now account for 15 percent of all Internet traffic, with 1.5 billion users worldwide. Done right, a simple iPhone application can reach 50 million users in days. When you have a successful mobile application it is difficult to ensure all of your users have a great experience. While good performance gets a high rating and strong sales in the App Store, poor performance will impact the application’s rating—and can cost a business tens of thousands of dollars for every second of delayed response or app failure.

The reality is often times the organization responsible for infrastructure and monitoring have limited influence with the mobile application owners. Most mobile application owners have visibility into the server-side applications, but limited visibility into the mobile experience. In a recent report from Gartner, Jonah Kowall calls out the growth in the mobile space:

By 2018, 25% of those organizations with current, five-dimensional APM investments will extend them to mobile devices and applications, up from under 1% today.

Monitoring native mobile applications is increasingly challenging due to the sheer amount of platforms, devices, and variability of connectivity. In order to truly understand your users you need to leverage end user experience monitoring (EUEM) to gain visibility into how your actual users perceive your application. You need to utilize Mobile APM to gain visibility into application crashes and hangs, understand the interactions between mobile applications and the server-side, and understand your users through analytics. AppDynamics makes it easy to get complete visibility across your web and mobile applications:

Mobile APM

 

Find out more about the key challenges in improving the quality of mobile applications in a Gartner analyst report written by the leading APM analyst Jonah Kowall.

Take five minutes to get complete visibility into the performance of your mobile applications with AppDynamics Mobile RUM today.

For a introduction to AppDynamics Mobile Real-User Monitoring, watch our On-Demand Webinar now.

Holographic Haystacks — Capturing Business Transactions

Why Archive Every Business Transaction?

The other day I found myself in a discussion with a prospect regarding the merits of capturing complete stack traces for every transaction that flowed through their production system. You see AppDynamics doesn’t do that; we capture problematic business transactions, and this bloke seemed concerned someone would one day ask him for the details of a specific transaction that our method would leave him underprepared to fulfill. As the conversation played on, we explored the cost – both literal, in the sense of hardware required for data warehousing, and virtual, like the compounding impact additional polling during bad times has on performance. Ultimately we arrived at the conclusion that understanding your haystack is better than having the haystack. This point can be subtle, so let’s explore how we got there…

Fullscreen capture 782014 31924 PMSince AppDynamics automatically baselines business transactions, we can very quickly tell the difference between a normal transaction and an outlier, or in the context of this analogy: hay vs. needles. When we encounter a piece of hay we record the metadata of the transaction. You know: how long the strand is, where it resides, who put it in the stack, etc. In fact, the part of me that humbly struggles with homonyms likes to think of this as a piece of “hey.” I imagine someone (a business transaction) flying past me on the highway and waiving “heeeyy!” as I feverishly jot down some key details about their ride. For the most part I already know enough about them, they’re speeding along unabated, just like the plethora of friends to follow, no need to dig too deep, just note the event so we can better understand our “heystack.”

In contrast, when a particular transaction performs several standard deviations outside of the norm, or perhaps worse than a given threshold, we need to know more than “it’s slow.” This is where the context before content idea really starts to come into light. Because we know what a good piece of hay looks like, and because we understand how our virtual haystack looks at any given moment, we have the context to both appreciate and spot the needles – the need for content. Needles are special. They’re shiny. And you probably want to trade them for hay. So we need to know all their specs. We need content.

 

No worries, that’s why AppDynamics captures complete transaction flows for these items. But what if – returning to the highway analogy for a moment – a funeral procession (a bout of bad performance) drives by? The act of gathering information during this kind of needle storm could exacerbate the issue and result in exaggerated wait times, inaccurate information, and a unnecessarily pitiful user experience. Well we have the context to understand when that’s happening so AppDynamics will automatically throttle back needle gathering to ensure our application never negatively impacts yours. Remember the ultimate goal here is to ensure you have all the tools you need to deliver an exceptional user experience, in the short and long term.

Thinking about the long term for a moment … we should probably use this information to improve the performance of the application. So let’s pass the data about the problems along, shall we? But … if all you ever gave your team were needles, they might forget what hay looks like. This lack of context could hinder their ability to remediate issues. To mitigate these risks, AppDynamics will snag the occasional piece of hay… just for comparison’s sake and ensure your team, your developers, have all the context and content they need to improve the performance of your applications.

So as you can see, you don’t need to house an entire haystack to effectively identify and remediate problems. With a clear understanding of what the haystack would look like – a holographic haystack, if you will – a few pieces of hay for reference, and a spotlight on all your needles, you’ll have everything you need to see, act, and know more quickly than ever before.

Also check out how Citrix uses AppDynamics to capture business transactions, increase visibility, and lower their MTTR for their complex applications.

 

Take five minutes to get complete visibility into the performance, and start finding those needles, of your production applications with AppDynamics today.

How to Triage a Busy Thread Count Alert in 14 Minutes

This is a real example of troubleshooting a production application issue provided by an AppDynamics customer. What you are about to see is a combination of run time analytics, adaptive data collection, intelligent alerting, and a proven problem solving workflow. From first alert to DBA handoff took only 14 minutes.

5:26 p.m. – Operations receives an email alert about Busy Threads breaching a threshold. The incident was automatically detected and alerted upon by AppDynamics when the Busy Threads JMX metric shot up to 182.

AppDynamics sends notifications detailing busy thread counts

5:34 p.m. – Details from AppDynamics show that call volume is down, response time is up, errors are up and network I/O is down. Initial suspicion is that the load balancer may be throttling traffic due to poor performance.

Thread_ART_Throughput_Errors

 

Thread_Network

 

Thread_CPU

5:38 p.m. – Company procedure is followed by disabling the server from the load balancer so that it will not receive any more traffic. Recycle of application server is considered as a possible temporary resolution to the issue.

5:40 p.m. – Details from AppDynamics are used to show that transactions are backing up because of a database issue. There is no need to recycle the application server. The issue is handed off to DBA team with full application context for resolution.

 

Thread_Txn_Map

Screenshot showing problematic JDBC call as the culprit.

Screenshot showing problematic JDBC call as the culprit.

Later that day: DBA team fixes the issue and application response time returns to normal. All nodes are restored into the load balancer rotation.

This is an example of a scenario that IT Operations teams deal with regularly. Without having AppDynamics in place to provide fault domain isolation this type of problem usually ends up in a long conference call where all support personnel for this application must participate until service has been restored. There is no need to waste significant company resources any more. Stop the “all hands on deck” madness and see how AppDynamics can help your company today.