Monitoring NGINX with AppDynamics

Did you know the AppDynamics Application Intelligence Platform includes monitoring for hundreds of IT components out of the box, including NGINX and NGINX Plus? They’re all featured in the AppDynamics Exchange.

With the AppDynamics extension for NGINX, you gain powerful capabilities and insights including:

  • Business Context: Application transaction tracing automatically associates IT components with business activities
  • Self-learning: Dynamically baselines behavior, eliminating static threshold configuration and false positives
  • Unified Monitoring: One platform and a single UI for end-user, application, and infrastructure monitoring
  • DevOps Automation: From remediation to cloud capacity scaling, automate for proactivity and efficiency

I’m getting ahead of myself though. Let’s back up a minute and cover the basics.

First and foremost, what exactly is NGINX?

From NGINX, they “accelerates content and application delivery, improves security, facilitates availability and scalability for the busiest websites on the Internet.” Next layer explanation: NGINX (open source) and NGINX Plus (commercial) are an HTTP server and reverse proxy, and an IMAP/POP3 proxy server. And if you want to get technical: NGINX is one of a handful of servers written to address the C10K problem. Unlike traditional servers, NGINX doesn’t rely on threads to handle requests. Instead, it uses a much more scalable event-driven (asynchronous) architecture. This architecture uses small, but more importantly, predictable amounts of memory under load.

That last part is a big deal, especially given the large-scale, complex, distributed web application architectures of today. Which is where AppDynamics comes in.

AppDynamics provides 24/7 visibility and management of your entire application environment, whether you’re running 10 servers or 10 thousand. We do this via our Application Intelligence Platform, a core pillar of which provides Unified Monitoring for your end-users, applications, and infrastructure.

Unifying monitoring sounds great and all, but it’s how we do it that makes the difference, which brings me back to the beginning of this blog.

Business Context and why it matters

Screen Shot 2015-01-28 at 4.13.56 PMTraditional monitoring solutions take a bottom-up approach: start with the infrastructure and attempt to logically associate it with the business. This barely worked with the static, monolithic IT systems of yesterday and is hopelessly inadequate for the service-oriented, high-distributed systems and agile processes of today.

A new top-down approach to monitoring is required, and it starts with the Business Transaction (BT): a unit of measure based on a business-level application transaction (e.g. customer checkout) that is automatically associated with the IT components used for its execution, including application code and infrastructure resources. This is accomplished by automatically tagging and tracing the execution of each and every application transaction through underlying IT systems. I wrote more in depth about this BT-focused method in a previous blog post.

Self-learning, so what?

If you’ve ever been a part of an IT operations team, you know what a pain it is to define altering thresholds. The entire process is predicated on knowing what’s normal, and from there when to be warned and when to be alerted there’s a critical problem. Static thresholds, on the other hand, are too rigid and in danger of either alert storms and/or missing performance issues.

AppDynamics dynamically baselines every metric – from CPU, memory, network, and disk utilization to end-user, transaction, and database query response times. This proactively detects anomalies and alerts intelligently.

Okay, so what about NGINX specifically? Take all the above and apply it to what we track directly from NGINX, including:

  • Active connections
  • Total number of requests
  • Accepted and handled requests
  • Reading, writing, and waiting metrics

Once you have the NGINX extension configured and integrated with AppDynamics, you can see all the NGINX metrics in the AppDynamics Metric Browser. For example, in the screen shot below you can see all the Active Connections to the NGINX during a certain time interval.

Screen Shot 2015-01-28 at 4.11.09 PM

As mentioned earlier, you can monitor the entire business transaction all the way from a mobile device or web browser to the NGINX web server through to the database. For example, you can analyze the NGINX metrics against the connected server. In the screenshot below, you can see the correlation between the number of NGINX active connections and the app server threads. As the traffic increased you can see the NGINX connection increase as well with the server responding with more threads serving more web requests. This causes the number of active connections to decrease.

Screen Shot 2015-01-28 at 4.11.16 PM

You can also set-up health rules triggering actions such as sending emails, executing remedial scripts, etc. after crossing a certain threshold.

Screen Shot 2015-01-28 at 4.11.23 PM

With AppDynamics, you can create custom dashboards for specific KPIs and metrics.

Screen Shot 2015-01-28 at 4.11.30 PM

Monitoring NGINX with AppDynamics is more than just stats and graphs. With automatic transaction tracing, you’ll see your web application in business context, while the self-learning baselines alert you with what’s normal – and abnormal – without configuring a thing.

If you’d like to learn more, check out our webinar with NGINX!

AppDynamics Now Offers More than 100 Extensions

At AppDynamics, it’s been our mission to equip enterprises with true Application Intelligence. As the expanded application ecosystem grows, many enterprises are using a combination of legacy, contemporary, and cutting-edge infrastructure components. So today we’re proud to announce we now integrate with over 100 extensions so users can leverage performance insights like never before. We strive to create a seamless extension ecosystem to merge with your current toolset.

The Application Intelligence Platform is open, extensible, and interoperable to fit any business need. Through our extension ecosystem, AppDynamics Exchange, our community comes together to share knowledge and contribute back extensions that provide deep integrations to the tools modern enterprises frequently use.
Screen Shot 2014-11-03 at 10.50.36 AM
A few extension categories I’d like to highlight are:

Monitoring extensions – integration metrics from other monitoring systems and services such as Apache, RabbitMQ, CouchBase, IBM DataPower, and Apica.

Alerting extensions – allow creation of custom notifications to integration AppDynamics’ health rule violations and events with alerting or ticketing systems such as PagerDuty, VictorOps, ServiceNow, and Atlassian JIRA.

Cloud auto-scaling extensions – integration cloud orchestration functionality such as creating, destroying, restarting, configuring, and validating machine and image instances. These include Amazon Web Services (AWS) and Microsoft Azure.

REST API & SDKs – enable retrieval of data from the AppDynamics system, including application and infrastructure metadata, metrics, events, and transaction snapshots. AppDynamics developer SDKs are available on Github. Through our API, you can download our Java SDK and the Python SDK.

With these integrations, users can monitor metrics within a business context, rapidly remediate issues, and automate baselining to detect glitches. Extensions also leverage the core Application Intelligence Platform including APM, mobile, and analytics.

Using AppDynamics and our slew of extensions, companies can help foster a DevOps culture and enable IT Ops, developers, and business professionals understand their application data, troubleshoot performance issues, and make strategic business decisions.

Come check out our community of extensions here, and download a FREE trial of AppDynamics today!

Black Friday Horror Story Averted with Alerting and Monitoring

AppDynamics recently announced the launch of our Application Intelligence Platform, which is the underlying infrastructure that delivers our portfolio of products to customers. A key component of the Application Intelligence platform is the notion of extensibility – we can integrate with many of the existing tools you already have in place so you can leverage AppDynamics analytics in the tools and dashboards your team knows and loves with minimal effort. These extensions as we call them are available on the AppDynamics eXchange section of our Community for download, and customers even have the option to submit extensions they’ve written themselves to be included in the eXchange.

To illustrate the power of the 75+ extensions we’ve published in our community, I’ll walk you through two scenarios that involve several common technologies that are prevalent across our customer base.

____

Before AppDynamics:

Jerry has been tossing and turning all night long. In fact, he’s had difficulty sleeping the past three weeks. His sub-optimal sleep patterns are in large part a result of the production application environment he is responsible for. “Things always seem to break in the middle of the night,” Jerry complained to his wife earlier that day.

As the DevOps lead for their company’s mission critical ecom app, Jerry is copied on most urgent application related alerts so that he can help manually forward the details he gets from his current monitoring tools to the admin from his team who happens to be on call at the time. Tonight, he only received 5 such notifications which is less than normal, but still sufficient to wake him up throughout the night. As he squints in the darkness and his eyes adjust to the bright screen, he sees a new notification that troubles him… “SSL Certificate Expired?” he mumbles to himself. “How is that possible?”

He checks the clock – 5:30AM. The person who handles the SSL Certificate isn’t going to be awake for a few more hours. Jerry’s heart drops because he knows that for every hour his ecommerce application is down it costs his company about $10,000 of revenue. “Why wasn’t this on my radar?” Jerry says. “We could’ve planned for this.”

Jerry gets to work early and starts sending emails and calling stakeholders to schedule an 8:30AM conference call. By 9:15AM the action items and deliverables are clear. By 10:30AM the SSL Certificate is renewed and the ecom store is back online servicing customers. Whew. “That could’ve been a lot worse than just 5 hours of downtime and $50K of revenue impact,” Jerry reasons with a colleague.

Back at his desk, Jerry looks at his calendar, his next meeting is ‘testing & capacity planning’ which is a weekly recurring meeting with him and his team.

Jerry’s company is preparing for the holiday season (Black Friday, Cyber Monday, etc.) which is still a few months away but for ecommerce stores, these peak seasons are huge operational and business challenges. You know that $10K per hour of revenue metric?  During those peak days in the holiday season that quadruples to $40K of revenue per hour. The ecommerce store can’t have any hiccups during that time or the impact would be massive, and that’s why this particular recurring meeting leading up to the code freeze are very important.

Jerry greets his team and looks over the shoulder of one of his sys admins. She’s just got the application infrastructure diagram drawn on the white board and has the first load test done and now they are analyzing the results. Looks like most of the synthetic tests they’ve run completed with relatively few errors and utilization was within the acceptable range even as the load increased over the duration of the load test. So far so good.

Jerry moves on to peek over the shoulder of his DBA who is currently analyzing the Cassandra cluster metrics after the load test. Disk I/O looked good and memory looked OK. Over the course of the next hour Jerry’s team tests 6 different load testing and failover scenarios. Today’s tests are done – until next week.

“Everything looks good… a little too good,” Jerry says to himself. “My team and I understand things like utilization and throughput but how does that translate to things my boss and the rest of the business care about?”

If only there was another approach to monitoring that would save Jerry from the fire drills, cut down on the constant testing and debugging, and give him a real-time view into how customers were engaging with his ecommerce application…

Luckily for Jerry, AppDynamics does just that…  Let’s look at this same situation one year later.

After AppDynamics:

Jerry wakes up from a great night’s sleep and checks his email for the daily AppDynamics events digest that gets sent to him with all of the application events over the last 24 hours. Only one event in the digest. Ever since Jerry’s organization invested into AppDynamics’ products that are delivered on the Application Intelligence Platform, his dev team has gotten code-level visibility into the root cause of performance issues inside his ecommerce application and has substantially cut down the number of bugs in the software. That means less production issues for his team to deal with downstream.

Using the PagerDuty alerting extension, the one issue that was sent in Jerry’s digest triggered the creation of a help ticket and was automatically assigned to the technician on duty with no manual intervention on Jerry’s part.

By the time Jerry checked on the status, the issue was already resolved. Nice.

On his way to work, Jerry smiles and thinks about last year’s SSL Certificate debacle. Since installing the SSL Certificate Monitoring extension from AppDynamics, his team has been able to build a dashboard that shows the number of days left until the SSL Certificate expires. No more SSL Certs expiring without anyone knowing ahead of time.

Jerry arrives at work and goes to his recurring ‘testing and capacity planning’ meeting that his team sets up every year around this time. Since deploying AppDynamics and installing two additional AppDynamics extensions – the Cassandra monitoring extension and the Amazon Web Services (AWS) cloud connector extension – his testing and capacity planning work for the holiday season has gotten a lot easier.

First, AppDynamics has given him and his team a great topology view that has relieved them of their needs for Visio diagrams and whiteboarded architectures. Being able to have a real-time view of how the different components of an application interact with each other, and have that map update automatically as new code is released, was hugely valuable for Jerry’s team.

Screen Shot 2014-06-18 at 10.26.02 AM

Second, during Cassandra testing, in addition to getting basic metrics like disk I/O and memory, the Cassandra extension provides configurable metrics like:

  • Cache size, capacity, hit count, hit rate, request count

  • Total latency, statistics, timeout requests, unavailable requests

  • Bloom filter disk space used, false positives, false ratio

  • SSTables compression ratio, live tables, disk space, compacted row size

  • Row size histogram

  • Column count histogram

  • Memtable columns, data size, switch count

  • Pending tasks

  • Read latency

  • Write latency

  • Pending and completed tasks

  • Compaction tasks pending and completed

  • Timeouts

  • Dropped messages

  • Streams

  • Total disk space used

  • Thread pool tasks: active, completed, blocked, pending

By leveraging these metrics, Jerry’s team is able to get granular visibility into Cassandra performance and see exactly where performance bottlenecks occur. This visibility has cut down the time needed to test their Cassandra implementation drastically. Pinpointing exactly where the performance issues are and what caused them enable Jerry’s team to proactively address Cassandra performance issues before they affect end users.

Finally, while capacity planning, Jerry now leverages the Amazon Web Services (AWS) cloud connector extension which allows his team to easily scale up and scale down in the cloud automatically based on policies that can involve a number of rules including:

•       Overall application health (load, response time, number of slow calls, etc.)

•       Business transaction health (load, response time, number of slow calls, etc.)

•       End User Experience health (pages / iFrames / AJAX requests per minute, first byte time, DOM ready time, etc.)

•       Databases & Remote Services health (calls per minute, errors per minute, etc)

•       Error rates (exceptions, return codes, etc.)

This year, Jerry’s team is putting a few different health rules in place that will automatically scale up the AWS EC2 resources when certain load & response time metrics are breached and scale down when those metrics go back down to a normal level. Jerry has also added an authorization step to these workflows that will alert him and ask for permission before spinning instances up or down. That way, they only pay for the EC2 resources they need to use and Jerry still has full control.

Screen Shot 2014-06-12 at 3.49.26 PMScreen Shot 2014-06-12 at 3.49.51 PM

Screen Shot 2014-06-12 at 3.50.17 PM

Jerry leaves the testing meeting with full confidence that his team has a good grasp on the upcoming peak season and has the visibility in place that will allow his team to quickly deal with any performance issues as they arise.

_____

As you can see, Jerry is in a lot better spot this year than he was 1 year ago. By leveraging AppDynamics he has one platform that can easily connect to the rest of the technologies he already uses and provide him a single UI in which he can manage the performance of his environment.

If you’d like to try AppDynamics for free and test drive some of the extensions we’ve highlighted in this blog post, click here.

AppDynamics now available to IBM customers via IBM Cloud marketplace and integrated with IBM DataPower

We’re excited to announce a collaboration with IBM that brings the power of the AppDynamics Application Intelligence Platform to the IBM Cloud Marketplace. IBM customers looking to further simplify application development and operations for increased innovation and the industry’s lowest overall TCO can opt to include AppDynamics at no additional cost.

As more and more enterprise companies rely on a cloud platform to support new and innovative applications to keep pace with rapidly changing business requirements, pairing the Application Intelligence Platform with the IBM Cloud marketplace simply makes sense. Together these technologies provide the nimbleness of the Cloud, the stable, reliable service expected from IBM, and AppDynamics’ unparalleled visibility and control for managing and improving application performance while simultaneously extracting intelligence from ‘in production’ apps to both inform line of business executives and help drive better business decisions.

AppDynamics has also partnered with IBM to develop support for IBM DataPower, IBM’s popular purpose-built messaging appliance used within highly secure enterprises, including leading banks around the world.

The AppDynamics Monitoring Extension for DataPower provides IBM DataPower customers with end-to-end visibility for the best possible control over applications with complex security and reliability requirements. All the metrics gathered from DataPower into AppDynamics by this extension are automatically and dynamically baselined and alerts can be set up on AppDynamics to:

  • notify administrators, operators or other ticketing applications if any metrics breach undesirable thresholds in real-time

  • automatically remediate problems based on health-rule violations

Unlike other application monitoring providers, AppDynamics allows DataPower users to mix and match which portions of the application intelligence platform they’d like to deploy to best meet individual needs. This is not an all-or-nothing offering with pay-for-support fine lines.

With the AppDynamics Monitoring Extension for DataPower, customers can now automatically correlate their dashboards and snapshots to their business critical applications:

Application dashboard with DataPower as a custom exit point

Fully correlated snapshots

Custom dashboard

Of course, AppDynamics has had long-standing support for IBM products like WebSphere application server, WebSphereMQ and IBM DB2, with customers also monitoring IBM commerce platforms like IBM Yantra and IBM Sterling Commerce. The addition of AppDynamics as an IBM Cloud Marketplace offering together with the new support for IBM DataPower are just two more examples of the collaboration with IBM for the benefit of our mutual enterprise customers.

We look forward to continuing to work alongside IBM for their customers as they create, manage and optimize their business critical applications.

Introducing AppDynamics X and AppSphere

Screen Shot 2013-06-28 at 5.27.40 PM

The AppDynamics team is excited to announce the official launch of AppDynamics X, a new exchange for plugins, integrations and extensions for AppDynamics Pro, and AppSphere, an AppDynamics community portal for application management professionals to maximize their deployment of AppDynamics Pro.

AppDynamics X
AppDynamics is a flexible and scalable platform for application management data. That means that there’s a lot you can do with your data in addition to what we provide in our Pro product. With the AppDynamics APIs you can build your own extension of AppDynamics Pro to collect data from any component of your infrastructure (databases, caches, queues, hardware, etc.), integrate with other tools, or even auto-scale your application in your favorite cloud. In addition, you can use the AppDynamics APIs to build custom alerts, remediation mechanisms and visualizations for your application.

But why should you do all that work yourself if someone else has already done it? That’s why we launched AppDynamics X. AppDynamics customers and employees have already built a bunch of extensions and integrations for AppDynamics Pro, and now you can find them all in one place. Don’t see what you’re looking for? Submit a request for us to build it for you, or contribute an extension yourself.

Some of the more popular extensions already on AppDynamics X include integrations for PagerDuty, Splunk, Nagios and Boundary and cloud connectors for Amazon, Rackspace, VMWare and Windows Azure. Check out AppDynamics X to get started.

AppDynamics Pro Integration Points


AppSphere

The AppSphere community enables IT operations and developers to share best practices and introduce new innovations for application performance management.

AppSphere

AppSphere is an open community that serves as a one-stop learning destination for all AppDynamics users. It enables application support teams to trade ideas, share best practices, and explore new trends in application performance management as well as IT operations. Forums exist for AppDynamics Pro customers who wish to discuss insights on how best to leverage the solution, but any IT professional with a vested interest in application performance management can join in and engage with issues pertaining to managing complex applications, Java/.NET/PHP performance tips, DevOps, and more.

I believe this quote from Jyoti Bansal (our CEO) best describes the spirit of the community:

“The AppDynamics customer base is passionate, as evidenced by our Net Promoter Score of 81 and the worldwide demand for our solution. Therefore, we’re happy to further empower both AppDynamics customers as well as application support teams everywhere with our AppSphere community and AppDynamics X platform. We believe that the best ideas and insights are generated through conversation and collaboration, and that’s why we’ve made these communities available to those with a vested stake in the performance of critical applications. We look forward to seeing what new innovations arise from their engagement, participation, and dialogue.” — Jyoti Bansal, Founder & CEO of AppDynamics

Integrations in action

The process monitor extension in action showing cpu and memory utilization:

AppDynamics Process Extensions

The Splunk integration in action:

Splunk Integration

Here is what John Martin of Edmunds.com had to say about our Splunk integration:

“Being able to bridge two indispensable tools such as AppDynamics and Splunk Enterprise is absolutely essential to our day-to-day operations. The ability to go into one of these tools and click my way into one or the other at that exact point in time for an issue that I’m researching is just fantastic.” — John Martin, Sr. Director of Production Engineering, Edmunds.com.

Contribute or suggest your own extensions on the AppSphere.