Today’s software defined and driven business requires fast changes in business models, this permeates successful businesses. What this means is that almost every company is learning how to make small rapid changes and adjustments within their business, especially within software systems. The result of this is that IT is feeling immense pressure to evolve, the net result is a major uptick of private cloud (especially Apprenda, Pivotal Cloud Foundry, RedHat OpenShift) and public clouds (especially Amazon and Microsoft) within our install base of large enterprises. These new platforms enable faster development, testing, and releasing of software. Enterprise customers are getting much smarter, by using automation to drive the software lifecycle including continuous integration, and even experimenting and beginning to use continuous release processes. This is no longer a startup scenario, it’s a strategy for our large enterprise customers. These companies trying to build agile development and operations teams, by changing people, process, and technology to enable a DevOps feedback loop:
Wikipedia describes the continuous delivery loop as follows:
Within this loop, APM can be part of both build, unit, regression, and load tests to drive the automated acceptance testing before new code is pushed to production. In fact, at the recent AppSphere 2015 user conference we had several great talks on this subject.
The first talk was put on by our customer The Container Store in this presentation August Azzarello digs into “How The Container Store uses AppDynamics in the Development Lifecycle”. in their environment they began first with APM in production. The goal was to improve software quality before production, hence they began the expansion by installing APM within test, and integrating it into functional and performance test suites. They also enabled alerting from dev/test so that developers and testers understood when they had performance deviations and issues before production. Key features they leveraged in pre-production included dashboards such as this one:
Comparative analysis view such as this one:
and being able to understand between releases if they were improving or degrading:
The Container Store is using open source tools including Selenium for functional testing, and Locust.io for performance testing. In the video August also explains some of the major benefits they saw from implementing APM in pre-production. Here are some best practices outlined by August in his discussion:
-
Monitor everything
-
Test continuously
-
Performance test early in development life cycle
-
Empower development & QA team members
Some benefits the container store has seen according to August are:
-
Set performance expectations before production deployments – ~40% improvement since we started using AppDynamics in test
-
Fine tune alert and Business Transactions policies prior to production deployments
-
Identify testing requirements, and testing gaps
-
Decrease performance test result reporting from 5 hours to 20 minutes
One of my favorite talks (which is why I selected it for the deep dive track at AppSphere) was the talk given by one of our Senior Sales Engineers Steve Sturtevant he came to AppDynamics from PayPal 2 years ago. In his talk titled “DevOps and Agile: AppDynamics in Continuous Integration Environments” Steve explains the integration work he did, but more importantly in dynamic environments can you scale and auto implement monitoring? Do you quantify the impact of change? Steve gives a good demo with integration between Puppet, Jenkins, ServiceNow, and AppDynamics in his examples with automation throughout the lifecycle.
One of our partners Electric Cloud specializes in continuous delivery, and embraces an open ecosystem of technology providers and open source software. They’ve put together some great resources for people looking to explore the rest of the feedback loop.
-
Deployment automation – http://electric-cloud.com/wiki/display/releasemanagement/Deployment+Automation
-
Jenkins integration and release tagging – http://electric-cloud.com/wiki/display/releasemanagement/Deployment+via+Continuous+Integration+Tools
-
Integration within existing [legacy] processes and frameworks – http://electric-cloud.com/wiki/display/releasemanagement/ITIL+Release+Related+Processes
-
Defining performance KPIs / correlating to business performance metrics – http://electric-cloud.com/wiki/display/releasemanagement/Release+Management+Metrics
