How I Discovered Application Performance Management and a World-Class Career Opportunity

That’s the question I found myself asking when I got a message from a recruiter on LinkedIn just over a year ago. Up until that point, I had very little exposure to “application performance management”, having spent my ten-plus years in IT working on layers 1 through 4 of the OSI stack. First, I had been a software engineer in the aerospace/defense industry, writing embedded code and drivers in C for real-time systems. From there I moved into financial services, where I sold, supported, and implemented low latency messaging architecture for high frequency stock trading systems. That was a fantastic job — I worked with incredibly smart people on the most challenging products I have ever known. I spent nearly seven years in that role, but as much as I enjoyed it, I felt like I needed a new challenge. So when I got that LinkedIn message in the spring of 2017, asking if I was interested in a new SE role, I wanted the answers to two very important questions: “What is APM?”, and, most importantly, “What is AppDynamics?”

Taking AppDynamics for a Test Drive

Having spent most of my career working with the C programming language at the hardware level, it’s no wonder I never came across an APM tool. Don’t get me wrong, I had foundational working knowledge of Java and .NET from my messaging role, but only just enough to be dangerous, and far from being an expert. I had actually spent a bit of time with PHP and MySQL for some pet projects on the side, so I wasn’t totally clueless to the technology in the space. When the time came to download the AppDynamics trial and take it out for a spin, I was honestly blown away. At that moment, I knew immediately I wanted to be a part of the company.

Not being a Java expert, I really wouldn’t have known where to look to debug a Java application outside of an application log. But with AppDynamics, I could see the Java application on the flow map, and all the calls being made to the Java application via Business Transactions. I could even dive into the code and see how long each line of code was taking, as well as the SQL statements being queried from the application. Finally, my personal favorite — when a Java application talks to another Java application via a web service call, AppDynamics will follow the transaction all the way through, allowing you to jump from the call stack of one app to the other in a single click.

Serving as an SE for AppDynamics has been an incredible experience. It’s the kind of role where you get back everything you put in. I spent a lot my time early on self-learning .NET, Java, and some containerization services like Docker and Kubernetes. I also took some time to get to know Azure and all its services. All this learning paid off immensely. Now, I can walk confidently into a room and talk to the senior leadership teams of Fortune 500 companies about their technology choices, the cloud, and about containerization/microservices. Best of all, I got to this point in less than 12 months. As an engineer, I love to solve problems — tough ones. And there’s no better platform for tackling tough problems than AppDynamics.

Cultivating an Entrepreneurial Spirit While Demonstrating Value

In addition to the technology behind the platform, AppDynamics has inspired my entrepreneurial spirit. That’s because AppDynamics isn’t just about showcasing what the product can do, but also, demonstrating the value it provides. While we’re looking for good technology fits, we’re also looking for good value fits. To do this, we work with our teammates and customers on building out business cases explicitly showing our customers and potential senior and executive management teams the ROI with AppDynamics. This part of my work has challenged me to learn more about a wide variety of business cases and uncover the answers to questions that technology executives regularly have during the consideration process. In many cases, doing this has led me to some of my favorite moments in my AppDynamics career thus far.

One of the best parts of the job is giving product demonstrations. Sometimes we demo to people who are familiar with APM and maybe even familiar with AppDynamics. However, the best demos are for people who aren’t familiar with AppD — kind of like myself a little over a year ago. This past spring, while I was doing a demo for a good-sized audience — maybe about 12 people or so — there was a VP of operations in the room, two directors, and some senior technical staff. I had started the demo and prodded the room for questions as the minutes passed by, and it was quiet. No comments, no questions — just silence.

If you’ve ever found yourself in a similar scenario during a demo, you know this is just about the worst thing that can happen. When it’s just you talking to a room full of people, time passes more slowly, and inside, you’re just dying for someone else to say something.

After getting about halfway through the demo of our Map iQ functionality, a director in the crowd finally interrupted and asked, “Is this real?” I hesitated for a moment to respond — not for very long, but long enough to think to myself, “Is this a real question?”

“Yes, this is very real,” I responded.

Then, the gentleman turned to his VP and said, “We need this, like, right now,” with a lot of enthusiasm. This opened a floodgate of questions from those in the room, and the use case potential became abundantly clear to the crowd. The rest of that demo was amazing. And the truth is, while he was the only person in my entire career to ask, “Is this real?”, he hasn’t been the only person I’ve spoken with that has been blown away by AppDynamics and its capabilities.

Holy moly, one year already?!

I can’t believe it’s already been one year into my journey with AppDynamics. This has been one of the best transitions in my career of more than 25 years, and I feel like it just started yesterday. For me, it all began back in January 2017, when Cisco announced their intent to acquire AppDynamics. Although the deal closed a few months later, I had already been down the path of exploring my opportunities and educating myself in the exciting world of application performance monitoring (APM) as a means to bridge my career to an increasingly app-centric world.

I was lucky enough to come across a regional sales team in the Carolinas who quickly got me in touch with a sales leader at AppDynamics. They were looking for people with experience in cloud and service providers to be part of an exciting new team. With my deep technical and customer-facing expertise in various related areas, I was keen on rolling up my sleeves to dabble in the world of APM.

As I found out, AppDynamics has a very comprehensive approach to interviews and hiring. I was tested not only on my professional experience, ability to learn, and deep technical expertise (both theory and hands-on skills), but also on my ability to handle high-pressure situations in front of customers. I was fortunate enough to pass through these gates and accepted an offer to transition from Cisco to AppDynamics. It wasn’t always a walk in the park, but I must say that at each point throughout the process I felt entirely welcomed and supported by the interview teams.

Once I joined, I realized for the first time how detailed and methodical this place is. In the first few days, I had a one-on-one call with an onboarding coordinator to help me set goals to hit for my first 30, 60, and 90 days in terms of learning, using tools like MindTickle. The IT team was always very responsive to my needs, just a Slack message away. Then there was the boot camp I needed to schedule, part of the process for my first 30 days.

Having been given some prior information via the sales engineer buddy program, I went into the boot camp with a combination of apprehension and a knot in my stomach, thinking this was going to be impossible for me to handle. The program, although very comprehensive and demanding with a lot of work to be done, was made easier by the onboarding team, who were there to welcome us and to make us feel at home from day one. Everyone was extremely knowledgeable, including the invited guests. The activities got us all involved right away, making the week go by fast. We did roleplaying, team and individual exercises, and even had time for a group outing — it was fun!

The best part of being at AppD by then was that I knew a lot of people and co-workers who I could reach out to, not only in person, but also online through a variety of means. My colleagues from across the globe were always on a Slack channel ready to help me resolve a tricky situation or question.

The online, self-service modules for my 60 and 90-day marks — plus my certification process — involved a lot of hands-on work, but it was quite enjoyable and I learned a lot. As the year progressed, I had the opportunity to meet and work with many individuals and leaders who had tons of experience in the world of APM and its related technologies. They were always eager to help a fellow co-worker. The local, regional, and company-wide events, both online as well as in-person, offered plenty of opportunities to learn and exchange ideas.

By the time I reached six months with the company, we were well on our way as a team to be a cohesive group that I feel privileged to be part of. Our team dynamics and ability to reach out to other regional groups as we support our global SP partners make me proud to be part of this great phenomenon. We’re helping our partners build world-class solutions using the best people and products. I’m thrilled to be part of this team!

As I write this, I’ve come to realize what a wonderful ride this past year has been. I’ve learned a lot, contributed a lot, and feel that being part of this exciting organization has given me the necessary boost for moving into the exciting world of applications, where things are moving fast and in a big way. By furthering my skills, I feel ready to solve even bigger challenges for the best possible business outcomes. Best of all, I love the fact that this is happening within Cisco, a place I have called home for a huge portion of my career.

Puppet, an Oldie but Goodie: A DevOps Eponym

Calling a software company that was founded in 2005 an “oldie but goodie” might sound a little ludicrous. After all, AppDynamics, founded in 2008, was not far behind. But technology advances at a dizzying pace, and it’s not unusual for “older” tech of a decade or so to feel antiquated, even quaint.

So does Puppet, one of the earliest infrastructure configuration management tools, still matter in 2018? The answer is a resounding “yes.” Puppet remains highly popular, and you don’t need an extensive programming background to use it. Its module/model approach doesn’t require significant expertise in the system that Puppet is automating.

When referring to infrastructure-as-code, I see Puppet as an eponym. “Puppet Manifest” for automation is like “Kleenex” for a tissue or “Coke” for a soda—a.k.a. an eponym in the DevOps community.

Puppet, the DevOps Master

Some of the best DevOps-centric studies are Puppet’s annual State of DevOps surveys, which chart DevOps trends and practices across the globe. When you compare these reports from the past few years, the phenomenal rise of the DevOps movement becomes clear. For instance, the 2017 study found the gap between high-performing teams and “low performers” is shrinking fast:

Between 2016 to 2017, the gap for frequency of code deployments narrowed: High performers are still shipping code as the business demands, while low performers went from shipping between once per month and once every six months in 2016, to shipping between once per week and once per month in 2017. Low performers in 2017 have also reduced their lead time for changes: from between one month and six months in 2016 to between one week and one month. This change does not mean that high performers are no longer performing as well. It simply means that low performers are doing better with throughput than they were, on average, and we applaud them for this improvement.

Tech moves fast. And the infrastructure-as-code movement is a key pillar of DevOps because of its ability to be consistent and repeatable across environments.

Puppet Today

In addition to maintaining a healthy ecosystem of products, Puppet backs several open source projects and recently received additional funding from a Cisco-led investment group. The Puppet platform on top of IaC automation has the ability to chart topology, manage tasks, be a container registry, and automate application deployments. Puppet also maintains Puppet Forge, a repository/gallery for modules.

Puppet Primer

There are many great tutorials and articles on how to get your first Puppet deployment up and running. If you’re new to Puppet, here are a few terms and links to make you dangerous.

Architecture

Puppet has a master-agent architecture. A centrally located Puppet Master will manage the Puppet Agents.

Puppet Enterprise brings additional features and support for security, orchestration and node management.

First Puppet Tasks

Once you’ve installed Puppet, check out this fantastic example from Digital Ocean on how to configure a LAMP stack on Ubuntu. (TL;DR—you can create manifests and/or modules to complete these tasks.) The Digital Ocean tutorial provides some Puppet code basics and strategy (e.g. multi-manifest/module).

Puppet and AppD Together: The Art of the Possible

Puppet is a great platform for automating the deployment of AppDynamics Agents. As enterprise topologies become ever more complex, the metrics that AppD provides can help you administer an infrastructure deployment/reprovisioning orchestrated by Puppet.

In addition to providing a wealth of information on the health of an application, AppDynamics can validate that SLAs/SLOs are being met.

  1. Blue-Green Deployment Validation

    1. My AppD colleague and fellow tech evangelist, Chase Aucion, conjured up this concept as the art of the possible—aiming for pragmatic solutions, not perfectionism. Discounting other platforms such as a PaaS, if you have to roll your own blue-green deployment, readying an identical production environment for the idle state (green) may take forever without infrastructure automation. The old paradigm—doubling your production infrastructure for a deployment/release, only to tear down one side once testing is complete—isn’t the answer. So how do you make sure the new release is meeting your SLAs/SLOs? With infrastructure laid down by Puppet and validated by AppDynamics, you’ll achieve the art of the possible.

  2. Elastic Scale

    1. Scale goes in both directions: Scaling up when load increases, and scaling down/reprovisioning when load doesn’t warrant the capacity. By delegating provisioning application infrastructure to the Puppet stack—with metrics provided by AppD—you can decide if additional infrastructure is needed to support SLAs/SLOs, or if certain infrastructure can be reprovisioned for other use.

  3. Happy Puppet Master

    1. As systems become critical in our pipeline, we must ensure they are performant. Having the Puppet Master (server) becoming a bottleneck would be problematic. The Puppet Master is written in C++, Clozure and Ruby. AppDynamics has the ability to monitor C++-based applications. It can validate the usage of number of Puppet Masters, as well as nodes in a high availability-based deployment.

Learn more about how AppDynamics can help your own organization on its DevOps journey.

The Pain of Not Doing DevOps

For planning new software projects and maintaining existing deployments, DevOps helps organizations creates stronger ties between all stakeholders throughout the development lifecycle. So it’s no surprise that DevOps is rapidly gaining popularity around the world. In the most recent Global Development Survey from Evans Data, 72% of developers reported working in a formal DevOps environment, ranging from specific development operations interactions to enterprise-wide implementation. DevOps practices may vary by organization, of course (for a quick overview, check out our AppDynamics primer).

In my experience, organizations that fail to embrace DevOps do so at considerable risk. Not long ago, a major real estate developer reached out to AppDynamics for help. The company’s .NET installation was having problems with a third-party web asset management library, which had specific write-to-disk configuration requirements.

These requirements were configured properly in the development environment, but not in production. And because because Dev was siloed off from Prod—with no process for keeping these environments in sync—the company was unaware of the oversight.

This end result: ongoing performance problems in production. And because IIS auto-restart settings recycle the application pool after a time interval, the company was unable to identify the root cause of ongoing application instance crashes.

Finding the Cause

The company contacted AppDynamics, asking if we could figure out why the application kept crashing. We instrumented their system, which hadn’t run APM software before. Upon reviewing the AppDynamics system logs, we immediately found the source of the problem: the application didn’t have permission to write to disk in the production environment.

This discovery spurred the company to undertake a lengthy internal testing and environment comparison process, which uncovered numerous configuration differences between Dev and Prod.

Lessons Learned

On a foundational level, a dysfunctional culture was largely responsible for the company’s production woes.

For instance, there was a lot of friction—including major arguments at times—between various stakeholders, including management, operations and development. This was a classic case of “code being thrown over the wall” from Dev to Prod. Communication between these factions was so poor, in fact, that a contractor was the primary liaison between Dev, Ops and management.

In addition, there was a loss of tribal knowledge whenever a technical practitioner left the company. When AppD arrived on the scene, none of the Devs knew anything about the troublesome third-party tool, nor how it was being used. Even the aforementioned contractor—the sole link between the siloed factions—was unaware of the problematic utility and the critical role it played.

Avoiding Pain through DevOps

Communication and knowledge-transfer between teams is critical. Each environment—Dev, Prod, and so on—should be as similar as possible for continuous integration and continuous deployment (CI/CD). Major configuration differences between environments often leads to negative outcomes.

A key tenet of DevOps is to standardize your environments. If you build code on one Dev’s machine, it should run everywhere the exact same way.

The software industry is moving in this direction. “Today, most developers state that some portion of their development team includes operations engineers.” the Evans Data study found. “The movement toward DevOps has not only impacted stakeholders’ roles, it has also influenced development team composition.”

Implementing DevOps practices, combined with a solid APM solution, will enable your organization to avoid pain and gain deeper insights into the performance of your code.

Learn more about how AppDynamics can help your own organization on its DevOps journey.

Sean Stanford is part of AppDynamics Global Services team, which is dedicated to helping enterprises realize the value of business and application performance monitoring. AppDynamics’ Global Services’ consultants, architects, and project managers are experts in unlocking the cross-stack intelligence needed to improve business outcomes and increase organizational efficiency.

Why Metrics Must Guide Your DevOps Initiative

Metrics-oriented thinking is key to continuous improvement – and a core tenant of any agile or DevOps philosophy. Metrics are factual and once agreed upon, these facts are used to drive discussions and methods. They also allow for a collaborative effort to execute decisions that contribute towards business outcomes.

DevOps, although becoming a commonly used job title, is not a role or person and there is no playbook or rule set to follow. Instead, DevOps is a philosophy which spans people, process, and technology. The goal is releasing better software more rapidly, and keeping said software up and running by joining development and operational responsibilities together. Additionally, DevOps aims to improve business outcomes, but there are challenges in selecting the right metrics and collecting the metric data. Continuous improvement requires continuous change, measurement, and iteration. What’s more, the agreed-upon metrics drive this cycle, but also create insights for the broader organization.

 

Data-Driven DevOps

A successful DevOps transformation focuses on a couple areas. To start, a culture change is needed between development and operations teams. Another core tenant of DevOps is measurement. In order to accomplish a true DevOps transformation, it’s important to measure the current situation and regularly review metrics which indicate improvement or degradation. One of the core tenants of DevOps is measurement, and using said measurements as facts when driving decision making. These metrics should span several areas which may have been considered disjointed in the past.

To help DevOps teams think of possible metrics and how these metrics relate to key initiatives, Gartner recently released this useful metrics pyramid for DevOps:

Many of these metrics span development, operations, and most importantly – the business. They measure efficiency, quality, and velocity. However, Gartner points out that the hardest part is often defining what we can collect, take action upon, audit, and use to drive a lifecycle. The second challenge (which Gartner does not discuss), is how these metrics should be linked together to offer meaningful insights. If the metrics do not allow linkage between a release and business performance, attribution gaps remain. And unfortunately, many enterprises today analyze metrics that have a lack of linkage or relationship between them.

To help with these relationships, context is critical. Without context, metrics can be open to interpretation, especially as you move up the Gartner pyramid. And this is precisely where AppDynamics can step in and help.

Based on our measurement technology, AppDynamics can link metrics together to allow you to attribute earnings or cash flow with a release or change that represents an incremental improvement in the application. We represent these within Business iQ, our event-based analytics system for the highly valuable data we extract.

Additionally, AppDynamics provides metrics to drive visibility inside the application without creating an additional burden for developers. In fact, with automated instrumentation as part of AppDynamics APM, metric data is produced consistently and comprehensively across all teams. This is extremely beneficial as many teams have different ways of collecting data, which can traditionally lead to inconsistencies. But with AppDynamics, consistent measurements are always obtained from the application components and desired business outcomes of the application.

Learn more about how Business iQ can accelerate your data-driven DevOps initiative.

Moving the DevOps Needle in Enterprise Organizations

DevOps has proven itself across many smaller organizations but large enterprises are usually slow to change. It can be a daunting task even identifying where to make changes since there are so many processes and organizational silos to get in the way. As a veteran employee of small, medium, and large enterprises I have figured out ways to drive organizational change based upon getting results. In this presentation I will describe my methods for creating change within and across organizations and provide specific examples of how to begin a meaningful shift towards making DevOps a standard practice within your organization. I’ll detail some of the roadblocks to making DevOps a reality and explain how to overcome these obstacles.


[slideshare id=26955993&doc=devopsdays-movingthedevopsneedle-final-131007174831-phpapp01]

AppJam 2012: Accelerating DevOps Culture at Edmunds.com

John Martin, Senior Director, Production Engineering, Edmunds.com
Collaboration between Dev and Ops is a critical part of how Edmunds.com manages its 30 critical web application. Development requires hard data from Operations so they can fully understand how to improve application performance. In this session John will discuss the key drivers for DevOps at Edmunds.com,  how Dev and Ops collaborate using AppDynamics to solve production incidents, and his top 5 tips for implementing a DevOps culture in your organization.

Slides:

[slideshare id=15678365&doc=edmundsfinal-121217201143-phpapp01]