Introducing Pivotal Cloud Foundry

You may be wondering why we’ve chosen to write an introductory blog on Pivotal Cloud Foundry (PCF). The platform, after all, has been around for a good while. The answer is simple: many of our largest customers have chosen to use Cloud Foundry, particularly PCF, as their strategic platform for multi-cloud deployment. With so many organizations already using AppDynamics to monitor their critical business applications running on Cloud Foundry, we thought it was high time we took a deeper look at the integration between AppDynamics and CF, and at some of the new initiatives we’re collaborating on. We’ll get into details in the next blogs in this series. But for now, if you’re new to Cloud Foundry, read on.

It’s hard to overstate the importance of cloud computing in the evolution of IT. Cloud platforms enable an organization to quickly deploy network apps and services that scale easily. Migration and build-out tasks that once took weeks, or even months, now take just a few minutes. And rather than focusing on the underlying infrastructure, PaaS frees you to focus on your applications and data.

Cloud Foundry (CF) is an open source, ready-to-use platform that allows your company to get up and running quickly in the cloud. Originally developed in-house at VMware, CF made its public debut in April 2011. A year later came BOSH—an open source tool for deploying Cloud Foundry PaaS—that unifies release engineering, deployment and life-cycle management of large-scale distributed services.

Why Cloud Foundry?

Enterprises today want the flexibility, ease of deployment, consistent management philosophy and framework of PaaS, but often need these capabilities in an on-premises environment. With Cloud Foundry, your organization can run applications on its own computing infrastructure, or deploy them on an infrastructure-as-a-service platform (IaaS) such as Amazon Web Services, Microsoft Azure, vSphere or OpenStack. Alternatively, you could use a PaaS deployed by a commercial CF provider.

There are myriad reasons why companies need to run specific apps within their data centers. A recent IDG/Sapho study shows that the vast majority of enterprise respondents (78%) plan to upgrade their applications in 2018, an effort driven largely by an inability to move cloud-based apps for reasons of security, compliance, and other factors (e.g., apps that are too ingrained in daily processes). A large number of AppDynamics’ customers, for instance, have committed to PaaS, which often involves moving to an internal cloud.

Today, CF is available from either the Cloud Foundry Foundation as open source software, or from multiple commercial providers as a product or service. It’s a good choice for enterprises that want to reduce the cost and complexity of configuring infrastructure for their apps. Developers can use existing tools— with zero modification to their code—to deploy apps to CF.

Highly Opinionated

Cloud Foundry is a prime example of an opinionated platform, one designed to operate consistently across environments, with every feature working as intended “out of the box,” notes O’Reilly. For example, CF provides the same user experience when deployed over different IaaS layers, and the same developer experience regardless of application language.

Many multinational corporations, including Ford and Gap Inc., have used Cloud Foundry to launch cloud projects and modernize their internal software development processes, reports GeekWire. AppDynamics has numerous Fortune 500 customers in major verticals—including banking, financial services, telecom and technology—that have been using Cloud Foundry successfully for years.

Enter Pivotal

Pivotal, a tech company backed by notable investors such as EMC, General Electric and VMware, launched a commercial version of Cloud Foundry in 2013. In fact, several large AppDynamics customers were the driving force behind the Pivotal project. Pivotal Cloud Foundry (PCF) provides extra functionality not included in the open software version, such as additional tools for installation and administration, enterprise services, support, documentation and certificates.

PCF is deployable on-premises and on many cloud providers, offering enterprises a hybrid and multi-cloud platform. It has many high-profile corporate users, including Allstate’s CompoZed Labs unit, which uses PCF to speed development of its new software services, and to build applications that run both on its internal servers and on third-party public cloud providers.

AppDynamics has long been an active supporter of the Cloud Foundry platform—both Pivotal and open source—and will continue to do so. Many of our largest customers have been successfully running large-scale CF deployments for years. In our next PCF blog, we’ll talk about challenges facing PCF users, and take a closer look at our expanding efforts in the Cloud Foundry ecosystem.

Getting Started with Pivotal Cloud Foundry and AppDynamics

This post originally appeared on Pivotal’s blog

The primary objective of a platform should be to provide a high-level of automation. This provides easy management of applications and services, while delivering consistent and error-free deployment of applications. While this high level automation provides a critical foundation, additional specialized services can be added to increase manageability of the applications deployed on the platform. To assist operators in this pursuit the Pivotal Cloud Foundry platform provides a number of integrated services out-of-the-box, including AppDynamics, New Relic, and CloudBees Jenkins. This blog will focus on the “out-of-the-box” integration between Pivotal Cloud Foundry and AppDynamics.

Pivotal Cloud Foundry Buildpacks and Agent Integration

A buildpack in Pivotal Cloud Foundry provides the framework and runtime support for your application. As applications are deployed or “pushed”, the platform detects the language and includes the appropriate versions of the runtime, as well as any additional application infrastructure. The buildpack also contains the AppDynamics agent, with numerous versions to support different deployments. To successfully deploy the agent we need to provide the proper configuration based on the environment during the deployment.

To create the proper configuration a User Provided Service (UPS) is created. A UPS is an external service created for an organization and space and is bound to an application in Pivotal Cloud Foundry. Once the service is bound to the application and the application is restaged, a process to prepare an application for deployment, the AppDynamics agent has the proper configuration to register with the AppDynamics controller and to transmit metrics to the designated endpoint.

A common practice in AppDynamics is to create a logical model representing different layers of an application. AppDynamics uses the application name, tier name and node name to align the deployed applications with the AppDynamics logical model. The Pivotal Cloud Foundry platform, specifically the buildpacks and UPS, provide the mechanism to align deployed applications to the logical model.

The diagram below depicts the Spring Trader application deployed on Pivotal Cloud Foundry. Spring Trader includes three Spring-based applications that are designed to represent a simple trading platform. The deployment artifacts include a web UI component written in Spring MVC, a RESTful API component that provides portfolio and trade data via a persistent store and RabbitMQ, and an asynchronous service for quote generation. Each of these applications are deployed into their own container, and each have their own UPS to define a tier within AppDynamics. This deployment aligns the applications to the AppDynamics logical model using the application and tier name.

image04

To establish a deeper understanding let’s walk through the process and commands required to create the user-provided service and applications on Pivotal Cloud Foundry.

Configure and Push Applications onto Pivotal Cloud Foundry

Before we can “cf push” the application, we need to create a user-provided service in the organization and space where we deploy our applications. In the example below, we use the Cloud Foundry CLI to create the user-provided service providing the required information in name/value format. Notice the application and tier name are configured, and we have created three different user-provided services to represent different tiers in our application.

cf cups app-dynamics-ST-web -p '{"account-name":"myacct","account-access-key":"key","host-name":"myacct.saas.appdynamics.com ","port":"80","tier-name":"web","application-name":"SpringTrader"}'

cf cups app-dynamics-ST-rest -p '{"account-name":"myacct","account-access-key":"key","host-name":"myacct.saas.appdynamics.com ","port":"80","tier-name":"rest","application-name":"SpringTrader"}'

cf cups app-dynamics-ST-asynch -p '{"account-name":"myacct","account-access-key":"key","host-name":"myacct.saas.appdynamics.com ","port":"80","tier-name":"asynch","application-name":"SpringTrader"}'

With our services defined, let’s deploy the Spring Trader application. Using the command line, issue the following commands:

## Create required managed services for deployment
cf create-service p-rabbitmq standard tradermessaging
cf create-service p-mysql 100mb-dev tradersql

## Deploying Rest component, binding services and restaging application
### Note the -b option to use the latest Java buildpack from github.com/cloudfoundry/java-buildpack
cf push -p dist/spring-nanotrader-services-1.0.1.BUILD-SNAPSHOT.war -b https://github.com/cloudfoundry/java-buildpack.git --no-start stfront
### Binds Services
cf bind-service stfront tradersql
cf bind-service stfront tradermessaging
cf bind-service stfront app-dynamics-ST-rest
### Specify which JVM version to use in the Java Buildpack
cf set-env stfront JBP_CONFIG_OPEN_JDK_JRE '[version: 1.7.0_+]'
### Specify which AppDynamics agent to use in the Java Buildpack
cf set-env stfront JBP_CONFIG_APP_DYNAMICS_AGENT '[version: 4.0.1_+]'
cf restage stfront.

## Deploying Web UI component, binding services and restaging application.
cf push -p dist/spring-nanotrader-web-1.0.1.BUILD-SNAPSHOT.war -b https://github.com/cloudfoundry/java-buildpack.git --no-start stweb
### Bind services
cf bind-service stweb app-dynamics-ST-web
cf set-env stweb JBP_CONFIG_OPEN_JDK_JRE '[version: 1.7.0_+]'
cf restage stweb

## Deploying the asynch service component, binding services and restaging application.
cf push -p dist/spring-nanotrader-asynch-services-1.0.1.BUILD-SNAPSHOT.war -b https://github.com/cloudfoundry/java-buildpack.git --no-start stback
### Bind services
cf bind-service stback stsql
cf bind-service stback tradermessaging
cf bind-service stback app-dynamics-ST-asynch
cf set-env stback JBP_CONFIG_OPEN_JDK_JRE '[version: 1.7.0_+]'
cf restage stback

As the “cf restage” command is executed the output will show which pieces are required to create the application and prepare it for the container. In the example below, you’ll see the AppDynamics agent is added to the container along with OpenJDK and the Tomcat server. If the AppDynamics agent is not included, verify the user-provider service is properly configured and its bound to the application.

image05

Once the commands are complete you should see the applications deployed in Pivotal Cloud Foundry Apps Manager. You should also see the user-provided services, with an application bound to each service. A number of applications can be bound to the user-provided service, but in our example its one per service.

image01

image08

We’ve successfully deployed our applications and bound AppDynamics services to each application instance. Let’s review the AppDynamics dashboard to understand how our application maps to the AppDynamics logical model.

The AppDynamics Model and Dashboards

In the home dashboard below you’ll notice the Spring Trader application. This application is made up of three independent applications in Pivotal Cloud Foundry.

image00

If we click into the Spring Trader application, we’ll see the app servers and you’ll notice how each application instance uses the tier name to represent different layers of the Spring Trader application. In our example we have the web, rest and asynch tiers. If you expand each tier you’ll see the node names, in this case node name is “0” for the first instance of the application. Node names are configurable, but I prefer to use the default node names as they align with the Pivotal Cloud Foundry application instance sequence number. As we scale different tiers to support new workloads, the new application instances will register with the AppDynamics controller, each with a sequential node name.

image02

By supporting the logical model in Pivotal Cloud Foundry, we can use the application dashboard to review transactions across the different applications.

image06

Troubleshooting AppDynamics Agents in Pivotal Cloud Foundry

During my initial installation, I had misconfigured the user-provided service which prevented the agent from registering with the AppDynamics controller. As a result, I needed to troubleshoot the agent configuration and review its logs. The good news is Pivotal Cloud Foundry makes this extremely easy. To access the agent logs, or any other buildpack information, just issue a “cf files stfront” command similar to the image below replacing “stfront” with your application name.

image07

The logs are located on app/.java-buildpack/app_dynamics_agent/ver4.0.1.0/logs and each instance of your application will have its own log. At the top of the agent log you can find AppDynamic JVM arguments used to configure the agent. If you configure application security groups in Pivotal Cloud Foundry you’ll need to modify the policy to allow egress traffic for the controller port. In the example below, you can see all logs available for review and each file can be output to the screen by issuing the “cf files” command with a full path to the file.

“cf files stfront app/.java-buildpack/app_dynamics_agent/ver4.0.1.0/logs/agent.2015_03_31__14_26_55_0.log”

image03

As we’ve shown,  Pivotal Cloud Foundry makes the delivery and management of applications extremely easy and error free. Working with our partners to create an ecosystem of integrated services for customers delivers tremendous value, and provides an elegant transition to the delivery of cloud native applications while preserving and integrating with existing solutions.

Monitoring Apps on the Cloud Foundry PaaS

At AppDynamics, we pride ourselves on making it easier to monitor complex applications. This is why we are excited to announce our partnership with Pivotal to make it easier to deploy built-in application performance monitoring to the cloud.

 

Getting started with Pivotal’s Cloud Foundry Web Service

Cloud Foundry is the open platform as a service, developed and operated by Pivotal. You can deploy applications to the hosted Pivotal Web Services (much like you host apps on Heroku) or you can run your own Cloud Foundry PaaS on premise using Pivotal CF. Naturally, Cloud Foundry is an open platform that is used and operated by many companies and service providers.

1) Sign up for a Pivotal CF account and AppDynamics Pro SaaS account

In the future, Pivotal Web Services will include the AppDynamics SaaS APM services, so you’ll only need to sign up for Pivotal Web Services and it will automatically create an AppDynamics account.

2) Download the Cloud Foundry CLI (Command Line Interface)

Pivotal Web Services has both a web based GUI as well as a full featured linux style command line interface (CLI). Once you have a PWS account, you can download a Mac, Windows or Unix CLI from the “Tools” tab in the PWS dashboard or directly for OSX, Linux, and Windows.

Pivotal Web Services CLI

3) Sign in with your Pivotal credentials

Using the CLI, log in to your Pivotal Web Services account. Remember to preface all commands given to Cloud Foundry with “cf”.  Individual Cloud Foundry PaaS clouds are identified by their domain API endpoint. For PWS, the endpoint is api.run.pivotal.io. The system will automatically target your default org (you can change this later) and ask you to select a space (a space is similar to a project or folder where you can keep a collection of app(s).

$ cf login

Cloud Foundry CLI 

Monitoring Cloud Foundry apps on Pivotal Web Services

Cloud Foundry uses a flexible approach called buildpacks to dynamically assemble and configure a complete runtime environment for executing a particular class of applications. Rather than specifying how to run applications, your developers can rely on buildpacks to detect, download and configure the appropriate runtimes, containers and libraries. The AppDynamics agent is built-in to the Java buildpack for easy instrumentation so if you have AppDynamics monitoring running, the Cloud Foundry DEA will auto-detect the service and enable the agent in the buildpack. If you start the AppDynamics monitoring for an app already running, just restart the app and the DEA will autodetect the new service.

1) Clone the Spring Trader demo application

The sample Spring Trader app is provided by Pivotal as a demonstration. We’ll use it to show how monitoring works. First git clone the app from the Github repository.

$ git clone https://github.com/cloudfoundry-samples/rabbitmq-cloudfoundry-samples

2) Create a user provided service to auto-discover the AppDynamics agent

$ cf create-user-provided-service demo-app-dynamics-agent -p “host-name,port,ssl-enabled,account-name,account-access-key”

Cloud Foundry CLI

Find out more about deploying on PWS in the Java buildpack docs.

3) Use the Pivotal Web Services add-on marketplace to add a cloud based AMQP + PostgreSQL database instance

$ cf create-service elephantsql turtle demo-db

$ cf create-service cloudamqp lemur demo-amqp

Cloud Foundry CLI

4) Bind PostgreSQL, AMQP, and AppDynamics services to app

$ git clone https://github.com/cloudfoundry-samples/rabbitmq-cloudfoundry-samples

$ cd rabbitmq-cloudfoundry-samples/spring

$ mvn package

$ cf bind-service demo-app demo-app-dynamics-agent

$ cf bind-service demo-app demo-amqp

$ cf bind-service demo-app demo-db

Cloud Foundry CLI

5) Push the app to production using the Cloud Foundry CLI (Command Line Interface)

$ cf push demo-app -i 1 -m 512M -n demo-app -p target/rabbitmq-spring-1.0-SNAPSHOT.war

Cloud Foundry CLI

Spring AMQP Stocks Demo App

Spring Trader

Pivotal Web Services Console

Pivotal PaaS CloudFoundry

 

 

Production monitoring with AppDynamics Pro

Monitor your critical cloud-based applications with AppDynamics Pro for code level visibility into application performance problems.

AppD Dashboard

Pivotal is the proud sponsor of Spring and the related open-source JVM technologies Groovy and Grails. Spring helps development teams build simple, portable, fast, and flexible JVM-based systems and applications. Spring is the most popular application development framework for enterprise Java. AppDynamics Java agent supports the latest Spring framework and Groovy natively. Monitor the entire Pivotal stack including TC server and Web Server, GreenPlum, RabbitMQ, and the popular Spring framework:

AppD

 

Take five minutes to get complete visibility into the performance of your production applications with AppDynamics today.