AppDynamics Database Visibility adds support for MongoDB

AppDynamics is proud to announce support for MongoDB within the Database Visibility module. MongoDB was created in 2007 as a commercial product, switching to open source in 2009. It has grown rapidly due to its storage of data in a small set of documents, field-indexing, high availability and replication, shard-based scalability, ad hoc queries, aggregation and use of JavaScript on the server. Currently, it is the fourth most popular database in the world. Organizations implementing it include SAP, MetLife, Viacom, Foursquare, and Cisco.

The AppDynamics for Databases MongoDB collector provides critical information

Monitoring of CPU usage percentages. High user CPU numbers might make a case for more CPU resources. High system CPU could indicate a more powerful processor should be implemented, and increased I/O wait states may show additional or faster disks are needed.

Automatically detected multi-shard configurations. AppDynamics will auto discover every shard and replica set.

 

Ongoing metrics. Metrics are collected for every shard in a cluster.

How MongoDB Differs From SQL Databases

MongoDB is a document-oriented database that allows both structured and unstructured data. Relational SQL databases arrange data in highly organized tables with explicit rules. They emerged in the 1970s because, at the time, there was no defined way to structure databases, fields, and data operations. Characteristics of relational databases include joins and support for both constraints and complex transactions.

This worked great until the current age of big data, where one application might handle millions of transactions. NoSQL databases like MongoDB provide the performance, availability, and scalability needed by modern apps. MongoDB also handles unstructured data, such as video, tweets, and audio files. With NoSQL, support for complex transactions and constraints must be dealt with by the application itself. At times, other languages are combined with NoSQL to handle these operations. That is why NoSQL is sometimes called “Not Only SQL” by some developers.

MongoDB Addresses Modern Data Challenges

Today’s apps are interactive, social and highly networked. They need to be able to handle tons of data, develop new features rapidly, and be deployed in a number of environments. They are storing significantly more data than ever before and accessing it at unprecedented speeds. At the same time, organizations running on a single server run into problems because they cannot scale.

MongoDB is a NoSQL database that was developed to address these challenges:

  • Allows companies to scale by adding more servers as they are needed.
  • Models data as documents, increasing query speed significantly.
  • Capably handles fast-moving environments of continuous deployment and agile development, demands that can cripple older data models with less flexibility.
  • Works well on lean servers, which saves money.

Storing and Managing Rapidly Growing Data Sets

Organizations using MongoDB rely on it to meet a variety of challenges, including:

  • How to store and access large quantities of data.
  • How to handle rapidly growing lists of diverse elements, such as server logs, twitter feeds and more.
  • How to effectively use unstructured data or data that changes rapidly.
  • How to develop applications quickly.

A MongoDB object is another name for the documents that store data. Objects are associated arrays and data structures represented in the database in a data interchange format called BSON (pronounced Bison). With MongoDB, data can be stored using multi-sharding, which is spreading data across several machines. Sharding is used to manage databases with rapid throughput demands and huge data sets.

MongoDB has grown rapidly in a few short years, and now only Oracle, MySQL and Microsoft SQL Server are ahead of it on the list of most popular databases. With AppDynamics for Databases now supporting MongoDB, you’ll be able to effectively monitor your critical applications to minimize errors, downtime and profit loss.

AppDynamics and MongoDB Work Together to Scale Databases [VIDEO]

If you haven’t noticed, enterprise IT is going through a generational inversion. Not necessarily in people (although that seems to be starting too) but in technology. There is a move from on-premise to cloud. From difficult to use enterprise applications with extended training required, to consumer-grade, intuitive interfaces that are nearly as easy to use as your mobile phone. From the legacy databases that dominated enterprise technology in the 90s and early 2000s for managing relational data types to agile, scalable NoSQL data stores for managing semi-structured, polymorphic, object-style, evolving structured data, and other data shapes.

AppDynamics is leading and learning from this generational shift and our partner, MongoDB, is too. We have seen enterprise IT developers across nearly all verticals — agriculture, financial services, government – leverage MongoDB to better manage their business. That is why in 2011, we built an extension for MongoDB support to plug into our Application Intelligence Platform. This monitoring extension provides visibility into the directory structure and a wealth of metrics on your servers like uptime, flushes, memory utilization, and query counts in addition to stats on your database like object counts and sizes, index counts, and namespace file sizes. But really, what is most interesting about this plugin is the ability to view across the application stack, automatically discover the MongoDB back-end, and identify when you have a problem that exists anywhere in between. Full end-to-end visibility is critical for developing and running modern, highly distributed applications in production.

With the continued rise of NoSQL databases in general, and MongoDB specifically, we decided to go one step further. In our 2014 Spring Release we announced native support for MongoDB in our Database Monitoring product. This release provides stand alone, enterprise-class tooling for MongoDB and highlights just how creaky and tired the monitoring supplied by the legacy database vendors has become. So, now application owners, DevOps teams, and DBAs can have monitoring for MongoDB whether as part of an end-to-end monitoring platform or as a stand-alone deployment for inspecting the MongoDB database. Identifying root causes of application latency is only a couple clicks away – without a bunch of training, a long installation process, or any of the baggage that came with the old generation of enterprise IT systems.

We recently caught up with Matt Asay, VP of Marketing and Business Development at MongoDB, to discuss the partnership. Check out the video below to see how customers are using AppDynamics and MongoDB as critical components in the new generation IT shop.

One of the reasons we wanted to work with AppDynamics is to give our users, our end customers, that holistic view the deep view of what’s going on in their application … We would like our customers to get to that root cause of the problem sooner rather than later … It’s fantastic working with AppDynamics.

Want to learn more about the future of Enterprise IT and NoSQL? Be sure to register for MongoDB World 2014 taking place in New York, June 24 & 25.  You’ll learn everything you need to know to build and manage modern applications built on MongoDB. AppDynamics will be in booth #111.

Want to start monitoring your MongoDB database? Check out a free trial of AppDynamics today!

Big Data Monitoring

The term “Big Data” is quite possibly one of the most difficult IT-related terms to pin down ever. There are so many potential types of, and applications for Big Data that it can be a bit daunting to consider all of the possibilities. Thankfully, for IT operations staff, Big Data is mostly a bunch of new technologies that are being used together to solve some sort of business problem. In this blog post I’m going to focus on what IT Operations teams need to know about big data technology and support.

Big Data Repositories

At the heart of any big data architecture is going to be some sort of NoSQL data repository. If you’re not very familiar with the various types of NoSQL databases that are out there today I recommend reading this article on the MongoDB website. These repositories are designed to run in a distributed/clustered manner so they they can process incoming queries as fast as possible on extremely large data sets.

MongoDB Request Diagram

Source: MongoDB

An important concept to understand when discussing big data repositories is the concept of sharding. Sharding is when you take a large database and break it down into smaller sets of data which are distributed across server instances. This is done to improve performance as your database can be highly distributed and the amount of data to query is less than the same database without sharding. It also allows you to keep scaling horizontally and that is usually much easier than having to scale vertically. If you want more details on sharding you can reference this Wikipedia page.

Application Performance Considerations

Monitoring the performance of big data repositories is just as important as monitoring the performance of any other type of database. Applications that want to use the data stored in these repositories will submit queries in much the same way as traditional applications querying relational databases like Oracle, SQL Server, Sybase, DB2, MySQL, PostgreSQL, etc… Let’s take a look at more information from the MongoDB website. In their documentation there is a section on monitoring MongoDB that states “Monitoring is a critical component of all database administration.” This is a simple statement that is overlooked all too often when deploying new technology in most organizations. Monitoring is usually only considered once major problems start to crop up and by that time there has already been impact to the users and the business.

RedisDashboard

Dashboard showing Redis key metrics.

One thing that we can’t forget is just how important it is to monitor not only the big data repository, but to also monitor the applications that are querying the repository. After all, those applications are the direct clients that could be responsible for creating a performance issue and that certainly rely on the repository to perform well when queried. The application viewpoint is where you will first discover if there is a problem with the data repository that is actually impacting the performance and/or functionality of the app itself.

Monitoring Examples

So now that we have built a quick foundation of big data knowledge, how do we monitor them in the real world?

End to end flow – As we already discussed, you need to understand if your big data applications are being impacted by the performance of your big data repositories. You do that by tracking all of the application transactions across all of the application tiers and analyzing their response times. Given this information it’s easy to identify exactly which components are experiencing problems at any given time.

FS Transaction View

Code level details – When you’ve identified that there is a performance problem in your big data application you need to understand what portion of the code is responsible for the problems. The only way to do this is by using a tool that provides deep code diagnostics and is capable of showing you the call stack of your problematic transactions.

Cassandra_Call_Stack

Back end processing – Tracing transactions from the end user, through the application tier, and into the backend repository is required to properly identify and isolate performance problems. Identification of poor performing backend tiers (big data repositories, relational databases, etc…) is easy if you have the proper tools in place to provide the view of your transactions.

Backend_Detection

AppDynamics detects and measures the response time of all backend calls.

Big data metrics – Each big data technology has it’s own set of relevant KPIs just like any other technology used in the enterprise. The important part is to understand what is normal behavior for each metric while performance is good and then identify when KPIs are deviating from normal. This combined with the end to end transaction tracking will tell you if there is a problem, where the problem is, and possibly the root cause. AppDynamics currently has monitoring extensions for HBase, MongoDB, Redis, Hadoop, Cassandra, CouchBase, and CouchDB. You can find all AppDynamics platform extensions by clicking here.

HadoopDashboard

Hadoop KPI Dashboard 1

HadoopDashboard2

Hadoop KPI Dashboard 2

Big data deep dive – Sometimes KPIs aren’t enough to help solve your big data performance issues. That’s when you need to pull out the big guns and use a deep dive tool to assist with troubleshooting. Deep dive tools will be very detailed and very specific to the big data repository type that you are using/monitoring. In the screen shots below you can see details of AppDynamics monitoring for MongoDB.

MongoDB Monitoring 1

 MongoDB Monitoring 2

MongoDB Monitoring 3

MongoDB Monitoring 4

If your company is using big data technology, it’s IT operations’ responsibility to deploy and support a cohesive performance monitoring strategy for the inevitable performance degradation that will cause business impact. See what AppDynamics has to offer by signing up for our free trial today.

AppDynamics Monitoring for MongoDB

Back in April this year, we announced new monitoring support for NoSQL tecnhnologies like Cassandra and Memcached which was received really well by our customers. Due to further demand we’ve seen over the past few months from prospects and customers we’ve officially added MongoDB to our platform support, thus extending our coverage of NoSQL technologies.

For those not familiar with MongoDB, here’s how it differs from the traditional relational database just so you know what data and context AppDynamics provides for monitoring MongoDB applications:

.NET Application Performance delivered with AppDynamics 3.3

It’s official: AppDynamics support for Microsoft .NET and Windows Azure is finally here! We’ve got the same Kick Ass Product with the same Secret Sauce–but now it sports a shiny new CLR agent. So whether your apps are Java,  .NET or hybrid, with AppDynamics you have the best of both worlds when it comes to managing application performance.

We thought it was only fair to share our secret sauce and love with the Microsoft community, given that 40,000+ users of the Java community have been enjoying it for over 18 months. Our mission is to simplify the way organizations manage their agile, complex, and distributed applications. For .NET, this means that AppDynamics supports the latest and greatest technologies from Microsoft, including their new PaaS platform Windows Azure.

So, what does this mean for organizations with .NET or Azure apps? Let me summarize:

#1 You get to visualize in real-time what your .NET application actually looks like, along with its health and performance across any distributed production environment. It’s the 50,000 foot view that shows how your application and your business performs in your data center or Windows Azure.

#2 Ability to track all business transactions that flow through your .NET application. This gives you insight into business activity, health, and impact in the event that a slowdown or problem occurs in your production environment. This unique context and visibility helps you troubleshoot through the eyes of the business, so you can see their pain instantly in production and resolve it in minutes. We auto-discover and map every business transaction automatically–so don’t worry about configuration. We’ve got that taken care of.

 

#3 Deep diagnostic information on how your business transactions actually execute through your CLRs and/or JVMs (if you’ve got a hybrid app). This means complete call stacks of code execution with latency breakdowns across all your namespaces, classes and methods, which your business transactions invoke. You get maximum visibility in production with zero configuration, allowing you to perform root cause analysis in minutes.

#4 Ability to plot, correlate, and trend any CLR or OS metric over time–whether it’s logic thread counts, garbage collection time, or simply how much CPU your application CLR is burning. We let you report and analyze all this so you understand your CLR run-time and OS system resource.

Don’t believe us? Sign up for our free 30-day trial and we’ll provision you a SaaS login. You can then download and install our lightweight agents and see for yourself just how easy it can be!

As well as our .NET support, we’ve also crammed in some great innovation into the 3.3 release.

Real-Time JVM MBean Viewer:

In addition to trending standard JMX metrics from the JVM, users can now discover and trend any MBean attributes on the fly for short term analysis in real-time. Our new UI dialogue allows the user to browse through hundreds of available metrics which are automatically discovered and reported at the touch of a button. If the user wishes to convert any MBean attribute into a standard JMX metric they can just click “Create Metric” and AppDynamics will collect and report that metric as standard in the JMX Metrics viewer.

Search Business Transactions by their content/payload:

For example, you might have launched a new product on your application or website and need to understand its performance by looking at all business transactions that interact with that product. With AppDynamics v3.3 users can now search business transactions by any transaction payload. For example, the below screenshot shows how a user can search for all business transactions that relate to the book “Harry Potter”.

Additional Platform Support:

  • Auto-discovery and mapping of LDAP, SAP and JavaMail tiers to business transaction flows for increased visibility.
  • MongoDB support allowing users to see BSON queries and associated latency for calls made from Java applications.
  • Enhanced support for WebSphere on Z/OS with automatic JVM naming pools to help customers identify and manage short-living and dynamic JVM run-times.
All in all another great release packed full of innovation from the AppDynamics team. Stay tuned over the next few weeks for more information on specific 3.3 features.
App Man.