As CTO of Data Science at AppDynamics, and in my previous role as co-founder of Perspica, I’ve seen machine learning make huge strides in recent years. ML has helped Netflix perfect binge watching, taught Siri how to sound more human and made Amazon Echo a fashion consultant. But when it comes to machine learning use cases for the enterprise, it gets a whole lot more complicated. It’s easy to apply an algorithm to a one-off use case, but comprehensive enterprise applications of machine learning don’t exist today.
Here are the top 8 challenges standing in the way of widespread adoption of machine learning in the enterprise.
1) Confusion Over What Constitutes Machine Learning
Part of the problem is a lack of understanding around what machine learning is. Machine learning is an application or subset of AI, which is generally thought of as higher-order decision-making intelligence.
Machine learning is really about applying mathematics to different domains. It locates meaning within extremely large volumes of data by canceling out the noise. It uses algorithms to parse the data and draw conclusions about it, such as what constitutes normal behavior.
2) Uncertainty About What Machine Learning Can Do
Machine-learning algorithms don’t enter chess tournaments. What they are really good at is adapting to changing systems without human intervention while continuing to differentiate between expected and anomalous behavior. This makes machine learning useful in all kinds of applications—think everything from security to health care—as well as classification and recommendation engines, and voice and image identification systems.
Consumers interact daily with dozens of machine learning systems including Google Search, Google ads, Facebook ads, Siri and Alexa, as well as virtually any online product recommendation engine from Amazon to Netflix. The challenge for enterprises is understanding how machine learning can add value to their business.
3) Getting Started Can Be Daunting
Machine learning is usually introduced into an enterprise in one of two ways. The first is that one or two employees start applying machine learning to gain insight into data they already have access to. This requires a certain amount of expertise in data science and domain knowledge—skills that are in short supply.
The second is by purchasing a solution, such as security software or application performance management solution, that uses machine learning. This is by far the easiest way to begin to realize some of the benefits of machine learning, but the downside is an enterprise is dependent on the vendor and is not developing its own machine learning capabilities.
4) The Challenge of Data Preparation
Machine learning can sound deceptively simple. It’s easy to assume that all you have to do is collect the data and run it through some algorithms. The reality is very different. Once you collect the data then you have to aggregate it. You need to determine if there are any problems with it. Your algorithm needs to be able to adapt to missing data, outlying data, garbage data, and data that’s out of sequence.
5) The Lack of Public Labelled Datasets
In order for an algorithm to make sense of a collection of data points, it needs to understand what those points represent. In another words, it needs to be able to apply pre-established labels to the data.
The availability of publicly labelled datasets would make it much easier for companies to get started with machine learning. Unfortunately, these do not yet exist, and without them, most companies are looking at a “cold start.”
6) The Need for Domain Knowledge
At its best, machine learning represents the perfect marriage between an algorithm and a problem. This means domain knowledge is a prerequisite for effective machine learning, but there is no off-the-shelf way to obtain domain knowledge. It is built up in organizations over time and includes not just the inner workings of specific companies and industries, but the IT systems they use and the data that is generated by them.
7) Hiring Brilliant Data Scientists Is Not a Panacea
Most data scientists are mathematicians. Depending on their previous job experience, they may have zero domain knowledge that is relevant to their employer’s business. They need to be paired up with analysts and domain experts, which increases the cost of any machine learning project. And these people are hard to find and in high demand. We are lucky at AppDynamics to have a team of data scientists with broad experience in multiple fields who are doing ground-breaking work.
8) Machine Learning Lacks a Shared Vocabulary
One of the challenges encountered by organizations with successful machine learning initiatives is the lack of conventions around communicating findings. They end up with silos of people, each with their own definition of input and their own approach to sampling data. Consequently, they end up with wildly different results. This makes it difficult to inspire confidence in machine learning initiatives and will slow adoption until it is addressed.
At AppDynamics we’re excited to apply our machine learning expertise to solving enterprise IT problems. And you may be interested in my insights on how the arrival of AI and machine learning in the enterprise will have a profound impact on IT departments.
