Over the past two decades, the biggest evolution of Artificial Intelligence has been the maturation of deep learning as an approach for machine learning, the expansion of big data and the knowledge of how to effectively manage big data systems, and affordable and accessible compute power that can handle some of the most challenging machine learning model development. Today’s data scientists and machine learning engineers now have a wide range of choices for how they build models to address the various patterns of AI for their particular needs.
However, The diversity in options is actually part of the challenge for those looking to build machine learning models. There are just too many choices. This, compounded by the fact that there are different ways you can go about developing a machine learning model, is the issue that many AI software vendors do a particularly poor job of explaining what their products actually do. Marketing skills and websites that actually explain what the product and/or company actually does are clearly lacking for many AI vendors. This makes it difficult for those looking to implement machine learning models to choose the best vendor option for them.
Untangling the complexity of machine learning model development
Machine learning is the cornerstone of AI. Without a way for systems to learn from experience and example, they aren’t able to achieve higher order cognitive tasks that require learning patterns from data. Without machine learning, machines need instructions and rules programmed and developed by humans to tell them what to do which isn’t intelligence – that’s just programming.
There are three primary approaches to machine learning: supervised learning in which machines learn from human-tagged examples, unsupervised learning in which machines discover patterns in the data, and reinforcement learning where machines learn from trial-and-error with a reward-based system. Each of these approaches are applicable and appropriate for different learning scenarios.
Furthermore, there are a wide range of algorithms that machine learning practitioners can use to implement those various learning approaches. These algorithms have different tradeoffs and performance characteristics. In addition, the end result of training a particular algorithm on particular training data is a machine learning model. The model represents what the machine has learned for a particular task. People seem to often confuse the machine learning algorithm, which tells machines the approach they should use to encode learning, and the machine learning model, which is the outcome of that learning. New algorithms are not frequently developed as new approaches to learning are few and far between. New models, however, are developed all the time since each new learning is encoded in a model, which can happen an infinite amount of times.
In addition to the above challenges, building machine learning models can be particularly challenging, especially for those that have limited data science and machine learning skills and understanding. Those with deep technical capabilities and strong statistical understanding can optimize and tweak models and choose appropriate algorithms with the right settings (“hyperparameters”) with their experience, while others that are fairly new to model creation might be stumped by all the choices that need to be made to select the right modeling approach. Tools on the market to develop machine learning models cater to a wide range of needs from novice to expert, making tool selection that much more challenging.
MORE FOR YOU
Five Major Approaches to ML Model Development
A recent report by AI market research and advisory firm Cognilytica identifies five major approaches to machine learning model development:
Machine Learning Toolkits
The field of machine learning and data science is not new, with decades of research from academicians, researchers, and data scientists. As a result, there are a large collection of toolkits that enable knowledgeable machine learning practitioners to implement a wide range of algorithms with low-level configurability. These machine learning toolkits are very popular and many are open source. Some toolkits are focused on specific machine learning algorithms and applications, most notably Keras, Tensorflow, and PyTorch which are focused on deep learning models, while others such as Apache Mahout and SciKit Learn provide a range of machine algorithms and tools. These toolkits are in turn embedded in many larger machine learning platforms including those mentioned below. In addition, many of the machine learning toolkits have the support and ongoing development resources of large technology companies. For example, Facebook supports PyTorch, Google supports Keras and TensorFlow, Amazon supports MXNet, Microsoft supports CNTK Toolkit, and others are supported by companies like IBM, Baidu, Apple, Netflix, and others.
Data Science Notebooks
The realm of machine learning is that of data science, since after all, we’re trying to derive higher value insights from big data. The primary environment for data science is the “notebook”, which is a collaborative, interactive, document-style environment that combines aspects of coding, data engineering, machine learning modeling, data visualization, and collaborative data sharing. Open source notebooks such as Jupyter and Apache Zeppelin have become widely adopted and have also found their way into the commercial platform offerings.
Data science notebooks offer the full breadth of machine learning algorithms through support and embedding of many of the popular machine learning toolkits mentioned above. While data science notebooks can be used to develop models of any type, they are primarily used during the experimentation and iteration phases of model development, since data science notebooks are optimized for that sort of iterative experimentation versus being focused on organization-wide aspects of management and deployment.
Machine Learning Platforms
Organizations that are looking to make mission-critical use of machine learning know that simply building a machine learning model is not all that needs to be taken into consideration for ML model needs. The full lifecycle of machine learning model development includes aspects of data preparation and engineering, machine learning model iteration including the use of “AutoML” to automatically identify the best algorithms and settings to achieve desired outcomes, machine learning model evaluation, and ML model iteration and versioning including the emerging area of “ML Ops”.
As a result, the last decade has seen the explosive emergence of full-lifecycle machine learning platform solutions that aim to not only simplify ML model development but also address these other areas of managing the ML model lifecycle. Many companies in this space have emerged as small startups to become major powerhouses in the industry with ever-increasing solutions that tackle a wider array of needs for data scientists and machine learning engineers.
Before data science was called data science, there was the field of analytics and business intelligence. Many tools that were once used for non-machine learning analytics have since added machine learning model development to their capabilities. Most of the analytics field is dominated by a few large commercial analytics firms, which are increasingly broadening their offerings. As such, data scientists that might have experience with those tools will find increasing capabilities for machine learning model development and broader lifecycle capabilities.
These solutions traditionally aimed at data analytics, statistics, and mathematics applications have realized the power of adding machine learning capabilities to their existing statistical and/or analytics offerings. Organizations that have already invested in analytics solutions will find that they can retain the skill, experience, and investment in their existing tools that now support machine learning development and deployment.
Cloud-based ML-as-a-Service (MLaaS)
In addition to the above approaches, most of the large cloud providers have jumped in with both feet into the machine learning space. Amazon, Google, IBM, and Microsoft have all added core capabilities for machine learning model development, management, and iteration as well as capabilities for data preparation, engineering, and augmentation. These cloud vendors also support and use many of the open source ML toolkits as well as the Data Science Notebooks popular in the field. As a result, the decision to use the cloud for ML model development is rarely an “either/or” decision, but more of a tactical decision on whether the use of cloud-based resources for computing, data storage, and value-added ML lifecycle capabilities is needed.
The growth of machine learning model markets
No doubt the field of machine learning model development continues to expand. Cognilytica expects the market for machine learning platforms to reach over $120 Billion USD by 2025, growing at a fast and furious rate. (Disclosure: I’m a principal analyst with Cognilytica) While there might be questions as to how long this latest wave of AI will last, there’s no doubt that the future of machine learning development and implementation looks bright.