Originally published in Chip Huyen Blog, June 22, 2020
To better understand the landscape of available tools for machine learning production, I decided to look up every AI/ML tool I could find. The resources I used include:
After filtering out applications companies (e.g. companies that use ML to provide business analytics), tools that aren’t being actively developed, and tools that nobody uses, I got 202 tools. See the full list. Please let me know if there are tools you think I should include but aren’t on the list yet!
Disclaimer
This list was made in November 2019, and the market must have changed in the last 6 months.
Some tech companies just have a set of tools so large that I can’t enumerate them all. For example, Amazon Web Services offer over 165 fully featured services.
There are many stealth startups that I’m not aware of, and many that died before I heard of them.
This post consists of 6 parts:
I. OverviewII. The landscape over timeIII. The landscape is under-developedIV. Problems facing MLOpsV. Open source and open-coreVI. Conclusion
I. OVERVIEW
In one way to generalize the ML production flow that I agreed with, it consists of 4 steps:
Project setup
Data pipeline
Modeling & training
Serving
I categorize the tools based on which step of the workflow that it supports. I don’t include Project setup since it requires project management tools, not ML tools. This isn’t always straightforward since one tool might help with more than one step. Their ambiguous descriptions don’t make it any easier: “we push the limits of data science”, “transforming AI projects into real-world business outcomes”, “allows data to move freely, like the air you breathe”, and my personal favorite: “we lived and breathed data science”.
I put the tools that cover more than one step of the pipeline into the category that they are best known for. If they’re known for multiple categories, I put them in the All-in-one category. I also include the Infrastructure category to include companies that provide infrastructure for training and storage. Most of these are Cloud providers.
To continue reading this article click here.
Source: https://www.predictiveanalyticsworld.com/machinelearningtimes/what-i-learned-from-looking-at-200-machine-learning-tools/11475/
