PHOTO:
Shutterstock
Last year, in research titled, The Best Jobs in the U.S. 2019, Austin-based Indeed, which develops a worldwide employment-related search engine, identified machine learning engineer as the fastest-growing job title in the world and said that the trend was likely to last into the future. With an average salary base of $146,085, those considering a career change or new to the workforce might want to consider training for this role.
Machine learning engineer was not the only tech role, though. Nine of the 25 best jobs of 2019 were in the tech field. Some roles are technical, requiring significant investments in education and training for job seekers. For example:Machine learning engineer #1Full-stack developer #3Salesforce developer #10Robotics engineer #11Computer vision engineer #13Data scientist, #22
What Is a Machine Learning Engineer?
But what exactly are we talking about here? Over the past years with the entry of artificial intelligence into the enterprise, the roles of those working in machine learning has evolved dramatically. The machine learning engineer is that of a computer programmers, but their focus goes beyond specifically programming machines to perform specific tasks. They create programs that will enable machines to take actions without being specifically directed to perform those tasks. But like everything else in the digital workplace, it too is evolving. So, what does a machine engineer do now?
Bridging Model-Building and Production
Cameron Faxon a senior data engineer at Valkyrie, an Austin-based strategic AI startup whose data scientists deliver custom machine learning capabilities to businesses, explained that the general purpose and goal of a machine learning engineer is to act as a bridge between the statistical and model-building work of the data scientist and the building of production-ready and robust ML and AI systems, platforms and services, in many cases collaborating with software engineering teams.
To put it another way, a great individual data scientist or data science team might be producing some of the most cutting-edge models available within a given domain, but until they are readily available to the target audience (e.g., customers, business clients, government agencies or the general public), they are somewhat hidden away behind a barrier that requires a certain level of technical ability to break through. “The role of the ML Engineer is to use their knowledge of ML/AI and combine it with programming and software engineering skills to enable easier use of and access to said models and analyses,” he said.
The situation is a little different for companies that are developing ML/AI for internal use, but the goal remains the same: implement isolated statistical analysis/ML/AI into high-performance, high-availability production level systems that provide quick and easy access to all interested users or stakeholders. Examples of specific tasks might include:
Translating the work of data scientists from environments such as python/R notebooks analytics applications.
Creation of web services/APIs for serving ML/AI model results and enabling access to customers or internal teams.
Automating model training and evaluation processes.
Automating feature engineering, ensuring data for model training is cleaned and readily available and facilitation of the flow of data between ML/AI models and an organization’s data systems.
Improving Systems
Jason Lichon is president and co-founder of Chicago-based BlueBolt, a digital agency that provides B2C and B2B clients with digital solutions for creative design, enterprise search and content management. He pointed out that an ML engineer is responsible for developing or applying machine learning algorithms to improve systems or processes by automating routine tasks that would otherwise be done by humans.
A successful ML engineer can observe a task currently being performed by people and understand how to break it down in a way that would allow for automation. The job requires skills in programming, analysis, and an understanding of the techniques and tools used to apply ML to real world tasks. Comfort with mathematical modeling is also beneficial. “We are still in the early days of machine learning, and relatively few programmers have acquired the skills necessary to implement ML solutions,” he said. “The use of ML is expanding rapidly, however, and more programmers are becoming educated in the techniques and tools.”
Improving Operational Efficiency
At Santa Clara, Calif.-based LeanTaaS, Ayush Parolia, senior data scientist, says they use machine learning to improve the operational efficiency of hospitals. There are two steps:
Customer needs: First, we look at customer needs that can be solved by machine learning and work backwards.
Identifying patterns: They then look at the data, identify patterns and come up with ways to improve our products.
For LeanTaaS and other companies that use machine learning, on a day-to-day basis, machine learning engineers spend their time doing several things:
Exploring data: Organizing, cleansing, and analyzing data and finding patterns and attributes that could be used to build machine learning models.
Exploring ideas: Brainstorming with product managers on customer needs and pain points that can be addressed with machine learning, then coming up with ideas and exploring them.
Enhancing data pipelines and lakes: Ensuring that data is clean, accurate and optimized for machine learning models at any given point in time.
Fine-tuning production models: Monitoring and fine-tuning the machine learning models we have in production on a daily or weekly basis.
Improving team productivity: Creating and enhancing frameworks and libraries that can improve our overall productivity and help us run our experiments faster and build better models.
Machine learning engineers come in many flavors, but fundamentally, machine learning is a field that any software engineer can build expertise in. Software engineers have the analytical and mathematical foundation for it and can explore a wide variety of ML models to solve specific problems and gain expertise over time.
Task Oriented Machine Learning
Finally, a machine learning engineer trains, monitors, optimizes, tests, and deploys machine learning algorithms for specific tasks. A ML engineer builds on top of the work of a data engineer, Rosaria Silipon principal data scientist at Switzerland-based KNIME, told us. The data engineer prepares the data, that is he or she cleans and standardizes them
However, the most important is that ML engineers should implement and carry on more ML specific transformations, such as dimensionality reduction, outlier detection, feature engineering, missing value imputation, normalization if needed, and probably other similar operations. Once the data is ready to feed the ML algorithm, the responsibility of the ML engineer is to set the training algorithm appropriately and execute it in a reasonable time to produce a satisfactory performance. Hyperparameter optimization and performance evaluation, of course, also belong to the set of responsibilities of the ML engineer.
However, collecting data from data sources, blending them together, extracting KPIs, cleaning and standardizing the results are activities more frequently needed than the training of a full-blown machine learning or deep learning model.
Machine Learning Engineer Demand
There is one problem, though. ML engineers are in short supply. Robert Weissgraeber is CTO at San Francisco-based AX Semantics. He says that most programming jobs nowadays require mid- to deep knowledge of machine learning. “There is a high demand with a limited supply of talent, which drives salaries higher and makes it hard for lesser known companies to attract talent. Engineers tend to be drawn to innovative, modern-culture companies and a lot of organizations struggle with such transformations. The U.S. is not alone in this either,” he said. In Germany, for example, 27% of companies (vs. 23% globally) say their main roadblock to AI is a shortage of specialists — though other European cities like Paris with “Station F” are already creating innovation hubs to solve for the shortage.
The trend now and in the near future is that companies will need or want to hire more data/machine learning engineers. In the recent past, many companies went on hiring sprees for data scientists as ML/AI became the hot new trend in technology, Cameron Faxon at Valkyrie shared. For some companies, this paid off quickly and they were able to get the full value of what they needed from data science teams (e.g., analytics dashboards, statistical reports, models that answered crucial internal business questions).
However, there is likely a large amount of untapped value in data science teams at many companies that do not have the engineering talent to really put the products of the data science team to full use and integrate it into their existing data systems so that non-technical members of the organization can realize that value.
Source: https://www.cmswire.com/information-management/what-do-machine-learning-engineers-do-anyway/
