MLOps: What You Need To Know

Tom Taulli Tom Taulli
August 14, 2020 AI & Machine Learning

MLOps is a relatively new concept in the AI (Artificial Intelligence) world and stands for “machine learning operations.” Its about how to best manage data scientists and operations people to allow for the effective development, deployment and monitoring of models. 

“MLOps is the natural progression of DevOps in the context of AI,” said Samir Tout, who is a Professor of Cybersecurity at the Eastern Michigan University’s School of Information Security & Applied Computing (SISAC). “While it leverages DevOps’ focus on security, compliance, and management of IT resources, MLOps’ real emphasis is on the consistent and smooth development of models and their scalability.”

The origins of MLOps goes back to 2015 from a paper entitled “Hidden Technical Debt in Machine Learning Systems.” And since then, the growth has been particularly strong. Consider that the market for MLOps solutions is expected to reach $4 billion by 2025. 

“Putting ML models in production, operating models, and scaling use cases has been challenging for companies due to technology sprawl and siloing,” said Santiago Giraldo, who is the Senior Product Marketing Manager and Data Engineer at Cloudera. “In fact, 87% of projects don’t get past the experiment phase and therefore, never make it into production.

Then how can MLOps help? Well, the handling of data is a big part of it.

“Some key best practices are having a reproducible pipeline for data preparation and training, having a centralized experiment tracking system with well-defined metrics, and implementing a model management solution that makes it easy to compare alternative models across various metrics and roll back to an old model if there is a problem in production,” said Matei Zaharia, who is the chief technologist at Databricks. “These tools make it easy for ML teams to understand the performance of new models and catch and repair errors in production.”

Something else to consider is that AI models are subject to change. This has certainly been apparent with the COVID-19 pandemic. The result is that many AI models have essentially gone haywire because of the lack of relevant datasets. 

“People often think a given model can be deployed and continue operating forever, but this is not accurate,” said Randy LeBlanc, who is the VP of Customer Success at RapidMiner. “Like a machine, models must be continuously monitored and maintained over time to see how they’re performing and shifting with new data–ensuring that they’re delivering real, ongoing business impact. MLOps also allows for faster intervention when models degrade, meaning greater data security and accuracy, and allows businesses to develop and deploy models at a faster rate. For example, if you discovered an algorithm that will save you a million dollars per month, every month this model isn’t in production or deployment costs you $1 million.”

MLOps also requires rigorous tracking that is based on tangible metrics. If not, a project can easily go off the rails. “When monitoring models, you want to have standard performance KPIs as well as those that are specific to the business problem,” said Sarah Gates, who is an Analytics Strategist at SAS. “This should be through a central location regardless of where the model is deployed or what language it was written in. That tracking should be automated–so you immediately know and are alerted—when performance degrades. Performance monitoring should be multifaceted, so you are looking at your models from different perspectives.”

While MLOps tools can be a huge help, there still needs to be discipline within the organization. Success is more than just about technology. 

“Monitoring/testing of models requires a clear understanding of the data biases,” said Michael Berthold, who is the CEO and co-founder of KNIME. “Scientific research on event, model change, and drift detection has most of the answers, but they are generally ignored in real life. You need to test on independent data, use challenger models and have frequent recalibration. Most data science toolboxes today totally ignore this aspect and have a very limited view on ‘end-to-end’ data science.”

  • Experfy Insights

    Top articles, research, podcasts, webinars and more delivered to you monthly.

  • Tom Taulli

    Tags
    AIDevOpsMLMLOps
    Leave a Comment
    Next Post
    10 Ways AI Is Accelerating DevOps

    10 Ways AI Is Accelerating DevOps

    Leave a Reply Cancel reply

    Your email address will not be published. Required fields are marked *

    More in AI & Machine Learning
    AI & Machine Learning,Future of Work
    AI’s Role in the Future of Work

    Artificial intelligence is shaping the future of work around the world in virtually every field. The role AI will play in employment in the years ahead is dynamic and collaborative. Rather than eliminating jobs altogether, AI will augment the capabilities and resources of employees and businesses, allowing them to do more with less. In more

    5 MINUTES READ Continue Reading »
    AI & Machine Learning
    How Can AI Help Improve Legal Services Delivery?

    Everybody is discussing Artificial Intelligence (AI) and machine learning, and some legal professionals are already leveraging these technological capabilities.  AI is not the future expectation; it is the present reality.  Aside from law, AI is widely used in various fields such as transportation and manufacturing, education, employment, defense, health care, business intelligence, robotics, and so

    5 MINUTES READ Continue Reading »
    AI & Machine Learning
    5 AI Applications Changing the Energy Industry

    The energy industry faces some significant challenges, but AI applications could help. Increasing demand, population expansion, and climate change necessitate creative solutions that could fundamentally alter how businesses generate and utilize electricity. Industry researchers looking for ways to solve these problems have turned to data and new data-processing technology. Artificial intelligence, in particular — and

    3 MINUTES READ Continue Reading »

    About Us

    Incubated in Harvard Innovation Lab, Experfy specializes in pipelining and deploying the world's best AI and engineering talent at breakneck speed, with exceptional focus on quality and compliance. Enterprises and governments also leverage our award-winning SaaS platform to build their own customized future of work solutions such as talent clouds.

    Join Us At

    Contact Us

    1700 West Park Drive, Suite 190
    Westborough, MA 01581

    Email: support@experfy.com

    Toll Free: (844) EXPERFY or
    (844) 397-3739

    © 2023, Experfy Inc. All rights reserved.