Why AI Models badly need Humans to stay Awesome?

Ganes Kesari Ganes Kesari
February 15, 2019 AI & Machine Learning

Ready to learn Machine Learning? Browse courses like Robotics Application Machine Learning developed by industry thought leaders and Experfy in Harvard Innovation Lab.

Companies often expect that AI models once implemented will stay smart and work magically, ever after

Nothing can be farther from the truth.

“What about self-healing algorithms or self-maintaining AI platforms” you may ask. Well, none are close to production-grade enterprise implementations today. And, it will be a while before they obviate any human intervention.

Business leaders often find it disappointing to learn that, a machine learning model, after consuming precious time and dollars of investment still needs humans for routine maintenance. Let’s see why this is the reality today and how companies can plan for it.

A Handyman’s relevance in an AI World

Photo by Todd Quackenbush on Unsplash

To understand why models still need humans, we’ll first start with a simple English explanation of how a machine learning model works, on the inside.

We’ll then cover 4 key scenarios where models usually scream for human help, based on learnings from the work we’ve done at Gramener.

Finally, we’ll conclude by listing specific steps that businesses must plan for, to fail-proof their implementations and keep their models sane and successful.

A sneak — peek into how typical ML models work

Let us say you are a Telecom company looking to solve the big problem of customer churn. You’d like early predictive warnings on those customers who would migrate out in the coming month.

You duly collect a dozen data feeds about customer demographics, purchases, subscription plans, and service interactions. You hand over these 100+ attributes from the past few years, which run into millions of data points.

Data scientists spend their days and nights analyzing and making sense of all this data. They then build and engineer models that can predict, say 8 of 10 customers who would eventually leave. When piloted, this works beautifully and lets you focus on the task of retaining these customers. All is well, so far.

Let’s pause now, and unwrap this magical model to inspect its internals.

Photo by Pierrick VAN-TROOST on Unsplash

From the 100+ factors of customer attributes that you supplied, you’d find just the key 3 to 5 used by the model. For example, these could be, ‘contract tenure’, ‘whether movies were streamed’, ‘kind of service complaints reported’, or more non-intuitive factors like ‘payment mode’, or the ‘number of dependents’.

After studying the strength of all signals, algorithms usually end up using a tiny set of factors (or maybe just 1!) that are most related to customer churn.

The true essence of all classification models can be encapsulated as: “Just give me a, b, c and I’ll tell you the chances for event z to occur.”

That’s all a model needs, and it does just that, at the unit level.

No, it doesn’t always crunch a million data points. No, it doesn’t model the customer brain nor does it perform any eerie reading of the human psyche. And deep learning models are not an exception either.

The entire discipline of machine learning is about identifying those few factors (predictors) and then figuring out their relationship (imagine a formula) to the outcome (target).

Models — Expectations vs Reality

4 Scenarios that shake the fundamentals of a model

Now, you get a sense of how fragile the internals of a model can be, in spite of the apparently sophisticated facade. Unfortunately, people often assume that machines have a grounded, comprehensive understanding of the situation.

To be fair, we humans also base complex decisions on a seemingly small set of factors. However the brain has a superior ability to play with these factors, or their relative importance for a decision. Models have ample catching up to do.

Here are the key scenarios that call for models to get back into the classroom:

1. Change in data structures

Every organization’s technology landscape is continuously evolving, with application upgrades, new tool purchases, and process changes. These impact the data formats and pipelines feeding the model, needing periodic updates.

2. Availability of new intelligence

Businesses routinely discover new feeds to tap into, such as the customer’s online behavior, social feeds or other contextual intelligence. The model must be re-evaluated to check if these new inputs could be stronger predictors.

3. Change in business dynamics

With a change in business strategies, customers could now be drawn to your company for its new rich product features, and no longer for the price economics. Shouldn’t the model be taught this new dynamic to stay relevant?

4. Ongoing refresh and validation

Assuming none of the above changes, the model still needs a health check, say every few months. The model must be re-trained with fresh data to avoid going stale, and accuracy may dip over time calling for some technical tweaks.

Business Scenarios that call for a retooling of Machine Learning models in production

What does it take to keep the models smart?

Having established the need for support, here are the ways that companies can plan to keep their data science investments a.k.a machine learning models refreshed and operational:

  • Budget for changes: We’ve seen that model decay and refurbishment is as natural as the growth cycle of living things. Set aside maintenance dollars by budgeting for the total cost of model ownership, upfront.
  • Plan for the right talent: While discovery and model development get the cheers from data scientists, maintenance is often seen as a drag. Ensure availability of the right talent to tweak and tinker with the models.
  • Watch out for technical debt: As seen in software development, models often carry the weight of past baggage. While some are unavoidable, put in place model processes and accountability to minimize its effect.
  • Create an ML-aware culture: With models increasingly seen as digital workers, companies should create a culture where people co-exist and collaborate with bots, knowing where they can support each other.

Closing thoughts for a more Automated future

Today, there is disproportionately large attention on the model creation and engineering phase, as opposed to model maintenance. Understandably, this is due to the data science industry still being in its infancy.

Upcoming ML advances may help solve some of these challenges. Techniques such as AutoML and Neural Architecture Search, though geared for model creation today, could well be adapted to keep models fit.

Business leaders must adopt a long-term outlook and plan for model upkeep and well-being, to avoid data science disillusionment. For those new to machine learning, I hope this introduction cleared up some of the confusion or perceived magical aura surrounding the world of AI and ML models.

  • Experfy Insights

    Top articles, research, podcasts, webinars and more delivered to you monthly.

  • Ganes Kesari

    Tags
    Artificial Intelligence
    © 2021, Experfy Inc. All rights reserved.
    Leave a Comment
    Next Post
    Math for Machine Learning

    Math for Machine Learning

    Leave a Reply Cancel reply

    Your email address will not be published. Required fields are marked *

    More in AI & Machine Learning
    AI & Machine Learning,Future of Work
    AI’s Role in the Future of Work

    Artificial intelligence is shaping the future of work around the world in virtually every field. The role AI will play in employment in the years ahead is dynamic and collaborative. Rather than eliminating jobs altogether, AI will augment the capabilities and resources of employees and businesses, allowing them to do more with less. In more

    5 MINUTES READ Continue Reading »
    AI & Machine Learning
    How Can AI Help Improve Legal Services Delivery?

    Everybody is discussing Artificial Intelligence (AI) and machine learning, and some legal professionals are already leveraging these technological capabilities.  AI is not the future expectation; it is the present reality.  Aside from law, AI is widely used in various fields such as transportation and manufacturing, education, employment, defense, health care, business intelligence, robotics, and so

    5 MINUTES READ Continue Reading »
    AI & Machine Learning
    5 AI Applications Changing the Energy Industry

    The energy industry faces some significant challenges, but AI applications could help. Increasing demand, population expansion, and climate change necessitate creative solutions that could fundamentally alter how businesses generate and utilize electricity. Industry researchers looking for ways to solve these problems have turned to data and new data-processing technology. Artificial intelligence, in particular — and

    3 MINUTES READ Continue Reading »

    About Us

    Incubated in Harvard Innovation Lab, Experfy specializes in pipelining and deploying the world's best AI and engineering talent at breakneck speed, with exceptional focus on quality and compliance. Enterprises and governments also leverage our award-winning SaaS platform to build their own customized future of work solutions such as talent clouds.

    Join Us At

    Contact Us

    1700 West Park Drive, Suite 190
    Westborough, MA 01581

    Email: support@experfy.com

    Toll Free: (844) EXPERFY or
    (844) 397-3739

    © 2025, Experfy Inc. All rights reserved.