Reality Check: Avoiding an AI Train Wreck

Tim Negris Tim Negris
January 10, 2019 AI & Machine Learning

Ready to learn Artificial Intelligence? Browse courses like  Uncertain Knowledge and Reasoning in Artificial Intelligence developed by industry thought leaders and Experfy in Harvard Innovation Lab.

According to Harvard Business Review, most current AI projects will fail. And yet, many customers win with AI.

They succeed because they are not AI newbies. They have already valiantly tried and sadly failed in early forays into machine learning and AI software applications. They gathered all kinds of data, hired numerous data scientists, secured executive commitment and ample budget, and engaged in lengthy projects only to produce little or no business value. And these are all huge companies with all the resources needed to succeed on their first try. And yet, they didn’t.

Despite being in differing industries and having widely differing use cases, our customers’ AI comebacks have many things in common, and here are the four biggest lessons they have learned for avoiding another AI train wreck. If your organization is just getting into AI, these lessons will help you win your freshman season.

1.      AI is not IT

Problem Seen – Many companies reflexively, and mistakenly look to the IT organization to initiate and manage AI projects, as they routinely do with other software-related endeavors. But AI software is not like database, middleware, and enterprise applications software, IT’s traditional technical domain. Enterprise software evolves through a series of measured, well-defined, carefully managed steps and narrow functional roles, with a mission of assuring operational continuity. AI software requires collaborative exploration and iteration by interdisciplinary, strategically managed groups with broad business, operational, and technical skills, and a mission of business transformation.

Lesson Learned – IT has two critical roles that bookend the successful AI projects. Initially, IT database skills are needed to acquire, integrate, and govern the information assets needed for machine learning, and finally, DevOps skills are essential to the effective, reliable deployment of new AI applications.

2.      Data Lake or Data Swamp?

Problem Seen – A common mistake made by many companies in their first AI project is to focus on data first, prediction second, and business problems third.  They fill a data lake to the brim with diverse information from many unrelated sources and then try to discover how AI might be able to produce business value from that data. In addition to being inefficient and ineffective, for companies using personal data from EU citizens and residents, this is also illegal. The EU General Data Protection Regulation (GDPR) has outlawed the indefinite retention and speculative use of personal data, turning many sparkling data lakes in companies around the world into perilous data swamps.

Lesson Learned – AI projects should first define a measurable business problem, like the cost of customer attrition. Then, they should identify, qualify, and certify relevant customer-touching business process data and regulation-compliant customer data. And finally, they should specify and model an actionable predictive decision that can solve the given business problem using the defined business and customer data.

3.      Start at the Glass

Problem Seen – In many AI projects, the people who will use the new application have little or no visibility or input into the solution until it is implemented and deployed. As a result, the AI team builds the application without the benefit of the data and process expertise possessed by the end-users; the application they build is usually disruptive – and not in a good way – to established business processes and best practices; and user acceptance of the application is thus low and slow. The lack of user involvement in defining and developing AI solutions results in projects that are longer, riskier, costlier, and less successful than those where end-users are involved from the start.

Lesson Learned – AI doesn’t take jobs; it takes tasks. And successful, profitable initial AI solutions should focus on automating low value business process steps that require high human effort. To ensure that the new AI solution will do that effectively, end-users must be part of the development team as co-equal partners to the data scientists, software engineers, and IT specialists on the team. Their experience and knowledge are essential to aligning the solution with the business problem and building an effective solution.

4.      Prepare to Evolve

Problem Seen – Most initial AI solutions solve yesterday’s problems. That is, by the time the solution is delivered, related business conditions and priorities have changed, creating a gap between solution function and business need. AI, just like human intelligence, is dynamic and it is never finished. The best that can be hoped for is incremental improvements in predictive accuracy and workflow efficiency. And yet, many organizations’ approach to AI solutions is the same as it is for static enterprise applications like accounting or asset management, which provide fixed functionality and are infrequently modified. Many executive AI newbies are dismayed to hear that a predictive model is correct three out of every four times, not realizing that is actually a great result in many cases because it is way better than a worker being right one in four times, and taking much longer than the algorithmic prediction.

Lesson Learned – Successful AI projects depend on agility and flexibility in goal setting, resource allocation, and ongoing optimization. Getting from 75% accuracy to 95% requires continuous improvements in data quality and availability, choice of machine learning tools and techniques, and target applications goals and functionality. And, predictive models can be brittle; they might work well today but, due to changing goals and business conditions, not work at all tomorrow. They must be continuously tested and maintained. So, in AI, the only constant is change; AI projects with fixed timelines, skills, and goals will certainly underperform and very likely fail altogether.

To avoid an AI train wreck in your first project and beyond, study these problems seen and lessons learned by others who have already tried, failed, learned, and then succeeded. And don’t be rigid, don’t be afraid, and, most importantly, don’t believe all the AI hype. Instead, be practical, be realistic, and be patient.

  • Experfy Insights

    Top articles, research, podcasts, webinars and more delivered to you monthly.

  • Tim Negris

    Tags
    Artificial Intelligence
    © 2021, Experfy Inc. All rights reserved.
    Leave a Comment
    Next Post
    What is Search Analytics and Can it Improve Self-Serve Data Discovery?

    What is Search Analytics and Can it Improve Self-Serve Data Discovery?

    Leave a Reply Cancel reply

    Your email address will not be published. Required fields are marked *

    More in AI & Machine Learning
    AI & Machine Learning,Future of Work
    AI’s Role in the Future of Work

    Artificial intelligence is shaping the future of work around the world in virtually every field. The role AI will play in employment in the years ahead is dynamic and collaborative. Rather than eliminating jobs altogether, AI will augment the capabilities and resources of employees and businesses, allowing them to do more with less. In more

    5 MINUTES READ Continue Reading »
    AI & Machine Learning
    How Can AI Help Improve Legal Services Delivery?

    Everybody is discussing Artificial Intelligence (AI) and machine learning, and some legal professionals are already leveraging these technological capabilities.  AI is not the future expectation; it is the present reality.  Aside from law, AI is widely used in various fields such as transportation and manufacturing, education, employment, defense, health care, business intelligence, robotics, and so

    5 MINUTES READ Continue Reading »
    AI & Machine Learning
    5 AI Applications Changing the Energy Industry

    The energy industry faces some significant challenges, but AI applications could help. Increasing demand, population expansion, and climate change necessitate creative solutions that could fundamentally alter how businesses generate and utilize electricity. Industry researchers looking for ways to solve these problems have turned to data and new data-processing technology. Artificial intelligence, in particular — and

    3 MINUTES READ Continue Reading »

    About Us

    Incubated in Harvard Innovation Lab, Experfy specializes in pipelining and deploying the world's best AI and engineering talent at breakneck speed, with exceptional focus on quality and compliance. Enterprises and governments also leverage our award-winning SaaS platform to build their own customized future of work solutions such as talent clouds.

    Join Us At

    Contact Us

    1700 West Park Drive, Suite 190
    Westborough, MA 01581

    Email: support@experfy.com

    Toll Free: (844) EXPERFY or
    (844) 397-3739

    © 2025, Experfy Inc. All rights reserved.