Ready to learn Data Analytics? Browse Data Analyst Training and Certification courses developed by industry thought leaders and Experfy in Harvard Innovation Lab.
“Insight” has become the ubiquitous catalyst for many business decisions. We analyze data to gain insights and use those insights to guide decisions, large and small. It is better than “instinct” or “intuition” as a decision enabler, but is it the best we can do? Or is it an actual impediment to effective Digital Transformation?
Think about it. Big Data and advanced analytics are digital, as are business applications and systems, but the decisions in the middle of it all are analog – humans interpreting information and telling machines what to do about it. Digital Transformation has made great strides at the operational level, but it won’t happen at the management level without digital decision-making.
Digital decisions require artificial intelligence; and AI neither needs nor produces insight. AI analyzes the data and makes the decision. The days of business insight are numbered. Let’s see how that could be.
In their article Unleashing Hidden Insights, published on the American Marketing Association website, authors Marco Vriens and Rogier Verhulst provide a serviceable definition of “Business Insight”.
“A thought, fact, combination of facts, data and/or analysis of data that induces meaning and furthers understanding of a situation or issue that has the potential of benefiting the business or re-directing the thinking about that situation or issue which then in turn has the potential of benefiting the business.”
For two decades, such business insights have been gained through spreadsheets, dashboards, and other analytical tools, under the rubric of “Business Intelligence” (BI). For the most part, the analytical computations underlying BI tools/platforms have been based on statistical methods which have been in use for many decades.
Now, here comes Artificial Intelligence (AI), a newer form of advanced analytics that uses machine learning algorithms, rather than statistical formulas, to process data. (You can learn more about Statistics vs Machine Learning here.) And the primary goal of AI is not to enable human insight, but rather it is to enable digital decisions.
BI and AI have orthogonal missions, bound only by the murky shared heading of “advanced analytics”, but nonetheless many BI software vendors are now grafting machine learning into their offerings and talking a lot about things like “AI-driven insights”. Why is this unnatural mashup happening?
Let’s call it “BI-agra Syndrome”. AI promises to turn BI professionals into sexy data scientists and BI products into sexy solutions to all your insight needs.
In this Google Trends chart, which line is the sexy one?
Holding aside the influence of SEO pixie dust, though, there is a deeper trend at work here, one that may herald the coming demise of human management decision-making and the business insight that enables it.
Despite the empirical veneer of dashboards and spreadsheets, many business processes still depend on heuristic choices made by human experts, as they have for centuries. The data may be digital now, but the decisions are still very analog.
This is a status quo with many defenders, mostly in middle management, where most of those analog decisions are made. Much of the “AI Eats Jobs” press concentrates on task workers like service agents and mechanics, or on knowledge workers like lawyers and accountants, and ignores middle management – the “decision workers”. But a recent executive survey by Marketforce said this.
“According to our research, 72 percent [of senior executives] think the increasing use of AI and robotics will dramatically reduce the number of middle managers in most organisations over the next decade. While this is bad news for middle management, it’s good news for those below them: almost eight out of ten (78 percent) believe support from artificial intelligence will allow workers to make informed decisions at a more junior level, leading to a flattening of traditional hierarchies.”
It doesn’t say how those junior workers will make those decisions, but you can bet it won’t be by gaining business insight from BI tools; it will be by taking guidance from AI applications powered by predictive analytics, as suggested by this Google Trends chart. Look familiar?
Digital decisions require foresight, not insight. In the current golden age of BI (pick your ‘I’), it seems like a great innovation with an unlimited future of dazzling data visualization and discovery, the Big Data Rosetta Stone. But maybe it is just the BI Bubble. BI has been around in some form for as long as there have been computers – mainframe decision support systems, PC spreadsheets, web GUIs with data lakes, and cloud Big Data services – always doing the same thing: statistical data analysis for human (analog) decision making. But a long history doesn’t mean a long future.
Bad human decisions are one of the greatest, most persistent causes of expense, risk, and disruption in business today. Better BI can help people make better decisions, but human decision capacity and accuracy are ultimately limited, and the BI tools may already be providing more insight than humans can assimilate.
By comparison, AI programs make far fewer, and generally much smaller decision mistakes than humans. In fact, with enough good data, they almost never make decision mistakes. Plus, AI decision capacity is virtually unlimited, and all it needs to make more accurate decisions is more, better data.
AI-infused BI tools may enable better decisions, but they are still analog, insight-driven decisions, not prediction-driven digital ones. That is refinement, not transformation.
Effective Digital Transformation starts at the middle, with AI-driven digital decisions replacing analog insight-driven ones across the enterprise – in Finance, Marketing, Planning, Human Resources, and other places where key processes currently depend on human perception and action. Although it may reduce their numbers, this does not remove humans from the process, but rather it transforms them too, turning analog decision makers into digital decision managers.
