Around March of 2013, The Economist Intelligence Unit conducted a poll to acquire and analyze realistic views on the role of data in organizational decision making. 212 business executives from enterprises with revenues exceeding US$1 Billion responded; and the results were presented in a white paper titled The evolving role of data in decision making.
This poll uncovered valuable insights about the growing importance of data science in enterprise decision making. As both traditional and non-traditional data sources continue to flood business operations, these businesses find the gap between the piles of data and their meaningful use beyond control.
Most market reports on data analytics confirm that judicious use of data in any form has helped improve the business operations. Now with the variety of data sources and types growing every day, the potential for deriving valuable insights can only increase. However, to mine and extract insights from high volumes of data, organizations need the appropriate technologies and skilled professionals.
Lets take the example of GE, where continuous streams of real-time, sensor data from jet engines pour in to report how the running engines are performing. Till the time GE started appropriately storing and analyzing these data, a systematic method for uncovering patterns of erroneous or malfunctioning events on running engines was not present. Now, industrial internet has made it possible for GE to acquire operational data in real time to improve such operations.
Alice in Wonderland: Inability to ask the right questions
Christopher Frank, vice president of a financial services firm, and the co-author of Drinking from the Fire Hose: Making Smarter Decisions without Drowning in Data, thinks that the basic limitation of big-data analytics lies in the inability to ask the right questions. Mr Frank has termed this phenomenon the Alice in Wonderland problem. Just as Alice in the classic childrens novel admitted she did not know where she wanted to go, big data analysts in todays data-deluded world do not know what to ask about their business data! Unless business executives learn to ask the right questions, no amount of technological advancement will improve the current state of data analytics. As the following graph show, a lack of talent is a primary factor hindering data-driven decision making in enterprises today.
This is where the popular argument that software tools will soon replace data scientists fails. Data scientists, with their right balance of quantitative, IT, and business skill sets will continue to provide a value-added service to enterprises. Data science prepares professionals to ask the right questions, and extract relevant data from huge masses of multi-structured data.
How can data scientists help?
Data scientists offer a unique combination of data collection and manipulation skills including modeling or creating smart algorithms from studied patterns, to enable advanced analytics on any amount or type of data. In fact, all ambitious enterprises have either already invested or looking to invest in well-equipped data centers and skilled data scientists.
Automation of business functions requires both better tools and more skilled manpower. Better tools will increasingly need thinkers (data scientists) to drive tools to better decisions making.
Missing Expertise in Data Collection and Collation
Some common obstacles to successful, data-based decision-making include inadequate tools for collecting or analyzing data, inaccurate or irrelevant data, and lack of expertise in using that data for analytics-driven decision making.
How can data scientists help?
This kind of problem has been addressed in current literature on data analytics. The marker-research reports indicate that organizations have to work towards a data-driven culture, where collection and storage of useful information becomes an integral part of the business processes. Second, these organizations must acquire or develop expertise both in data science and in their internal operationsin order to maximize the returns from their data analytics efforts. Third, enterprises must consciously apply the combined knowledge derived from insights and domain expertise to actual business decisions.
Overcoming Obstacles: Data Science in Action
According to The McKinsey Global Institutes 2011 report, the term big data refers to data sets whose size is beyond the ability of typical database software tools to capture, store, manage and analyze. The biggest challenges facing enterprises in harvesting and analyzing big data are the wide variety of data sources and types; the amount and speed of delivery of data, often causing data deluge; lack of accurate, timely, or relevant data; and the lack of proper reporting of operational data.
More and more, enterprises are using sophisticated BI such as data visualizations, predictive analysis, sentiment analysis, or collaborative analysis to make strategic decisions. Also, the mentioned poll results revealed non-C-level-executives possess either primitive or basic (68%) analytics skills. The majority of senior executives who completed the survey represented companies with revenues of US$5M or higher; and they understood that with an acute shortage of data-analytics professionals, enterprises will struggle to conduct proper analysis, and may not (even) be able to make decisions in a timely manner.
Another interesting observation made by this poll was that businesses which were making timely decisions with big data also rated their analytics skills more highly. This proves that organizations lacking the right tools and resources make slower decisions.
How can data scientists help?
In Understanding the DNA of Data Science, author Booz Allen Hamilton takes real-life case studies to explain how data scientists use data functional or operational data to find decision-based solutions to business problems. To derive answers from decision-based systems, data scientists have to build expert models from the available data.
The Economist Intelligence Unit poll results confirmed that 28% said they already have big-data talent, 44% say they plan to hire more, 65% say they do not have the requisite talent but plan to hire over the next 12 months or later. These trends make it a sellers market for data scientists.
Data Science in Interpreting Visuals
Interestingly, poll respondents who admitted success with data analytics reported they tend to use visualization tools more frequently. This result indicates that visualization tools facilitate easy comprehension of questions that they should be asking of their tools. Christopher Frank, co-author of Drinking from the Fire Hose: Making Smarter Decisions without Drowning in Data says that in an effective analytical culture, users do not want data analytics to give answers so much as to set parameters, winnowing down a large number of variables to find the few that really matter.
