Domain Expertise vs Machine Learning Debate

When I’m getting ready to reason with a man, I spend one-third of my time thinking about myself and what I am going to say and two-thirds thinking about him and what he is going to say. Abraham Lincoln (1809-1865)

If human beings had to reason with a machine, or more specifically, had to teach machines to reasonis Lincoln’s formula still relevant?

Framing the debate on the superiority of machine learning vs. domain expertise

This debate has gone on for quite some time now. It was even rehearsed by several luminaries of the data science world at a Strata conference in 2012.

With the rapid progress in the field of artificial intelligence, human beings are apprehensive about robots conquering human thinking! If one extends that argument to expert systems of data science, then one can easily see why some people would see machine learning supplanting domain knowledge.

But the real question lies not in exploring whether data scientists require domain knowledge to build expert systems, but whether the representation phase of data can be accurately achieved without involving domain experts. Domain experts are presumed to be far more capable of identifying, articulating, and demonstrating day-to-day process problems in business. As these experts can jolly well explain a research problem to peers, it is probably absurd to even consider that an expert system can be constructed without their involvement or guidance. The same should hold true in the case of superior algorithms required to create such systems.

Let us take the example of creating a machine learning system that can distinguish spam messages from non-spam messages based on the science of algorithmic filters. In this case, the email users, including the data scientists themselves, are domain experts, since they happen to use the email system every day. To build an expert anti-spam application, developers would be expected to work closely with the user community to understand their spam-related problems. Here the developers themselves may be a sub-domain expert within the broad category of broad mail-domain experts (users).

Thus, we realize that the fundamental strategy behind designing an expert system lies in identifying patterns in usage data; and then deriving general rules or principles from those patterns. This principle of generalization points to the ability of an expert system to perform new, tasks after having experienced a learning data set.

So here we establish the first criterion for performance: Performing tasks based on prior experience with data. We may even conclude that machine learning operates from the standpoint of predictionbased on known properties discovered from the training data.

Domain experts usually have an in-depth knowledge of operational processes or tasks and also understand the rules of thumb that control the domain. Domain knowledge is gained from actual, practical experience. In reality, data scientists attempt to convert this practical knowledge into meaningful algorithms to automate processing tasks.

Thus data scientists and domain experts are the two complimentary sides of a complete system development project. In developing expert systems, it is not enough for data scientists to ask questions or find patternsthey also need to understand the results.

An example from the field of Molecular Biology

Machine learning may be necessary to research the human cell, but it cannot be the starting point. Therefore, data scientists must collaborate with molecular biologists to understand the complexity of cell behavior under different conditions to analyze or process the findings of one’s research. The biologist is equipped with a priori knowledge, known as domain expertise that provides accurate insights for monitoring and analyzing the results of an experiment or a series of experiments.

In ideal situations, machine learning will be utilized not just to predict the behavior of complex processes or organisms but also to harness the power of machine learning to help intellectual communities understand the reasons behind the behavior.

The flip side of the argument: Is domain knowledge really necessary?

An example of a competition in a crowd-sourced environment

In a Kaggle competition, a panel of space agencies developed a competition writing algorithms for studying the impact of darkness on images of space. The winner happened to be a student of glaciology, thus proving domain knowledge was inconsequential in winning this competition.

This competition routinely raises this question: if data scientists can develop such fine algorithms in any discipline or business field, then who needs domain experts?

If now we compare the two opposite viewpoints presented above, the general consensus is likely to sway in favor of a compromisehitting the middle ground between machine learning and domain expertise.

The domain experts still play a critical role in helping data scientists understand and articulate the business or process problem and help understand the results. As a case in point, an economist may create the best algorithm to automatically grade SAT answer papers, but the education experts must be involved in designing the grading criteria and sample questions and answers. These educators are the only people proficient in interpreting the output.

Let’s compare the relative strengths and weaknesses of domain experts and data scientists.

Machine Learning Experts/Data Scientists: Pros

Can ask questions without understanding processes or tasks
Can study data to discover repetitive patterns
Can reconstruct process knowledge by studying data
Can use data patterns to predict results

Machine Learning Experts/Data Scientists: Cons

Cannot analyze the existing models of business processes accurately
Have the potential to misuse models
Lack depth of understanding of business functions

Domain Experts: PROs

Can provide practical insights from past experience
Can help refine a question with practical knowledge
Can accurately shape or model tasks for analysis
Can guide analytics in the right direction
Can evaluate the effectiveness of a result

A domain expert’s strength lies in close observation of day-to-day process problems, while a data scientist’s strength is building generalized solutions in the form of algorithms by studying specific data patterns.

After reviewing the previous arguments, we can take a more balanced view that both the data scientists and domain experts need to collaborate or work in harmony to accurately solve business process problems. A final example helps show the need for collaboration.

An example: Learning Management Systems (LMS)

A critical first step in developing a learning system starts with gathering information about potential learner’s competency in specific skills or tasks. The objective behind collecting this information is to estimate the scope of the learning system in terms of topic or task coverage. Quite often, the approach is back-to-front; the learning outcomes and performance tests are defined before creating the learning content.

While trying to create an automated system that claims to help the learner achieve the desirable performance objectives, the machine learning experts have to collaborate with education experts, subject-area experts, and learning and development experts to design and create an effective, automated learning system.This example helps us see the value of cross-domain experts in developing learning systems.