There is a trend happening right now in machine learning where subject matter expertise is being replaced. Approaches that previously required a subject expert now have naive approaches that are beating the world’s best experts.
What the smartest and brightest experts know, which was previously respected, in some cases offers minimal to no value now. Consider these examples:
Linguistics Experts:
One of the major problems that most people see when dealing with natural language processing problems (NLP) is the issue of sparsity. The words that are being picked up in the documents, social stream, or whatever feed you care about are too unique. If those words do not happen frequently enough, your modeling method will struggle to weigh their value appropriately. Consider an example where someone said “Happy” and in the next example someone said “Joy”. Both “Happy” and “Joy” are treated as completely different things by the computer when they should be associated somehow. Here, a language expert might offer a lot of value by building out a customer dictionary of word associations. These words (i.e. Joy, Happy, smiling, friendly) mean HAPPY, while other words are associated with ANGRY, etc… Historically, this type of expert assistance was valuable and worth paying for. However, taking the entire 40Gb dump of Wikipedia, you can use neural networks to automatically group common word associations. The final outcome will provide more groupings, more words, and more predictive value than the best human word association experts on the planet combined. A slightly different topic is also being seen in language translation. Neural networks capable of translating (i.e. English > German) are becoming so sophisticated that simply from supervised learning they can accurately capture noun genders and word reordering strategies that previously required a language expert for both languages.
Image Experts:
This next one tends to be controversial. It really shouldn’t be, but for some reason society seems to be resisting it. Doctors tend to be one of the most respected professions in the world for good reason. They take insane amounts of schooling, working terrible residency hours, and they can make the difference between you or a loved one surviving illness or injury. They also tend to be one of the highest paid professions out there.
Ok, so here comes reality. It is 2016, soon to be 2017. It is NOT 2001 anymore. We have entered a regime within deep-learning where image classification is now super-human, meaning it can beat a human expert head-to-head. Mankind has the technology right now to beat any radiologist or pathologist in disease diagnosis for thousands of diseases. [Quick note here: So I believe we have all of the technology elements demonstrated and ready but because of a lack of data accessibility due to HIPAA having an end-to-end solution has not been realized yet. We are not limited by technology here, but by the politics and constraints surround data access to reach appropriate volumes.]Think about that. Medical malpractice, human inconsistencies in performance and talent, the computer could diagnose you better than the world’s best experts when given enough data to train on. To extend it even further, I would say that hundreds of thousands of humans die unnecessarily every year because of our lack of big data and machine learning adoption within healthcare.
In the photo at the top, the respected expert reviewing the 3D image data should really get a LOL or facepalm from us. In 10-20 years from now I think it will. Why the hell were we thinking we could comprehend that information? At first you will see the argument for doctor augmentation, but eventually, it will lead to complete expert replacement. When my kid is having a brain scan, I would prefer the computer which has two orders of magnitude lower error rate than the best human expert in the world, and knows more disease outcomes than any human has ever seen in a career. Wouldn’t you?
Assessment Experts:
Have you ever had to take a long employment assessment? 100 questions? 200 questions? The Myers-Briggs personality assessment? These assessments have not existed to make your life miserable, there is actually a lot of science and human expertise going into most of them. They offer real validity on helping employers decide if they should select you. IO-Psychologists are experts when it comes to understanding job performance competencies, questions, and performance data gathering. I would recommend having them for any organization. Now consider just the assessment piece. Last year at HireVue we were able to demonstrate the same types of validity seen from a lengthy assessment in a 3 question video interview. The candidate experience was greatly improved because they didn’t have to go through a lengthy assessment, but the performance validity was there for the desired outcome. The questions required subject matter expertise from an IO, but eventually, just like the linguistics example above, big data sets will drive questions that split performance and competencies better than any human expert previously.
Threat Experts [Law inforcement]:
This one really deserves an entire post. Recently in the news we have seen the tragedy of unjust cop shootings and the backlash from that. In my opinion, this example above is one of the worst out there. A black therapist is shot trying to help his patient while holding his arms in the air. What happened with this particular threat assessment? Epic fail. I am grateful he survived.
It is interesting to watch the push and pull between #blacklivesmatter and #alllivesmatter protestors. I feel like the protestors are completely missing the point with this. The main point is that the “experts”, the humans that are trained to identify and deal with risk and crime, aren’t very good at it. Why?
Consider this scenario, coming to a future near you, someone calls in a suspect with a gun, the police respond, once they find the suspect they release a small threat assessment drone. The drone circles the subject from a distance to determine risk. Does he really have a gun? Does this person seem distressed? What is the overall threat assessment? Are there other people at risk? Are they willing to comply with the instructions? Do they speak english? Are they agitated like other suspects the drone networks have seen? Even with all of their training and experience, you cannot remove emotion from the human response. This drone, with today’s technology, will beat a human reviewing the same video footage on response time and accuracy. A big bonus here is the drone is not emotionally invested in its continued existence. Therefore, the suspect can shoot and terminate the drone, and the drone doesn’t care (yet… hahaha, j/k). The police, now out of harms way, are able to think logically instead of emotionally. No more rash decisions. I have the utmost respect for individuals that risk their lives every day trying to protect us.
Speech Experts:
Speech recognition has been something that humans have desired for decades. There are so many applications using dictation to write emails or take notes, voice-driven commands for a smart home or device, etc… The most familiar voice recognition systems right now might be Apple’s Siri or Amazon’s Echo. Having been involved myself with speech vendor selection and research, I have seen an industry being torn in two. On one side you have industry experts, they have attended the best colleges and programs in the world, and have spent the last 1-2 decades researching speech. On the other hand you have deep learning experts that really don’t care about what the first group has done. They only care more about improving their deep net architectures.
It seems the first group of “classical” experts resists the idea that their decades of research (they really are the smartest humans) could suddenly become worthless, I would too. Today, August 6th, 2016, if you were going to set out and build the most powerful speech recognition system the world has even seen, you would be a fool to hire a single speech expert. A little harsh? Maybe. In a year or two you may see I’m right. Instead, you would hire a young kid who was an expert in LSTM, CTC, CNN, ResNets, GPUs, and distributed computing. Besides deep-learning, the other talent I would consider on this speech dream team, would be signal processing experts that understand FFT, spectrograms, wavelets, etc.. no speech experience required.
General Conclusion (WHY?)
So what does the human expert lack? Why can the world’s best experts be beaten using subject naive methods? A human can’t weigh historical observations fairly, they put too much weight on experience A and not enough on experience B. A human is also limited to their own personal experience and wisdom, where a computer can learn from more data than a human can see in a lifetime (medical imaging being a great one). Lastly, a computer can overtake a human expert’s ability to experiment, do feature discovery, and validate new ideas.
