Ready to learn Machine Learning? Browse courses like Machine Learning Foundations: Supervised Learning developed by industry thought leaders and Experfy in Harvard Innovation Lab.
In recent years, it has become quite evident that AI-based technology will fundamentally change economies, politics, the planet, and indeed humanity. Even today we are only just beginning to see some of these changes come to fruition. For better or for worse, society will be permanently altered due to artificial intelligence. Just think of the dramatic changes we’ve witnessed just in our own lives as the age of the Internet has disrupted the landscape.
Given the dramatic pace of innovation today, one can’t help but wonder what humanity might look like in a few decades as compared to today. How will we, as a society, fare in the brave new world of tomorrow?
Humans have been interested in predicting the future for thousands of years. Fortune tellers were highly coveted advisors to Assyrian kings as far back as 2500 BC and astrology has been dated to at least 2000 BC. Ancient cultures ranging from the Mayans to the Chinese ascribed special meaning to the movement of celestial bodies. Prophets in the Bible foresaw terrible events like wars, floods, earthquakes. Nostradamus, a 16th-century astrologer, wrote his collection of prophecies, Les Propheties, in 1555.
Even today we long to know the future. Only today, our fortune tellers are software engineers and statisticians. We have turned to artificial intelligence, machine, and deep learning to help us predict everything from election outcomes to weather patterns and even crime.
It is estimated that 2.5 quintillion bytes of data are generated each and every day! In accordance with Moore’s Law, this daily data generation is likely to double every 12-18 months. I believe that data will actually outpace Moore’s Law over the next decade, particularly as more remote parts of the planet join the internet age. All of this data makes it increasingly difficult to identify a signal from all this noise. And yet, machine learning thrives on oceans of data. In many ways, the explosion of data over the past few years has been the most important and direct cause of the explosion in artificial intelligence.
Every day, whether we know it or not, we make calculated decisions with varying degrees of accuracy in everyday life decisions. Is the benefit of me arriving at my destination worth the risk of being hit by a car when I cross the street here? Should I quit my nice stable job and found a startup?
Virtually every decision we make can be formalized into a problem to be solved with predictive analytics. Games such as poker and baseball are well-studied in this respect. Statistics and predictive analytics are being increasingly applied to other domains such as politics and social science. The 2016 American presidential election was the most data-rich elections in history. But despite the enormous amounts of data available, polls and statisticians seemed to get it wrong.
Early polls had the odds of a Clinton victory hovering in the 70-80% probability. However, closer to the election date, the picture was a little different. Although the majority of political experts still expected a Hillary Clinton victory, by November 7th, the polls had tightened significantly.
It’s not yet clear why, despite all of the data available, pollsters still got it wrong. To be certain, predicting chess and poker outcomes are one thing — predicting political outcomes are quite another thing entirely. Regardless of the country in question, democratic elections are extremely complex systems that are not entirely fact-driven. Human emotion is a major component of a democracy and artificial intelligence has not yet achieved a good understanding of human emotion. At any rate, statisticians, researchers, and political experts will look back on the election of 2016 to try to understand the implicit biases and incorrect assumptions baked into their models. Since the time of the election, researchers have introduced algorithms that have been able to predict which congressional bills have the highest likelihood of passing and becoming law. Every day huge amounts of data are continually added to the ocean of data and everyday models become smarter and more accurate.
Generally speaking, it’s all about probability more than anything else. It’s been said that the only certainties in life are taxes and death. Insurance companies take this quote to heart and have been building predictive models for centuries. Life insurance is an interesting if morbid, case study. The question isn’t whether the insured individual will die or not — it’s about how long it’ll be before it happens! There are hundreds, if not thousands, of variables that are considered in the final premium price for life insurance, not least of which are the health factors. Insurance companies like writing contracts with less risky people (compared to the average) and dislike making contracts will more risky people and they invest heavily in producing highly accurate models. In the many years that life insurance has been around, it’s clear that the majority of these bets have paid off — the law of large numbers is a powerful thing!
The law of large numbers is a concept very familiar to Las Vegas casinos. Stated simply, the law of large numbers is the phenomenon that the average of the results of a large number of trials should be close to the expected value. For example, if you roll a die one million times, the law states that the average value will be 3.5. This simple theorem plays a central role in virtually all of statistics and has proven to be quite reliable for casinos and insurance companies alike.
Predicting Crime:
So predictive models are great for insurance companies and casinos, but what about the rest of us? Crime prediction in one such area where the fruits of predictive modeling can be shared by all. Imagine a world with no crime — no more violent assault, no robbery, no killing. Sound like a science fiction utopia? Researchers and police departments across the country are working hard to make this world a reality. Several police departments using algorithmic policing have reported anywhere from a 9% to 20% reduction in certain crimes. This is quite a way off from 100% but it certainly seems to be a step in the right direction.
One of the major ideas in predictive policing is to identify sparks before they turn into wildfires. Algorithms will consider minor crimes which have the potential to turn into more serious crimes down the road. For example, an escalation of violence between two rival gangs or simple assaults leading to more aggravated assaults. If police are able to identify and address minor crimes, they might be able to get the perpetrators the help they need before they consider more serious crimes.
It’s very important to ensure these predictive systems do not carry any bias. The ACLU has rightly raised concerns that predictive policing could have created a feedback loop that might actually reinforce crime. This is where an intelligent, thoughtful, and responsible debate must take place. It is my opinion that AI-assisted predictions will only continue to grow in popularity over the years and this is great — as long as we work tirelessly to ensure a fair and equitable system.
Epidemic Prediction:
Interestingly, many of the same principles in crime prediction are very closely related to disease and epidemiological models. Here too is another area where predictive modeling can have tremendous benefits for humankind. Cities have long been magnets for pests and rodents and these pests pose a serious public health crisis for cities. Recent research suggests that due to warming temperatures, rats are multiplying like we’ve never seen before in cities like Washington D.C. and New York City. This is a problem that is perfectly suited for predictive modeling solutions. Just like with crime prediction, we need to stamp out small problems before they turn into big problems.
The gross-factor is reason enough to address the pest problem. But large rodent populations can be extremely dangerous in epidemiological terms as well. Rodents have been known to carry and spread more than 35 diseases including the plague. The 1924 Los Angeles plague outbreak killed 30 people in just two weeks. A swift response by the Center for Disease Control and Prevention and a citywide rat extermination campaign is widely credited with the relatively contained outbreak.
Rodents can also present major ecological problems as well. In 2016, the New Zealand government announced an ambitious plan — to rid the country of all non-native rodents by 2050. Rats, stoats, and possums — all non-native, invasive animals — are blamed for the deaths of tens of millions of birds and cost the government more than $2.3 billion each year. In order for the New Zealand government to achieve this ambitious goal, predictive analytics and epidemiological models will be crucial.
Weather Prediction:
One final area where I believe predictive analytics will become more important is that of weather forecasting. We’ve seen incredible improvements in the accuracy of weather prediction in just the last 50-60 years. But when we’re talking about extremely dangerous weather like tornados and hurricanes, it’s important that we continue to improve accuracy. Weather prediction is the most complex problems I have discussed so far — there are thousands, maybe tens of thousands of variables that must be accounted for in building accurate weather models. And climate change is making that even harder. Scientists warn that warming oceans, sea level rise, and a rising global temperature could be the cause of the increased incidence of extreme weather events.
As the number of extreme weather events continues to rise and populations increase, weather events will be a major source of concern and it accurate weather predictions will be of utmost importance. Government agencies such as NASA and NOAA collect massive sets of data from sensors placed all over the planet.
In addition to weather, NASA tracks and monitors asteroids that are considered “potentially hazardous.” In 2005, Congress directed NASA to find and observe asteroids. NASA has found approximately 2,000 asteroids that could pose a serious threat to the planet. Were it to hit the planet, an asteroid 140 meters in diameter would ram into the planet with an energy of 288 megatons. For comparison, the largest nuclear weapon ever built had a payload of 50 megatons.
With an estimated 250 billion stars in the milky way alone, discovering an asteroid as small as 150 meters might sound completely impossible. And it would be, without the use of predictive analytics!
There is a great quote from Spider-Man — with great power comes great responsibility.Data is all around us. This data gives us incredible power and it is important that we use this data for good and not for bad. Data gives us the power to build extremely accurate predictive models which can be applied to problems that we as a species have dealt with since the beginning of time. We live in truly exciting times but I predict that the future will be even more exciting with the use of predictive analytics for applications worldwide!
