If you are a big retail chain, you must be faced now with a pressing issue: you have thousands of stores with dispersed, siloed data and can’t see clearly what is actually going on. For example, you may overlook that a certain product sells once in six months yet calls for effort and money to be maintained and kept on the shelf. Or you’ve failed to predict that another product was going to be out of stock soon – and now unhappy customers are buying it from your competition.
Big Data Analytics and Machine Learning help to address these problems. In fact, their results plus common sense can work wonders in almost any area of retail business.
That’s why many businesses are considering adopting Machine Learning in retail.
To get deeper into it, we’re going to shed some light on whether you need it, the key aspects it can improve, and how to implement it.
KEY PROBLEMS MACHINE LEARNING CAN HELP TO SOLVE IN RETAIL
Demand prediction and stock optimization
As Maksym Nechepurenko, a Senior Data Scientist at N-iX, points out, most retail companies are now looking to solve a common problem – predict demand and ensure their products are always in stock. And that’s a vital issue to address. For example, when I want to buy champagne before the New Year’s Eve at a store in my neighborhood and I can’t see it on the shelves, 1) I won’t be happy; 2) I will go to a competitor’s store and buy it there; 3) probably, I won’t shop in that store for a while.
Demand prediction is a complex task. Even a simple store has 10,000+ different products, and the buying trends for bananas and microwave ovens, for example, will be different. Some products sell 100 per day, some – once a week. You can’t build a universal model that will work the same way for all products.
However, this is one of the critical tasks. Solving it helps to reduce shrinkage and wasted products on one hand and lost opportunity – on the other hand. For example, if a grocery store sells 30 kg of bananas per 3 days but we didn’t know that beforehand and bought from a supplier just 20 kg – that’s a lost opportunity. Thus, it’s crucial to be able to predict the demand and be geared to satisfy it.
In fact, it’s better to have more in stock than needed than to lose an opportunity and potential clients. Therefore, when building models for demand prediction, Data Scientists usually predict a slightly bigger demand to be on the safe side and not to risk experiencing the shortage of certain products and losing customers.
Gross profit optimization (price/demand balance)
Price formation and designing demand is another important issue to be addressed by retailers. It’s a common truth that we can create the desired demand by setting a certain price. However, finding the right price/demand balance to maximize your profits is also a common pain.
Factors such as competition, market positioning, production costs, distribution costs, the period of the year, the current state of the market and others should be taken into account. Big Data and Machine Learning can help you find the right balance and make the right move. And instead of using, for instance, general markdowns, you as a retail company can benefit from predictive models that will use time-tested techniques such a statistical analysis to determine the most suitable price for each specific product or service.
Solving logistics issues
Machine learning enables retail companies to spot patterns in supply chain data by quickly pinpointing the most important factors to the supply networks’ success and routing optimization, while constantly learning in the process. Also, it can be efficiently used to optimize warehousing (as champagne is better to be stored in warehouses before the New Year for a while, and bananas, for example, are better to be delivered to the store points as soon as possible). The same concerns thousands of other different products. And you’d better rely on algorithms in this aspect.
Merchandising optimization with visual search
Merchandising is a task that requires much time and effort and may be prone to carelessness if people are to do it. That’s why many companies start delegating this job to code.
For example, you can monitor merchandising in stores using cameras. For example, OSA Hybrid Platform partnered with Neuromation to help retail companies control merchandising and customer satisfaction.
Personalized offers
Personalized offers improve the customer experience by offering relevant information which in turn provides retailers with improved data about the customer’s brand engagement.
For example, one of our clients, a next-generation one-to-one CRM marketing automation platform leverages NLP and machine learning to deliver more engaging e-mail content to consumers. The solution helps retailers, e-commerce, and travel brands to increase their conversion rates and enhance their CRM activities.
Another example: outdoor product company The North Face has built its own virtual personal shopper using the IBM Watson platform. The service uses customers’ vocal queries, shopping needs, and travel plans as input and recommends items that not only meet customers’ criteria but are also suitable for the location where the customer plans to use them—even taking into account the weather forecast.
Fraud detection
Machine learning helps data scientists efficiently determine which transactions are the most likely to be fraudulent. The techniques are extremely effective in fraud prevention and detection, as they allow for the automated discovery of patterns across large volumes of real-time transactions.
Churn prediction
When talking about Data Science and churn prediction, it makes sense regarding products that are daily used as you can track the activity and customer behavior better, and you’ve got enough data to see the trends. However, you need to understand that there may be something you can’t see regarding an individual customer’s behavior, even leveraging their buying history and ML models. Thus, both your predictions and effort to retain the customer may prove to be ineffective.
Selecting location
Another important task to solve by retailers is choosing where to build a new store and finding the best location for a specific type of products. It should take into account such factors as demographics, the closest competitors, the number of population in the neighborhood, etc., and once again machine learning can come in handy.
Sentiment analysis
Many companies use NLP techniques and sentiment analysis to monitor and track customer reviews and customer satisfaction. In retail, you can also use it to analyze a certain brand customer reviews to see if it’s a good idea to cooperate with a supplier and sell a specific product.
Document work automation
Using NLP and automated processing of documents and agreements goes a long way towards simplifying and speeding up paperwork in many industries. For example, manual review of 12,000 annual commercial credit agreements would typically take up around 360,000 labor hours. Whereas, machine learning allows reviewing the same number of contracts in just a few hours. Retail industry doesn’t make an exception here, and many retail companies are already using it to automate their work with multiple suppliers and processing of customer claims.
HOW TO IMPLEMENT MACHINE LEARNING IN RETAIL
Big Data engineering is a foundation
Like in any other domain, the biggest part of any data science project comes down to building an orchestrated ecosystem of platforms that collect siloed data from hundreds of sources like CRM, reporting software, spreadsheets, Excel tables, and more.
Most businesses, including retail companies, don’t need complex prediction models or any Data Science whatsoever. In many cases, businesses try to apply DS models even if they don’t bring any actionable results or when they reveal the truth that is already known to everybody. However, what is the most crucial part is to collect the data from dispersed sources and visualize it to actually see what’s going on, what needs fixing and more effort, and what needs to change.
And that’s actually the hardest part. Before using any ML models, you need to have the data structured and cleaned up. Only then can you further turn that data into insights. In fact, ETL (extracting, transforming, and loading) and further cleaning of the data account for around 70-80% of the machine learning project’s time.
If you’ve got several thousand stores and they all have their own systems, CRM, and other data sources, it’s a tall order to integrate and aggregate those terabytes of siloed data. That’s especially a problem if retail companies have legacy IT systems. And it becomes even a bigger headache with GDPR as now you can’t store the users’ data anywhere.
That’s why DataOps that brings together DevOps teams with data engineer and data scientist roles and provide the tools, processes and organizational structures to support the data-focused companies, is becoming so popular nowadays.
Applying Machine Learning models
Most machine learning projects deal with issues that have already been addressed. Such companies as Google, Microsoft, Amazon, Facebook, and IBM sell machine learning software as a service. For instance, N-iX Data Scientists are using Prophet, an open-source tool by Facebook, for Time Series Forecasting. Google offers a wide range of effective plug-and-play recommendation systems.
To apply these services, you need a machine learning engineer who can implement the system focusing on your specific data and business domain. The specialist needs to extract the data from different sources, transform it to fit for this particular system, receive the results, and visualize the findings.
SUMMARY
Let’s face the truth, you can’t make absolutely precise predictions and can’t be 100% sure that a fact correlates to another fact not in a random way as there is often something that we can’t see, especially regarding human behavior, and our actions can be caused by a million factors. However, what you need and can do is to have a vivid picture of what is going on in your business, as the more info you have at hand, the more clearly you can see that something is going wrong and needs fixing.
Data Science and Big Data analytics is not a magic pill that can solve all your problems. However, it’s a strong competitive advantage as it gives you knowledge and a better sense of control.
Co-authored by Maksym Nechepurenko
