What if you could use data to predict what a customer will buy, one year before they even know they want it? Automating through machine learning (ML) allowed Amazon to predict future demand for millions of products globally in seconds. Leaders at the multinational tech giant successfully reinvented their data infrastructure to improve buying systems, automate the placement of inventory in fulfilment centres, and deliver on their promise of two-day shipping to customers.

Through a comprehensive predictive model built entirely on the cloud, Amazon is using data to make better decisions, streamline operations, and deliver winning consumer experiences.

Reinventing manual product forecasting through machine learning

Predicting customer demand is no easy task in ecommerce since delayed inventory or inaccurate shipments can be costly and disrupt the supply chain. Although 80% to 90% of all planning tasks can be automated, many industries still rely on manual forecasting.

Ecommerce retailers sometimes need to forecast hundreds of millions of products, and “no amount of human brain power can forecast at that scale daily,” says Jenny Freshwater, vice president of Traffic & Marketing Technology at Amazon, and former VP of Forecasting. Freshwater’s team led the forecasting of over 400 million products at Amazon.

Engineering teams, no matter how advanced, can’t do it all: assess historical trends, develop unit sales projections, and conduct independent research for such a high volume of products. Even when combined with more sophisticated models, legacy systems, like outdated computing software or manual inventory logs, won’t be as accurate as machine learning models.

How Amazon stepped up in product forecasting

Amazon’s journey with machine learning began about 10 years ago to improve forecast accuracy. “We started to use machine learning because our moving average models were just not as accurate as we had hoped they would be.”

Company leaders saw a need to use data and machine learning to deliver on customer promises and achieve cost-effective functionality at scale. With those goals in mind, Amazon set out to become an AI-driven leader in product forecasting.

To accelerate the process in the face of rising demand, the company built machine learning models that grew in terms of the magnitude of data, the features used to predict demand, as well as the complexity of the algorithms, to where today, they’re using neural network models to predict demand for the products they sell on Amazon. They looked at how our human forecasts were performing and how our machine-learning forecasts were performing. And it was night and day in terms of the difference.

Amazon uses machine learning on AWS to aggregate and analyse purchasing data on products and run their forecasting models. Additionally, the company uses browsing and purchasing data to provide more tailored product recommendations. Machine learning allows for data experimentation that enables data scientists to create a better and more personalised experience for customers.

This does however illustrate the advantage the ecommerce retailer has over Amazon, in that deep learning can only be achieved with hyper-personalised data with direct access to the consumers themselves. Knowing historical purchases only offers one dimension. If you know each consumer, and what they’re also looking at you get the second dimension, and as all good statisticians know, once you then also introduce idiosyncratic data, in terms of usage and what they’re looking at most, you are in by far the best position from which to predict imminent purchases.

For Amazon prototyping and iteration were key to achieving machine learning success. Using a prototyping approach, looking at specific use cases, measuring the results against existing models, and, at a certain point, able to achieve a 15 times greater improvement than ever achieved before, through these neural network models,” Freshwater says. “So, it was very much an iterative process.

Key takeaways for business leaders using predictive models

ML on the cloud is key to deriving valuable insights from data and making better business decisions. Consider these practices to maximize ML modelling in your reinvention journey.

1. Trust the model. Regular monitoring of millions of products takes up valuable engineering time and resources. Nearly all of Amazon’s forecasting is automated through machine learning models, and human beings and business users only interact with the forecast and override it when they have some information that the models couldn’t possibly have. Leaders should encourage teams to selectively interact with the forecast, letting the models work while business users focus on other critical tasks. Overrides should be considered when you are confident you have more qualitative or trend-focused information than the model.
   
   
2. Define a clear data strategy. It’s impossible to think of machine learning – and getting real value out of ML models – without first having a data strategy in place. At Amazon, preparing the data for ML use was a key part of the strategy. Features are all about getting the data in the right place. Without really spending that time and effort, they would risk either getting poor results or biased results because the data didn’t properly represent our decision set. Building a data strategy that aligns with business goals, prioritising data cleansing, and making data representative of what you’re trying to predict or optimise is key for leaders starting their ML journey.
   
   
3. Know what you’re measuring. Early on in your reinvention journey, it’s important to be crisp about what you’re measuring and how you know you’ve improved. In Amazon’s case, the problem is that they were forecasting more and more products every year. So just by looking at year-over-year comparisons, it wasn’t good enough. They had to implement a series of benchmarking models that used completely different scientific methodologies to know if, versus the benchmark, their new models were better. Spend time upfront defining your measurement and success criteria to prevent time wasted on churn.
   
   
4. Build a data-driven culture. Getting the most out of your machine learning initiatives requires nurturing a culture of innovation and data-driven thinking across the organisation. Reinvention projects are about being able to take risks and being able to fail fast Pick people who are OK with several failures before a success. I think that culture needs to be encouraged when you’re working on any prototype – because the first results aren’t always a winner. When experimenting with machine learning, make sure you have people at all levels of the organisation who are passionate about the prospect of what ML can deliver. Given that Amazon is such a data-driven culture, they were able to move the needle almost entirely towards machine learning, just by looking at the fact that our models were much more accurate from a forecasting accuracy perspective.
   

Amazon has paved the way for predicting the future, but they’re just getting started. Your organization too can reinvent itself using AI technologies and data. Learn more about AI ML hyper-personalisation here today for an immediate hike in your success.