Business strategy & Data science – AI or not AI ?
To share our day-to-day experience with the design of smart decision-making business strategy solutions , we’ve initiated a new series of Karetis articles on the use of data science in Business Strategy problems.
Recently, we have been hearing a lot about AI and its potential use in Business. This article focuses on the use of AI in the design of accurate forecasts, and how the machine needs human expertise to produce accurate forecasts.
Building forecasts is one of the key challenges of business strategy.
Forecasting is a hard topic, because by essence it won’t be perfectly accurate. But not investing in it is a strategic bias, favoring the statu quo, ignoring signals, or dangerously relying on belief/ intuition only … In business, use cases are everywhere: from long range strategic planning (market trends, scenarios building,…) to short-term signal-based recommendations on operational tasks (eg: customer ordering, stocks, behaviours, next best messaging…).
There are different ways of building accurate forecasts (1). One could either assume what is going to happen in the future, build scenarios and forecast accordingly, or use models to evaluate the future outcomes. For this, today, either Machine Learning (ML) or traditional statistical methods can be used.
The M competition (2) that has been happening for 20 years, is a forecasting competition that focused attention on forecast models results, rather than on the mathematical properties of those models. This has challenged the status quo observed on many different methods. In 2000, the M3 competition results (3) stated that in general and for univariate time series (i.e. depending on only one variable) statistical methods were far more efficient that ML methods. Recently, the M4 competition (4) resulted in a different conclusion; when combined appropriately: mixed ML and statistical methods give the most promising results. Uber has been facing forecasting issues since its launch (5) , and it is no surprise that one of their engineer developed the best forecast for this competition. Facebook has also been giving away as an open source software its statistical forecasting model, Prophet (6) , which has been used in several innovative forecast methods (7).
From a strategic point of view, and in a simple way, we have 2 major forecasting topics within a company to tackle:
- Forecasting long term sales to evaluate the budget of the next 3-5 years at a national/regional level
- Forecasting short term sales to help operations plan/anticipate the day/week/month at a precise granularity level
Those are very different topics requiring quite different methodologies !!
Forecasting short term sales (1-2 month)
One of our clients was having difficulties evaluating its French sales at the hospital level for a very specific rare disease. The forecast was designed as a threefold mechanism:
- Forecasting the sales at the national level – At the national level, the sales were plateauing with a very fixed trend, so that statistical models such as Prophet gave accurate results at this granularity level.
- Forecasting the sales at the hospital level – At the hospital level, there are many hospitals, which prevent the use of usual statistical methods. Moreover, the sales of small hospitals have a pattern like (0-1-1-0-1-…) which cannot be accurately forecasted using statistical models. Using ML and a gradient boosting framework such as LightGBM (8) gave fast and accurate results at the hospital level
- Projecting the sales of the national level onto the hospital level – In order to be consistent, the sum of the different hospital forecasts should be equal to the forecast at the national level. Using a top-down approach such as described by Rob J Hyndman and George Athanasopoulos (9) also improved the hospital level forecast.
Compared to using the same method with only statistical models, this model was able to get improved performance. Average error : reduced by 30% !
Forecasting long term sales (3-5 years)
One of our clients wanted to change the design of its forecasting platform. The main objectives of this platform were to:
- Get an accurate forecast for the next 3-5 years at the national level for its markets, products and competitors – As the time range was long and the granularity of the data was quite large, using statistical methods, such as Prophet, gave rather accurate results. Using a top-down approach such as the one described earlier provided a better evaluation of the forecast at the bottom level.
- Get strategic modules for adjusting the forecast and better evaluate the objectives and business performance of teams. Adjusting the forecast with those specific modules to introduce “human intelligence” was able to improve by 5% the average error of the 1 year forecast compared to using standard statistical methods. The business modules were designed such as the user could design events happening in the future, not known from the forecast, in order to get a better evaluation of the next years.
As a conclusion, to produce forecasts with a with better accuracy than usual statistical forecasts, the methodology relies on both data science and human intelligence & insight. Both are necessary. Disconnecting discussions related to business expertise to the data management expertise is the major pitfall that we observe within our clients, mistaken by false promises.
If you want to discuss further with the Karetis team about forecasting use cases in your organization, contact the authors. Authors: Arnaud Jaoul (PhD), Nicolas Cordier
(7) https://dl.acm.org/citation.cfm?id=3355417, adv-geosci.net/45/201/2018/, https://www.onepetro.org/conference-paper/SPE-197419-MS, http://dspace.bracu.ac.bd/xmlui/handle/10361/10121.