It's very important to analyze and measure the underlying root cause effects between variables. Verifying correlation is only the first step in this process. Applying deep and vetted domain expertise that includes conversations with your customer to validate assumptions and then apply the right statistical formula to confirm the probability that two or more variables are correlating by coincidence.

Just confirming Greg's view below: - the most challenging part is to identify the relevant business drivers for demand. In machine learning jargon, the feature engineering is the hardest part of the exercise. In line with what is the current thinking, this is a hybrid approach of what experts know and what you uncover in the data - always pay attention to the correlation causation trap - if you have a suitable factor model, finding the right model is a lesser challenge than to know what drives your business

According to the most recent forecasting research, it is best practice to choose variables utilizing Bayesian variable selection or a search algorithm such as stepwise search. These methods evaluate which combination of variables results in the highest forecasting accuracy for a specific model. In contrast, correlation or lagged correlation does not have a connection to the accuracy of a forecasting model. This approach is also limited to analyzing the relationship between just two variables and is incapable of grasping complex interdependencies among multiple variables.

Predicting which model will yield the best performance is inherently challenging, making it essential to employ a variety of models and assess their relative performance. When evaluating its performance, it's standard practice to split the available data into separate training and test datasets (and sometimes also a validation dataset). The model is trained on the training dataset and then evaluated on the test dataset, which it has not seen before. This allows researchers and practitioners to get a sense of how well the model is likely to perform on new, unseen data, like forecasting, in real-world scenarios.

Clean data is key to accurate forecasting. Ensuring data is in a consistent format and frequency allows for the best modelling. Then stationarity, collinearity and auto correlation come into play. One classic issue is forecasting Chinese economic data - where often January and February values are combined into one, resulting in 11 observations in a year. Dealing with issues like this correctly/thoughtfully can be the difference between an accurate forecast and a mistake.

Another aspect that you might want to consider is the seasonal and calendar effects that you should handle before applying a causal forecasting model

Ideally you'd be re-forecasting at least daily or weekly, and reviewing forecast performance on a weekly basis based on the accuracy metric you've previously determined. Separately, you should be re-evaluating and training your models monthly or quarterly, and backtesting it against previous model to identify areas of improvements.

Evaluating the forecasts to a baseline is crucial. If your forecast can't in-sample outperform a naïve or trend forecast, what insight does it provide?