Causal inference and model explainability analysis of a real world sales dataset

summary

Video file (mp4)

The gist

The gist The authors aim to provide a recipe for applying model interpretability and causal inference for deriving sales insights by reviewing existing literature and applying it to a real-world

In short

The study applies model interpretability and causal inference to retail sales data. It finds that inherently explainable models yield more stable explanations (lower SHAP variance). Furthermore, using a double machine learning approach with multiple confounders allows researchers to correctly determine the sign of causal effects, proving that controlling for variables is crucial for accurate causal conclusions.

Key concepts

Model Interpretability
This involves understanding *why* a predictive model makes certain predictions. It can be local (explaining one specific prediction) or global (understanding the model's overall behavior). The paper focuses on SHAP values, which are a tool used to explain feature contributions in complex models.
Causal Inference
This helps answer 'what if' questions by determining if a change in one variable actually *causes* a change in sales. Double machine learning is used here; it trains one model to handle confounding factors and then another model to estimate the true causal effect of the feature of interest.
Double Machine Learning
This is a technique for observational causal inference that uses two separate machine learning models sequentially. The first model helps remove the influence of confounders from the feature being studied, and the second model estimates how changing that feature truly affects sales, ensuring a more reliable causal sign.
SHAP Values
Shapley Additive Explanations is a method rooted in cooperative game theory used to explain predictions. The authors found that inherently explainable models produce SHAP values with less variance than other models, suggesting they offer more reliable explanations for critical decisions.

Terminology used across episodes

This episode discusses

The paper

Causal inference and model explainability analysis of a real world sales dataset · Read on arXiv

Pranav Gupta, Nithin Surendran

Lowe’s Companies Inc

Most major retailers today have multiple divisions focused on various aspects, such as marketing, supply chain, online customer experience, store customer experience, employee productivity, and vendor fulfillment. They also regularly collect data corresponding to all these aspects as dashboards and weekly/monthly/quarterly reports. Although several machine learning and statistical techniques have been in place to analyze and predict key metrics, such models typically lack interpretability. Moreover, such techniques also do not allow the validation or discovery of causal links. In this paper, we aim to provide a recipe for applying model interpretability and causal inference for deriving sales insights. In this paper, we review the existing literature on causal inference and interpretability in the context of problems in e-commerce and retail, and apply them to a real-world dataset. We find that an inherently explainable model has a lower variance of SHAP values, and show that including multiple confounders through a double machine learning approach allows us to get the correct sign of causal effect.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Hi, I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Causal inference and model explainability analysis of a real world sales dataset".

Jane: Here's the gist of the paper. The gist The authors aim to provide a recipe for applying model interpretability and causal inference for deriving sales insights by reviewing existing literature and applying it to a real-world retail dataset, finding that an inherently explainable model has a lower variance of SHAP values, and showing that including multiple confounders through a double machine learning approach allows us to get the correct sign of causal effect.

Tom: Model Interpretability Model interpretability can be at 2 levels: local and global, where local interpretability is a feature-dependent prediction, while global interpretability measures are independent of x<ref:2512.12605#pg4>.

Jane: Whether to use local or global interpretability measures depends on the application<ref:2512.12605#pg4>.

Tom: Another classification of interpretability is into post-hoc and inherently interpretable modeling, where post-hoc interpretability is more commonly used when a model optimizes accuracy and then invokes necessary tools to explain predictions<ref:2512.12605#pg4>.

Title and authors: Tom: Pranav Gupta and Nithin Surendran are the authors here, and the title really tells you they're tackling a problem where you need both understanding *and* cause-and-effect in sales data.

Jane: It’s about moving past just knowing what happened to figuring out what would happen if we changed something, which is the whole point of causal inference.

Lu: The paper sets up the context by pointing out that while most retail models lack interpretability and can't discover causal links, they are collecting tons of data from marketing, supply chain, and customer experience divisions.

Meng: So they’re starting with this big problem: you have these complex systems driving sales metrics but you can’t trust the predictions or find the real drivers.

Lalam: They introduce the idea of a recipe for applying both interpretability and causal inference together to derive those sales insights from a real-world dataset.

The paper's summary: Tom: The core finding they report is that an inherently explainable model has lower variance in its SHAP values compared to some other models they tested, even when comparing it against models like the explainable boosting regressor.

Jane: That’s interesting because usually, you trade off accuracy for interpretability, so finding a way to keep the explanation stable while keeping the model useful is important.

Lu: They also show that using a double machine learning approach, by including multiple confounders—things that can mess up the relationship—they can get the correct sign of the causal effect.

Meng: Including more confounders sounds like it adds complexity to training, but if it helps you get the right direction for a cause-and-effect analysis, that’s valuable for decision making.

Lalam: They specifically mention single-link hierarchical clustering on features to find redundancy between things like customer perception and lack of seller responsiveness, which helps them control for those variables.

The paper's improvements: Tom: One major suggestion they make is that we need to move beyond just using Shapley values when we want to discover causal relationships because the authors suggest they aren't entirely reliable for that purpose.

Jane: So what’s the fix then? They point toward integrating SHAP values into other explanation techniques, like c-SHAP, which sounds like a way to ground those explanations in actual causal knowledge.

Lu: They also suggest expanding the work by using computationally efficient methods for calculating SHAP values and for estimating DAGs, which is Directed Acyclic Graphs.

Meng: I’m thinking about deployment—if we want this to be production ready, we need methods that are fast enough to run on live data without taking too much time or memory.

Lalam: They also suggest using single-link hierarchical clustering on features to identify observable confounding, which helps select the right variables for their causal inference analysis.

Conclusion: Tom: So, to wrap up, the paper "Causal inference and model explainability analysis of a real world sales dataset" shows that inherently explainable models have lower variance in SHAP values, and by using multiple confounders via double machine learning, you can get the correct sign of causal effect.

Jane: It suggests that for high-stakes decisions in retail, we should prioritize inherently explainable models when interpretability is critical because they provide more stable explanations.

Lu: The next steps involve integrating SHAP into causal inference methods and using more efficient ways to calculate those values and estimate the DAGs.

Meng: For us at the startup side, it means we need to be careful about which features we control for when we try to isolate a cause-and-effect relationship in our models.

Lalam: I'm also sure that this work on understanding how these tools interact is a solid foundation for how we can build better analytical systems for sales insights.

More episodes

← Home