Causal AI explores the integration of causal inference into machine learning
Welcome to our series on Causal AI, where we discuss integrating causal inference into machine learning models. We discuss numerous practical applications across a variety of business contexts.
In the previous article Verifying the causal impact of synthetic control methodsIn this article, Enhanced marketing mix modeling with Causal AI.
If you missed our previous article on synthetic control, you can find it here.
Continuing challenges with digital tracking have seen a recent resurgence in Marketing Mix Modeling (MMM). At the recent Causal AI conference, Judea Pearl suggested that marketing may be the first industry to adopt Causal AI. So I decided it was time to start writing about my learnings over the past seven years in terms of how MMM, Causal AI, and experimentation intersect.
Investigate the following areas:
- What is MMM?
- How can Causal AI enhance MMM?
- What experiments can you perform to complete the triangulation?
- Unresolved challenges in marketing measurement.
You can find the complete notebook here:
MMM is a statistical framework used to estimate how much each marketing channel contributes to sales. It is heavily influenced by econometrics, and in its simplest form is a regression model. Let's go through the basics of its main components:
Regression
A regression model is built in which a dependent variable/target (usually sales) is predicted based on several independent variables/features, which typically include spending on different marketing channels and external factors that may affect demand.
The coefficient of an expenditure variable indicates how much it contributes to sales.
Python's PyMC marketing package is a great place to start exploring MMM.
Ad Stock
Ad stock refers to the long-term impact that marketing spend (or advertising spend) has on consumer behavior. It helps model the long-term impact of marketing. It is not a typical behavior to rush out and buy a product the first time you hear about a brand. The idea of ad stock is that the impact of marketing is cumulative.
The most common ad stock method is the geometric decay method, which assumes that the impact of ads decays at a constant rate over time. This is relatively easy to implement, but it is inflexible. It is also worth trying the Weibull method, which is more flexible; the PyMC marketing package implements this, so be sure to check it out.
Saturation
Saturation, in the context of marketing, refers to the concept of diminishing returns: Increasing marketing spend may increase customer acquisition, but over time it becomes harder to influence new audiences.
There are several saturation methods you can use, the Michaelis-Menton function is one of the more common ones, you can also see this in the PyMC marketing package.
MMM frameworks typically use flat regression models. However, there are complexities in the way marketing channels interact with each other. Are there any tools in the Causal AI toolbox that can help with this?
Causal Graph
Causal graphs are great at separating cause from correlation, making them a great tool for tackling complex situations where marketing channels interact with each other.
If you are not familiar with causal graphs, please read our previous article for a better understanding.
Understanding the Marketing Graph
Inferring a causal graph when little domain knowledge is available is difficult, but you can get started using causal discovery. For more information, see our previous post on causal discovery.
It has limitations in discovering causal relationships and should only be used to generate initial hypotheses for your graph. Fortunately, there is a huge amount of domain knowledge about how marketing channels interact with each other that can be incorporated.
Below, I share the knowledge I’ve gained from working with marketing professionals for many years.
- Pay per click Paid search hurts SEO. The more you spend on PPC, the fewer SEO clicks you get. But there's a key confounding factor… demand. Flat regression models can't capture this complexity and often overestimate PPC.
- Social Spending There is a strong effect on social clicks, and the more you spend, the more likely a prospect will click on your social ad. However, some prospects may view your social ad and then visit your site the next day via PPC, SEO, or Direct. A flat regression model would not detect this halo effect.
- A similar case is Brand SpendingNow, you target your prospects with long-term branding messaging, but without a direct CTA to click. These prospects may visit your site at a later stage via PPC, SEO, or Direct after becoming aware of your brand.
- of click Mediating variables. Running a flat regression and including mediating variables can cause problems when estimating causal effects. We won't go into too much detail on this topic here. Causal graphs allow you to carefully control for the appropriate variables when estimating causal effects.
From the above examples, we can see that using causal graphs instead of flat regressions can significantly strengthen the solution. The ability to calculate counterfactuals and perform interventions is also very attractive.
Keep in mind that it's still worth incorporating ad stock and saturation transformations into your framework.
When working with observational data, you should strive to run experiments to validate hypotheses and complement causal inferences. There are three main tests you can use in acquisition marketing. Let's take a closer look:
Conversion lift test
Social platforms like Facebook and Snapchat allow you to run conversion lift tests, which are AB tests that use treatment and control groups to measure conversion lift. These can be extremely useful when assessing counterfactuals from a causal graph of your social spend.
Geolift Test
GeoLift tests can be used to predict the impact of marketing blackouts or new channel launches. This is particularly useful for digital and TV for brands that don't have a direct call to action to measure. We cover this in more detail in our previous article.
Switchback Test
PPC campaigns can be scheduled to toggle on and off every hour. This creates a great opportunity for switchback testing. Schedule your PPC campaigns to toggle on and off every hour for a few weeks and calculate the difference in PPC + SEO clicks between the off and on periods. This will give you an idea of how much PPC you could be getting through SEO and help you evaluate the counterfactual from a causal graph of your PPC spend.
I think running experiments is a great way to fine-tune your causal graph and make it more reliable. But you can also use the results to tune your model. Let's see how the PyMC team is working on this.
Today we've talked about how you can use Causal AI to power your MMM. But Causal AI doesn't solve all your acquisition marketing challenges. Unfortunately, there are many.
- Spend according to demand forecasts — One reason marketing spend is highly correlated with sales is because marketing teams spend according to demand forecasts. One solution in this case is to randomly shift spend by -10% to +10% each week to introduce variation. As you can imagine, marketing teams are usually not big fans of this approach.
- Forecasting demand — Demand is a critical variable in our model. However, collecting the data can be very difficult. A reasonable option is to pull Google Trends data for search terms that match the products you sell.
- The long-term impact of brands — The long-term impact of a brand is hard to understand since there usually isn't much signal on this. Long-term geographic lift testing can help.
- Multicollinearity — This is actually one of the biggest problems. All the variables are highly correlated. Using ridge regression, this problem is mitigated a bit, but it can still be a problem. Causal graphs also help a bit, because they break the problem down into smaller models.
- Buy-in from the marketing team — In my experience, this is the biggest challenge. Causal graphs are a great visual way to engage your marketing team, and also an opportunity to build relationships with them while coming to terms with the complexity of the graph.
That's all for now. I'd love to hear your thoughts in the comments section.
