Choosing the Right Causal Quantity in Business Analysis

In applied causal analysis, we often spend a lot of time thinking about methods. We think about experiments, matching, difference-in-differences, instrumental variables, synthetic controls, causal forests, double machine learning, and many other approaches.

But before we choose a method, there is a more basic question we need to answer:

Which causal quantity are we trying to estimate?

This question is the foundation of the entire causal analysis that follows, and the answer should match the business question we are trying to answer. Are we trying to decide whether a treatment should become the default for everyone? Are we trying to evaluate whether an intervention worked for the group that actually received it? Or are we trying to understand which types of customers, regions, or more generally, units benefit most from an intervention?

These are different questions. And because they are different questions, they often require different causal quantities.

This matters because the causal quantity we choose influences the identification strategy, the estimation procedure, the assumptions we need, and the way we interpret the result. If we choose the wrong target, we may end up with an analysis that is technically correct, but not very useful for the conclusions we want to draw or the decision we need to make.

In business settings, three causal quantities appear especially often: the Average Treatment Effect, the Average Treatment Effect on the Treated, and the Conditional Average Treatment Effect. These are usually abbreviated as ATE, ATT, and CATE.

In this blog post, I want to explain what each of these quantities means and when each one is appropriate in practice.

Average Treatment Effect

Let’s start with the Average Treatment Effect, usually abbreviated as ATE. The ATE answers the question: what is the average effect of the treatment in the full target population?

To make this more concrete, suppose a company is considering a new website version, a new onboarding flow, a discount campaign, or a new recommendation algorithm. The outcome of interest could be conversion, activation, revenue, retention, engagement, or another business outcome. The ATE asks how much the outcome changes on average because of the treatment, for units in the full target population. In other words, it summarizes the average difference between the outcome we would expect under treatment and the outcome we would expect without treatment, across the full target population.

In business, the ATE often becomes a natural target when the question is about a rollout or default decision. We often choose the ATE when we want to know whether a treatment should be applied to all units in a target group. The treatment may currently only be tested on a smaller group, but the decision we want to make concerns the larger population.

A classic example is an A/B test on website versions. A company may run an experiment for a limited period of time, with some visitors seeing the current version and others seeing the new version. The purpose of the experiment is often to answer a rollout question: should the new version become the default for all eligible users? In that case, the ATE is the natural target. We are using the experiment to learn what the average effect of the new website version would be if we rolled it out to the broader group of users for whom the rollout decision is relevant.

The same logic applies to many other business use cases. If we want to know whether a new onboarding flow should become the default for all new users, whether a discount campaign should be sent to the full eligible customer base, or whether a new recommendation algorithm should become the default for all eligible users, the ATE is often the right causal quantity.

Average Treatment Effect on the Treated

Now let’s turn to the Average Treatment Effect on the Treated, usually abbreviated as ATT. The ATT answers the question: what is the average effect of the treatment for the units that actually received it?

The difference with the ATE is that the ATE focuses on the full target population, including both treated and untreated units, while the ATT focuses specifically on the units that actually received the treatment. In other words, it asks how the treatment affected the customers, stores, regions, or, more generally, units that were actually exposed to it.

This makes the ATT a natural causal quantity when the business question is about evaluating the impact or effectiveness of an intervention that was applied to a specific group. We usually choose the ATT when we want to know whether a campaign, programme, offer, or intervention actually worked for the group that received it.

A classic example is a customer programme. Suppose a company launches a loyalty programme, and customers choose whether or not to join. After the programme has been running for some time, the company may want to know whether it actually improved the outcomes it was meant to improve, such as retention, purchase amounts, or engagement. In that case, the ATT is the natural target. The question is not necessarily whether the programme should be offered to all customers, since the company cannot force every customer to join. The question is whether the programme actually had an effect for the customers who joined.

The same logic applies to many other business use cases. If we want to know whether a retention offer reduced churn among the customers who received it, whether a sales outreach programme improved conversion for the leads that were contacted, whether extra customer support improved renewal outcomes for the accounts that received it, or whether a win-back campaign increased purchases among the customers who were targeted, the ATT is often the right causal quantity.

Conditional Average Treatment Effect

Finally, let’s turn to the Conditional Average Treatment Effect, usually abbreviated as CATE. The CATE answers the question: how does the average effect of the treatment differ across groups or types of units?

This is different from both the ATE and the ATT. The ATE focuses on the average effect in the full target population, and the ATT focuses on the average effect for the units that actually received the treatment. The CATE focuses on treatment effect heterogeneity. In other words, it asks whether the treatment effect is larger, smaller, absent, or even negative for different types of customers, stores, regions, or other units.

This makes the CATE a natural causal quantity when the business question is about understanding who benefits most from a treatment, who benefits least, and whether there are groups for whom the treatment may not be worth applying or may even be harmful.

A classic example is a marketing campaign. Suppose a company is considering sending a promotional message to customers who have not purchased in a while. Some customers may be reminded by the message and make a purchase they otherwise would not have made. Other customers may have purchased anyway, even without the message. For those customers, the message creates little or no incremental value. And for some customers, the message may even have a negative effect, for example because they find it irrelevant, intrusive, or annoying and become less likely to engage with the company in the future.

In that case, the CATE is the natural target. The question is not only whether the marketing campaign works on average. The question is which types of customers are most positively affected by the message, for which customers the message creates little or no incremental value, and whether there are groups for whom the message may create negative effects.

This information can then be used to move from a one-size-fits-all campaign to a more targeted decision rule. We may want to send the message to customers for whom it is expected to create positive incremental value, avoid sending it to customers for whom it is unlikely to matter, and be especially careful with groups for whom the message may have a negative effect.

The same logic applies to many other business use cases. If we want to know whether a new product feature works better for first-time users than for experienced users, whether proactive customer support is more effective for high-risk accounts than for low-risk accounts, or whether a recommendation algorithm works differently across product categories or user groups, the CATE is often the right causal quantity.

Conclusion

The main point is that ATE, ATT, and CATE answer different causal questions.

The ATE helps us answer broad rollout or default-decision questions. It asks what the average effect of the treatment is in the full target population.

The ATT helps us evaluate interventions that were applied to a specific group. It asks what the average effect of the treatment is for the units that actually received it.

The CATE helps us understand treatment effect heterogeneity. It asks how the average effect of the treatment differs across groups or types of units.

So before thinking about estimation and identification methods, make sure you are clear about what you are targeting. Otherwise, you risk spending time and effort on an analysis that may be technically correct, but does not answer the question you ultimately care about.