Defining External Validity

There are a variety of definitions for external validity in use, focusing on whether a particular parameter (say the average treatment effect, or one or more conditional average treatment effects) is the same across contexts.

I advocate for a broader definition, which nests the existing ones I’m aware of. External validity is the ability to use data from reference contexts to accurately predict a feature of a new, target context. Existing definitions assume that we use estimates of the parameter of interest (for example, the average treatment effect) from the reference contexts directly to predict the parameter in the target context, but we may want to use additional information from the reference and target contexts. Using a structural model for prediction of the average treatment effect in a new context, for example, will typically involve using information beyond the average treatment effect estimates from the reference contexts.

I discussed this in a recent lecture at Oxford’s Blavatnik School of Government (slides). A video of the lecture is below.

Recently, I’ve been happy to see other authors also adopting this approach!