The Transfer Performance of Economic Models

Co-authors: Isaiah Andrews, Drew Fudenberg, Lihua Lei, and Chaofeng Wu

Abstract
Economists often use models estimated on data from a particular domain to make predictions in another. We provide a tractable formulation for this "out-of-domain" prediction problem and define the transfer error of a model  based on its performance in the new domain. We derive finite-sample forecast intervals that are guaranteed to cover realized transfer errors with a user-selected probability when domains are i.i.d. We then apply these intervals in an application to compare the transferability of economic models and black box algorithms, finding that black box algorithms outperform economic models for prediction within-domain, but generalize more poorly across domains.