Experiments as Instruments: Heterogeneous Position Effects in Sponsored Search Auctions

Google and Bing employ a generalized second price (GSP) auction to allocate billions of dollars of sponsored search advertising. Despite evolving from a naive, first-price, rank-by-bid auction, the GSP has been shown to achieve an efficient allocation and favorable revenue properties via a tight theoretical link with the equilibrium of the truthful Vickrey-Clarke-Groves mechanism. This link hinges on a critical assumption about the causal impact of ad position on user click probabilities: moving up increases click-through-rate by the same scaling factor for all ads. Because an ad’s position is the endogenous outcome of an auction, these assumptions have gone largely untested. We overcome this identification problem by developing a novel method to re-purpose internal business experimentation at a major search engine. Using a broad cross-section of advertisers we strongly reject the conventional multiplicatively-separable model, instead finding substantial heterogeneity of the causal impact of position on click-through-rates. For brand queries, off-brand competitors benefit much more from position than the advertiser matching the query. For generic queries, higher quality and less-well known websites benefit more from position. The impact of position differs by up to 100% across ad types. These findings raise serious concerns about the efficiency properties of the GSP as currently employed.