sg 16(11): e2

Research Article

Experiments as Instruments: Heterogeneous Position Effects in Sponsored Search Auctions

  • @ARTICLE{10.4108/eai.8-8-2015.2261043,
        author={Matthew Goldman and Justin Rao},
        title={Experiments as Instruments: Heterogeneous Position Effects in Sponsored Search Auctions},
        journal={EAI Endorsed Transactions on Serious Games},
        keywords={mechanism design, sponsored search auctions, online experimentation, iv estimation},
  • Matthew Goldman
    Justin Rao
    Year: 2015
    Experiments as Instruments: Heterogeneous Position Effects in Sponsored Search Auctions
    DOI: 10.4108/eai.8-8-2015.2261043
Matthew Goldman1,*, Justin Rao2
  • 1: Microsoft
  • 2: Microsoft Research
*Contact email:


Google and Bing employ a generalized second price (GSP) auction to allocate billions of dollars of sponsored search advertising. Despite evolving from a naive, first-price, rank-by-bid auction, the GSP has been shown to achieve an efficient allocation and favorable revenue properties via a tight theoretical link with the equilibrium of the truthful Vickrey-Clarke-Groves mechanism. This link hinges on a critical assumption about the causal impact of ad position on user click probabilities: moving up increases click-through-rate by the same scaling factor for all ads. Because an ad’s position is the endogenous outcome of an auction, these assumptions have gone largely untested. We overcome this identification problem by developing a novel method to re-purpose internal business experimentation at a major search engine. Using a broad cross-section of advertisers we strongly reject the conventional multiplicatively-separable model, instead finding substantial heterogeneity of the causal impact of position on click-through-rates. For brand queries, off-brand competitors benefit much more from position than the advertiser matching the query. For generic queries, higher quality and less-well known websites benefit more from position. The impact of position differs by up to 100% across ad types. These findings raise serious concerns about the efficiency properties of the GSP as currently employed.