Auctions, Market Mechanisms and Their Applications. First International ICST Conference, AMMA 2009, Boston, MA, USA, May 8-9, 2009, Revised Selected Papers

Research Article

Turing Trade: A Hybrid of a Turing Test and a Prediction Market

Download
554 downloads
  • @INPROCEEDINGS{10.1007/978-3-642-03821-1_10,
        author={Joseph Farfel and Vincent Conitzer},
        title={Turing Trade: A Hybrid of a Turing Test and a Prediction Market},
        proceedings={Auctions, Market Mechanisms and Their Applications. First International ICST Conference, AMMA 2009, Boston, MA, USA, May 8-9, 2009, Revised Selected Papers},
        proceedings_a={AMMA},
        year={2012},
        month={5},
        keywords={prediction markets Turing tests games with a purpose deployed web-based applications using points as an artificial currency},
        doi={10.1007/978-3-642-03821-1_10}
    }
    
  • Joseph Farfel
    Vincent Conitzer
    Year: 2012
    Turing Trade: A Hybrid of a Turing Test and a Prediction Market
    AMMA
    Springer
    DOI: 10.1007/978-3-642-03821-1_10
Joseph Farfel1, Vincent Conitzer1
  • 1: Duke University

Abstract

We present Turing Trade, a web-based game that is a hybrid of a Turing test and a prediction market. In this game, there is a mystery conversation partner, the “target,” who is trying to appear human, but may in reality be either a human or a bot. There are multiple judges (or “bettors”), who interrogate the target in order to assess whether it is a human or a bot. Throughout the interrogation, each bettor bets on the nature of the target by buying or selling human (or bot) securities, which pay out if the target is a human (bot). The resulting market price represents the bettors’ aggregate belief that the target is a human. This game offers multiple advantages over standard variants of the Turing test. Most significantly, our game gathers much more fine-grained data, since we obtain not only the judges’ final assessment of the target’s humanity, but rather the entire progression of their aggregate belief over time. This gives us the precise moments in conversations where the target’s response caused a significant shift in the aggregate belief, indicating that the response was decidedly human or unhuman. An additional benefit is that (we believe) the game is more enjoyable to participants than a standard Turing test. This is important because otherwise, we will fail to collect significant amounts of data. In this paper, we describe in detail how Turing Trade works, exhibit some example logs, and analyze how well Turing Trade functions as a prediction market by studying the calibration and sharpness of its forecasts (from real user data).