VALUETOOLS 2008
INTER-PERF 2008WNS2 2008MODENETS 2008GAMECOMM 2008SMCTOOLS 2008

    VALUETOOLS

    3rd International ICST Conference on Performance Evaluation Methodologies and Tools

    Welcome to Athens! On behalf of the whole organizing team, we wholeheartedly welcome you to the third installment of Valuetools, Valuetools 2008. The first two installments took place in Pisa (2006) and Nantes (2007). So, again, the conference is taking place in a city of rich cultural heritage. We…

    Welcome to Athens! On behalf of the whole organizing team, we wholeheartedly welcome you to the third installment of Valuetools, Valuetools 2008. The first two installments took place in Pisa (2006) and Nantes (2007). So, again, the conference is taking place in a city of rich cultural heritage. We look forward to emulating the previous two installments also in terms of their scientific success. The motivation behind Valuetools is the observation that an impressive range of methodologies and tools have been developed recently for the purpose of performance evaluation, across many disparate research fields. Valuetools is meant to be a forum that will allow researchers to compare and debate the full range of these tools and methodologies, and in addition promote the interdisciplinary flow of technical information.

    more »
    Editor(s): John Baras and Costas Courcoubetis
    Publisher
    ICST
    ISBN
    978-963-9799-31-8
    Conference dates
    20th–24th Oct 2008
    Location
    Athens, Greece
    Appeared in EUDL
    29th Nov 2011
    Appears in
    ACM Digital Library

    Copyright © 2011–2013 ICST

    • Efficient reinforcement learning in parameterized models: discrete parameters

      Research Article in 3rd International ICST Conference on Performance Evaluation Methodologies and Tools

      Kirill Dyagilev, Shie Mannor, Nahum Shimkin

      Abstract
      We consider reinforcement learning in a parameterized setup, where the controlled model is known to belong to a finite set of Markov Decision Processes (MDPs) under the discounted return criteria. We…We consider reinforcement learning in a parameterized setup, where the controlled model is known to belong to a finite set of Markov Decision Processes (MDPs) under the discounted return criteria. We propose an on-line algorithm for learning in such parameterized models, called the Parameter Elimination (PEL) algorithm, and analyze its performance in terms of the the total mistake bound criterion, which upper-bounds the total number of suboptimal actions performed by the algorithm over the infinite time horizon. The proposed algorithm relies on Wald's Sequential Probability Ratio Test to eliminate unlikely parameters, and uses an optimistic policy for effective exploration. We establish that, with high probability, the total mistake bound for the algorithm is linear (up to a logarithmic term) in the cardinality |Θ| of the parameter set, independently of the cardinality of the state and action spaces. We further demonstrate that much better dependence |Θ| may be obtained for this algorithm, depending on the specific information structure of the problem.
       more »