About | Contact Us | Register | Login
ProceedingsSeriesJournalsSearchEAI
3rd International ICST Conference on Performance Evaluation Methodologies and Tools

Research Article

Efficient reinforcement learning in parameterized models: discrete parameters

Cite
BibTeX Plain Text
  • @INPROCEEDINGS{10.4108/icst.valuetools.2008.56,
        author={Kirill  Dyagilev and Shie  Mannor and Nahum  Shimkin 	},
        title={Efficient reinforcement learning in parameterized models: discrete parameters},
        proceedings={3rd International ICST Conference on Performance Evaluation Methodologies and Tools},
        publisher={ICST},
        proceedings_a={VALUETOOLS},
        year={2010},
        month={5},
        keywords={},
        doi={10.4108/icst.valuetools.2008.56}
    }
    
  • Kirill Dyagilev
    Shie Mannor
    Nahum Shimkin
    Year: 2010
    Efficient reinforcement learning in parameterized models: discrete parameters
    VALUETOOLS
    ICST
    DOI: 10.4108/icst.valuetools.2008.56
Kirill Dyagilev1, Shie Mannor1, Nahum Shimkin 1
  • 1: Technion Israel

Abstract

We consider reinforcement learning in a parameterized setup, where the controlled model is known to belong to a finite set of Markov Decision Processes (MDPs) under the discounted return criteria. We propose an on-line algorithm for learning in such parameterized models, called the Parameter Elimination (PEL) algorithm, and analyze its performance in terms of the the total mistake bound criterion, which upper-bounds the total number of suboptimal actions performed by the algorithm over the infinite time horizon. The proposed algorithm relies on Wald's Sequential Probability Ratio Test to eliminate unlikely parameters, and uses an optimistic policy for effective exploration. We establish that, with high probability, the total mistake bound for the algorithm is linear (up to a logarithmic term) in the cardinality |Θ| of the parameter set, independently of the cardinality of the state and action spaces. We further demonstrate that much better dependence |Θ| may be obtained for this algorithm, depending on the specific information structure of the problem.

Published
2010-05-16
Publisher
ICST
http://dx.doi.org/10.4108/icst.valuetools.2008.56
Copyright © 2008–2025 ICST
EBSCOProQuestDBLPDOAJPortico
EAI Logo

About EAI

  • Who We Are
  • Leadership
  • Research Areas
  • Partners
  • Media Center

Community

  • Membership
  • Conference
  • Recognition
  • Sponsor Us

Publish with EAI

  • Publishing
  • Journals
  • Proceedings
  • Books
  • EUDL