Eventually-stationary policies for Markov decision models with non-constant discounting

Yair Carmon; Adam Shwartz

3rd International ICST Conference on Performance Evaluation Methodologies and Tools

Research Article

Eventually-stationary policies for Markov decision models with non-constant discounting

Download579 downloads

Cite: BibTeX Plain Text

@INPROCEEDINGS{10.4108/ICST.VALUETOOLS2008.4392,
    author={Yair Carmon and Adam Shwartz},
    title={Eventually-stationary policies for Markov decision models with non-constant discounting},
    proceedings={3rd International ICST Conference on Performance Evaluation Methodologies and Tools},
    publisher={ICST},
    proceedings_a={VALUETOOLS},
    year={2010},
    month={5},
    keywords={Markov Decision Processes Discounted Cost Mixed Discounting Hyperbolic Discounting General Discounting Function},
    doi={10.4108/ICST.VALUETOOLS2008.4392}
}

Yair Carmon
Adam Shwartz
Year: 2010
Eventually-stationary policies for Markov decision models with non-constant discounting
VALUETOOLS
ICST
DOI: 10.4108/ICST.VALUETOOLS2008.4392

Yair Carmon¹, Adam Shwartz¹^,*

1: Department of Electrical Engineering, Technion — IIT, Haifa 32000, Israel

*Contact email: adam@ee.technion.ac.il

Abstract

We investigate the existance of simple policies in finite discounted cost Markov Decision Processes, when the discount factor is not constant. We introduce a class called "exponentially representable" discount functions. Within this class we prove existence of optimal policies which are eventually stationary---from some time N onward, and provide an algorithm for their computation. Outside this class, optimal policies with this structure in general do not exist.

Keywords: Markov Decision Processes Discounted Cost Mixed Discounting Hyperbolic Discounting General Discounting Function

Published: 2010-05-16
Publisher: ICST
Modified: 2010-05-16

: http://dx.doi.org/10.4108/ICST.VALUETOOLS2008.4392