Simulation Tools and Techniques. 11th International Conference, SIMUtools 2019, Chengdu, China, July 8–10, 2019, Proceedings

Research Article

The Adaptive PID Controlling Algorithm Using Asynchronous Advantage Actor-Critic Learning Method

Download
148 downloads
  • @INPROCEEDINGS{10.1007/978-3-030-32216-8_48,
        author={Qifeng Sun and Hui Ren and Youxiang Duan and Yanan Yan},
        title={The Adaptive PID Controlling Algorithm Using Asynchronous Advantage Actor-Critic Learning Method},
        proceedings={Simulation Tools and Techniques. 11th International Conference, SIMUtools 2019, Chengdu, China, July 8--10, 2019, Proceedings},
        proceedings_a={SIMUTOOLS},
        year={2019},
        month={10},
        keywords={Deep Reinforcement Learning Asynchronous Advantage Actor-Critic Adaptive PID control},
        doi={10.1007/978-3-030-32216-8_48}
    }
    
  • Qifeng Sun
    Hui Ren
    Youxiang Duan
    Yanan Yan
    Year: 2019
    The Adaptive PID Controlling Algorithm Using Asynchronous Advantage Actor-Critic Learning Method
    SIMUTOOLS
    Springer
    DOI: 10.1007/978-3-030-32216-8_48
Qifeng Sun1,*, Hui Ren1, Youxiang Duan1, Yanan Yan1
  • 1: University of Petroleum
*Contact email: sunqf@upc.edu.cn

Abstract

To address the problems of the slow convergence and inefficiency in the existing adaptive PID controllers, we proposed a new adaptive PID controller using the Asynchronous Advantage Actor-Critic (A3C) algorithm. Firstly, the controller can parallel train the multiple agents of the Actor-Critic (AC) structures exploiting the multi-thread asynchronous learning characteristics of the A3C structure. Secondly, in order to achieve the best control effect, each agent uses a multilayer neural network to approach the strategy function and value function to search the best parameter-tuning strategy in continuous action space. The simulation results indicated that our proposed controller can achieve the fast convergence and strong adaptability compared with conventional controllers.