Towards Automated Reinforcement Learning: Applications and Prospects of Large Language Models as Cognitive Components in the Full RL Lifecycle

Deep reinforcement learning (DRL) has achieved remarkable progress in recent years, yet its practical deployment remains constrained by low sample efficiency, challenging reward design, and limited general cognitive capabilities. Recent advances in large language models (LLMs) offer a promising avenue to address these challenges through their powerful reasoning and code-generation abilities. This paper proposes a taxonomic framework of LLM-driven reinforcement learning across the entire lifecycle, systematically examining how LLMs can act as proxies for human expertise in four key stages: environment construction, reward design, decision-making, and collaboration. The study shows that LLM-based code generation enables automated creation of simulation environments and dense reward functions, while hierarchical planning effectively balances commonsense reasoning and real-time decision requirements. In addition, natural-language interfaces significantly enhance the interpretability and efficiency of multi-agent collaboration. These findings demonstrate that the deep integration of LLMs and reinforcement learning establishes a closed-loop paradigm of cognition and action, providing a promising technical pathway toward the development of artificial general intelligence (AGI).