Reinforcement Learning for Rack-Level Cooling

In recent years, we have seen a rapid growth in big data and cloud computing industry, because of the rapid development in Internet technology. That’s why the number and scale of data center have increased rapidly. A data center is a warehouse-level IT facility that hosts many servers. Because of the uneven heat production and heat dissipation of the servers in a rack, the hot spots emerges. In order to maintain the CPU temperature hence the computing performance, the server cooling is very critical. A common solution is to increase the speed of the Computer Room Air Handler (CRAH) blower and increase the flow of cold air. Nevertheless, this solution can only partially address the issue and raise the cooling energy consumption.

In this paper, we study how to mitigate rack hot spots without significantly increasing the power of air conditioning system. We propose the Active Ventilation Tiles (AVTs), i.e., ordinary ventilation tiles with attached fans, to enhance the local cold air delivery and improve the cooling performance. In particular, we propose an AVT control algorithm adapted from the Reinforcement Learning techniques to tackle the complex data center environment and thermo dynamic process. The reinforcement learning algorithm adjusts the temperature distribution of the rack by controlling the fan speed installed on the ventilation tile, and guides the fan speed according to the feedback temperature to mitigate hot spots. Due to the slow learning speed of the traditional Tabular-Q-Learning algorithm, we integrate the Tabular-Q-Learning algorithm with the Dyna architecture to accelerate the learning speed and improve the algorithm performance in the early stage. Experimental results reveal that Tabular-Q-learning based on Dyna has better performance.