Multi-agent Quantum Reinforcement Learning for Digital Twin Placement in 6G Multi-tier Systems

The digital twins (DTs) represent critical components for the simulation, analysis, and optimization of physical systems, with significant implications for efficiency and cost management in 6G network applications. The introduction of multi-tier computing has the potential to enable a more streamlined integration of DTs in 6G networks by offering services at the edge network level. However, this integration brings about various complexities related to the placement and maintenance of DTs within edge networks, thus increasing the processing latency. To address these problems, we investigate the application of quantum computing, which exploits quantum principles such as superposition and entanglement, and offers potential resolutions for these computational dilemmas. In this paper, we formulate the DT placement problem to incorporate variational quantum circuits for learning agents within a multi-agent reinforcement learning framework. In particular, quantum multi-agent reinforcement learning is proposed to establish an optimal policy for associating DTs with edge networks. This approach aims to reduce latency while adhering to the computational resource limitations of the edge server. Simulation results illustrate the proficiency and robustness of quantum multi-agent actor-critic networks in acquiring a policy that ameliorates the reward function, hence decreasing latency while adhering to the optimization constraints. This study contributes to the evolving field of quantum computing applications in multi-tier environments and provides methodological insights for optimizing DT deployment in 6G networks.