On the Limits of Digital Twins for Safe Deep Reinforcement Learning in Robotic Networks

Successfully training models which address the complex nature of real environments in Smart Cities and robotic networks is challenging, due to the vast amount of data needed. When employing Reinforcement Learning (RL) models, it is also impossible to let them explore all the actions in the real world, due to potential damages and inappropriate actions. The inherent trial-and-error nature of RL, especially in real-world applications like traffic management, makes an integration with Digital Twins (DTs) attractive: DTs provide a secure environment for iterative training and testing, ensuring the refinement of the models before testing. However, the use of DTs has some significant limitations, as the difference between the model and reality may cause significant risks. This study focuses on the adaptive and self-learning characteristics of this approach, considering a standard navigation task in a highway environment, and analyzes its advantages and potential pitfalls.