IEEE Access, 13
Deep reinforcement learning (DRL) has become increasingly popular for solving sequential decision-making problems in control applications, particularly for multi-physical systems such as vehicle and robotic motion control. The application of DRL to control applications is especially appealing, as control engineering is strongly conducted with the utilization of simulation models. Leveraging these preexisting simulation models for DRL agent training provides a promising and effective strategy. Although DRL has gained popularity in controlling complex multi-physical systems, existing DRL frameworks still lack support for the entire training process of DRL agents utilizing preexisting simulation models. In this work, we present a structured workflow accompanied by its implementation, which utilizes the Functional Mock-up Interface (FMI) to leverage the use of preexisting simulation models for DRL training. We introduce the concept of scenarios as a set of environment parameters and formalize the concept of evaluation metrics. In addition, we extended the scenario concept to uncertain scenarios, to account for environmental uncertainties during training. Our proposed workflow is demonstrated on a path following control problem in automated driving, where a residual policy is learned to enhance the performance of a baseline controller. Using the proposed workflow and toolchain, the trained agent reduced the maximum lateral displacement error and the velocity tracking error by up to 93.4% and 41.3% on seen paths, and by 88.7% and 25.3% on an unseen path, respectively. These results demonstrate the effectiveness of our approach in leveraging pre-existing simulation models for DRL training, paving the way for efficient and effective control applications.
Copyright © 2008-2026 German Aerospace Center (DLR). All rights reserved.