train_fmu_gym

The train_fmu_gym library provides a framework to generate and manage Functional Mock-up Unit (FMU) based reinforcement learning (RL) training environments implementing the gymnasium API. The library enables users to leverage preexisting simulation models for deep reinforcement learning by utilizing the FMI standard, supporting both FMI 2.0 and FMI 3.0 Co-Simulation FMUs. train_fmu_gym offers automatized environment generation from FMUs, handling of different reward implementations and observation sets, definition and utilization of scenarios including uncertain scenarios via Modelica Credibility integration, as well as introduction of evaluation metrics to assess trained agents. The training process is supported by more than 20 command line interface (CLI) user functions, enabling fast setup, training and evaluation of RL agents without the need to write extensive additional Python code. train_fmu_gym is compatible with both Linux and Windows platforms. A community edition with limited input/output support is freely available under the CC BY-NC-ND 4.0 license; for the full version without restrictions, please contact the DLR Institute of Vehicle Concepts, Department of Vehicle System Dynamics and Control.

For further information have a look at the accompanying paper:
J. Ultsch, K. Ahmic, and J. Brembeck,
„train_fmu_gym: A Functional Mock-up Unit-based Framework to Train Reinforcement Learning Agents for Multi-Physical Systems,“
IEEE Access, 2025, DOI: 10.1109/ACCESS.2025.3636787

Want to learn more or try it yourself?
Code & documentation (Helmholtz GitLab): https://codebase.helmholtz.cloud/dlr-vsdc/train_fmu_gym