Dynamical System Transfer Learning with Reduced Order Models

Reinforcement Learning (RL) faces challenges with lengthy training times when applied to complex physical systems due to computationally expensive simulations. This problem can be addressed using transfer learning, where an RL agent is first trained on a simpler, faster simulation before moving to the complex one. The article proposes creating these simpler simulations as Reduced Order Models (ROMs) derived from data. Specifically, it details using the Sparse Identification of Nonlinear Dynamics (SINDy) method to generate a data-driven ROM of a turbojet engine. This ROM is then used to pre-train an RL agent for an autothrottle control task, aiming to reduce overall training time compared to training on the full-fidelity simulation from scratch.