Reinforcement Learning for Optimal Control of Electric Drives

• Applicant: Professor Dr.-Ing. Oliver Wallscheid
• Subject Area: Electrical Energy Systems, Power Management, Power Electronics, Electrical Machines and Drives ; Automation, Mechatronics, Control Systems, Intelligent Technical Systems, Robotics
• Term: since 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 459524199

Electric drives are used to perform a wide range of tasks in production lines, centrifuges and cranes as well as in road, water and rail vehicles and aircrafts. Usually the drives are operated under closed-loop control, whereby, depending on the application, control of currents, torque, speed, position or (reactive) power can take place. In addition to the typical objectives of stationary control accuracy and high bandwidth, numerous other requirements are set, such as operation with minimal energy conversion losses. Field-oriented control (FOC) is still the most commonly used method for controlling electric drives. The classic FOC is characterized by linear control laws, whose parameterization is carried out with established standard procedures. This enables a (supposedly) rapid control synthesis, but quickly reaches its limits with regard to the above-mentioned control objectives: manipulated variable limits and non-linear plant models as well as deviations between the idealized model and actual system behavior lead, in combination with controllers that are only linear, to a clearly suboptimal control performance. Due to the typical model simplifications (in particular linear-time invariant model assumptions) and the resulting deviation from the actual system behavior, the application of FOR-based control systems is also typically associated with considerable effort, for example in the area of heavily utilized engines in automotive engineering. Against this background, methods from the field of optimal control for electric drive systems have been increasingly investigated in the last two decades. In this domain, however, only methods from the field of model-predictive control (MPC) have been applied, while alternative concepts based on reinforcement learning (RL) have been neglected so far. The former has the disadvantage that target-oriented performance can only be achieved with a sufficiently high prediction horizon and model adaptation at runtime (system identification). This leads to a high computational effort at runtime, which often hinders the spread of MPC approaches due to a control clock in the microsecond range. In contrast, RL works on an infinite horizon and does not require a system model, since an optimal control strategy is derived explicitly and data-driven. Due to a lack of research in the field of electrical drive technology, there are numerous unsolved challenges in the use of RL as part of the control system, including the establishment of safe and efficient learning and the integration of available expert knowledge to support the data-driven learning process. Therefore, this project shall contribute to the evaluation of RL for the control of electric drives.

• DFG Programme: Research Grants