Optimale Stochastische Regelung basierend auf Regression mittels Gauß-Prozessen
Zusammenfassung der Projektergebnisse
In general, the stochastic optimal control problem with non-linear dynamics, continuous state space and imperfect state information is computationally intractable using classical dynamic programming. One solution to overcome this burden is given in form of the so-called point-based value iteration approaches. In this project, we developed a novel point-based method using Gaussian processes over probability distributions. In the point-based value iteration, the value function is only determined on a finite set of reference points. To evaluate it on other inputs a regression method is required. As we consider a control problem with imperfect state information the inputs of the value function are probability distributions. The key idea of this project is to use Gaussian processes of probability distributions as regression method for the value function. As first step, we enhanced an existing approach for Gaussian processes over probability distributions represented by a set of weighted samples (Dirac mixtures). Additionally, the approach was modified for inputs in form of Gaussian mixtures. The key idea in both approaches is to replace the Euclidean distance in the covariance function by a distance between probability distribution. Alternatively, a parametric regression approach for functions over Dirac mixtures was developed. This approach is based on splines, more precisely B-Splines and truncated power functions. The most important challenge was to define a difference between Dirac mixtures, and closely related to define a sorting for them. The developed regression methods are needed to calculate the Bellman recursion in the point-based value iteration approach. The Bellman update equation involves another challenge. For continuous state space and imperfect state information an integral have to be solved. For probability distributions in form of Dirac mixtures, it can be solved easily exploiting the sampling property of the Dirac distribution. For probability distributions in form of Gaussian mixtures, a numerical method has to be applied. Except for the special case of of a cost model in form of a Gaussian mixture where a part of the solution can be calculated analytically. The approach can be applied to non-linear systems with imperfect state information and continuous state space which is often required in real-world problems. However, it is limited to applications with low-dimensional state space.
Projektbezogene Publikationen (Auswahl)
-
Stochastic Optimal Control Using Gaussian Process Regression Over Probability Distributions. In Proceedings of the 2019 American Control Conference (ACC 2019), Philadelphia, PA, USA, July 2019
Mayer, Jana, Maxim Dolgov, Tobias Stickling, Selim Ozgen and Uwe D. Hanebeck
-
Position and Speed Estimation for BLDC Motors Using Fourier-Series Regression. In Proceedings of the 23rd International Conference on Information Fusion (Fusion 2020), Virtual, July 2020
Basarur, Ajit, Jana Mayer, Antonio Zea and Uwe D. Hanebeck
-
Position and Speed Estimation of PMSMs Using Gaussian Processes. In Proceedings of the 1st Virtual IFAC World Congress (IFAC-V 2020), Virtual, July 2020
Mayer, Jana, Ajit Basarur, Mariana Petrova, Fabian Sordon, Antonio Zea and Uwe D. Hanebeck
