Bridging Black-Box Optimization and Machine Learning for Dynamic Algorithm Configuration

• Applicant: Professor Dr. Marius Lindauer
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: since 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 530130944

It is well known that the choice of a best-possible solver for a given optimization problem crucially depends on the problem characteristics as well as on the available computational resources. Understanding this dependency is a core research task in optimization. To support researchers and practitioners in choosing the right optimization approaches for their problem at hand, automated machine learning (AutoML) proposes a plethora of algorithm selection and configuration techniques that are explicitly trained to recommend the most suitable algorithms for a given optimization task. Until very recently, however, these AutoML approaches were restricted to a static selection of algorithm and configuration, thereby ignoring the fact that the best choices can drastically change during the optimization process, e.g., by requiring exploration in the early phases while preferring exploitation as more information about the problem at hand becomes available. Dynamic Algorithm Configuration (DAC) addresses this shortcoming by explicitly asking for a state-dependent selection of algorithms and configurations. The Opt4DAC project will bring together experts in Optimization and in AutoML to leverage the untouched potential of DAC. We aim specifically to come up with approaches that identify suitable switches between different solvers and their configurations on the fly, i.e., in direct response to the requirements of the current optimization progress. To this end, we aim for (i) warm-starting optimizer with information obtained from other solvers, (ii) designing relevant features to describe the current state of the optimization process, and (iii) mechanisms to identify the best moment to reconfigure solvers. Theoretical insights will be challenged through extensive empirical evaluations, ranging from configuring evolutionary algorithms for solving traditional black-box optimization problems to Bayesian Optimization for Hyperparameter Optimization of Deep NeuralNetworks.

• DFG Programme: Research Grants
• International Connection: France
• Cooperation Partner: Dr. Carola Doerr