Workshop SAMOURAI


When & where

The workshop will take place IRL, on December 10-11, 2024 at Institut Henri Poincaré, Paris.
Room: amphithéâtre Hermite (ground floor).




Presentation of the workshop

Workshop of the SAMOURAI ANR project (ANR-20-CE46-0013)
Simulation Analytics and Meta-model-based solutions for Optimization, Uncertainty and Reliability AnalysIs

Organizers: Delphine Sinoquet, Morgane Menz (IFPEN)

SAMOURAI TEAM
♣ CEA: Amandine Marrel (WP leader), Gabriel Sarazin
♣ Centrale Supélec: Romain Ait Abdelmalek-Lomenech, Julien Bect (WP leader), Emmanuel Vazquez
♣ EDF: Vincent Chabridon (WP leader), Bertrand Iooss, Merlin Keller (WP leader), Julien Pelamatti, Sanaa Zannane
♣ EMSE: Rodolphe Le Riche (WP leader), Babacar Sow
♣ IFPEN: Reda El Amri, Morgane Menz, Miguel Munoz Zuniga (WP leader), Delphine Sinoquet (Project leader)
♣ Polytechnique Montréal: Sébastien Le Digabel (WP leader), Christophe Tribes
♣ Safran: Raphaël Carpintero Perez (CMAP), Sébastien Da Veiga (ENSAI, formerly Safran) Brian Staber (WP leader)


Registration

Registration is free, but compulsory.


Agenda

December 10

December 11

Speakers

Invited researchers

Nathalie Bartoli (ONERA) abstract

Mickaël Binois (Centre Inria d'Université Côte d'Azur) abstract

François-Xavier Briol (University College London) abstract

José Miguel Hernandez Lobato (University of Cambridge) abstract

SAMOURAI Team

Gabriel Sarazin (CEA) abstract

Raphaël Carpintero Perez (CMAP/ Safran) abstract

Amandine Marrel (CEA)

Romain Ait Abdelmalek-Lomenech (Centrale Supélec)

Mohamed Reda El Amri (IFPEN)

Babacar Sow (EMSE/EDF)

Morgane Menz (IFPEN) abstract

Delphine Sinoquet (IFPEN) & Sébastien Le Digabel (GERAD/ Polytechnique Montréal)


Abstracts

Nathalie Bartoli (ONERA)

Title: Bayesian optimization to solve black box problem with hidden constraints
Abstract: This work focuses on developing innovative methodologies for optimizing computationally expensive and complex systems, such as those found in aeronautical engineering. The proposed surrogate-based optimization, commonly known as Bayesian Optimization, leverages adaptive sampling to efficiently balance exploration and exploitation. Our in-house implementation, SEGOMOE, is designed to handle a large number of design variables—whether continuous, discrete, categorical, or hierarchical. SEGOMOE also relies on the use of a mixture of experts (local surrogate models) for both objective functions and constraints. Extensions to handle hidden constraints have also been incorporated. The performance of the resulting methods has been rigorously evaluated on a benchmark of analytical problems, as well as in realistic aeronautical applications.


Mickaël Binois (Centre INRIA d Université Côte d Azur)

Title: Options for high-dimensional Bayesian optimization
Abstract: Bayesian optimization (BO) aims at efficiently optimizing expensive black-box functions, such as hyperparameter tuning problems in machine learning. Scaling up BO to many variables relies on structural assumptions about the underlying black-box, to alleviate the curse of dimensionality. To this end, in this talk we review several options for Gaussian process modeling, with emphasis on the additive and low effective dimensionality hypothesis. We discuss several practical issues related to selecting a suitable search space and to the acquisition function optimization.


François-Xavier Briol (University College London)

Title: Robust and Conjugate Gaussian Process Regression
Abstract: To enable closed form conditioning, a common assumption in Gaussian process (GP) regression is independent and identically distributed Gaussian observation noise. This strong and simplistic assumption is often violated in practice, which leads to unreliable inferences and uncertainty quantification. Unfortunately, existing methods for robustifying GPs break closed-form conditioning, which makes them less attractive to practitioners and significantly more computationally expensive. In this work, we demonstrate how to perform provably robust and conjugate Gaussian process (RCGP) regression at virtually no additional cost using generalised Bayesian inference. RCGP is particularly versatile as it enables exact conjugate closed form updates in all settings where standard GPs admit them. To demonstrate its strong empirical performance, we deploy RCGP for problems ranging from Bayesian optimisation to sparse variational Gaussian processes.


José Miguel Hernandez Lobato (University of Cambridge)

Title: Meta-learning Adaptive Deep Kernel Gaussian Processes for Molecular Property Prediction
Abstract: We propose Adaptive Deep Kernel Fitting with Implicit Function Theorem (ADKF-IFT), a novel framework for learning deep kernel Gaussian processes (GPs) by interpolating between meta-learning and conventional deep kernel learning. Our approach employs a bilevel optimization objective where we meta-learn generally useful feature representations across tasks, in the sense that task-specific GP models estimated on top of such features achieve the lowest possible predictive loss on average. We solve the resulting nested optimization problem using the implicit function theorem (IFT). We show that our ADKF-IFT framework contains previously proposed Deep Kernel Learning (DKL) and Deep Kernel Transfer (DKT) as special cases. Although ADKF-IFT is a completely general method, we argue that it is especially well-suited for drug discovery problems and demonstrate that it significantly outperforms previous state-of-the-art methods on a variety of real-world few-shot molecular property prediction tasks and out-of-domain molecular property prediction and optimization tasks.


Raphaël Carpintero Perez

Title: Learning signals defined on graphs with optimal transport and Gaussian process regression
Abstract: Machine learning algorithms applied to graph data have garnered significant attention in fields such as biochemistry, social recommendation systems, and very recently, learning physics-based simulations. Kernel methods, and more specifically Gaussian process regression, are particularly appreciated since they are powerful when the sample size is small, and when uncertainty quantification is needed. In this work, we introduce the Sliced Wasserstein Weisfeiler-Lehman (SWWL) graph kernel which handles graphs with continuous node attributes. We combine continuous Wesifeiler Lehman iterations and an optimal transport between empirical probability distributions with the sliced Wasserstein distance in order to define a positive definite kernel function with low computational complexity. These two properties make it possible to consider graphs with a large number of nodes, which was previously a tricky task. The efficiency of the SWWL kernel is illustrated on graph regression in computational fluid dynamics and solid mechanics, where the input graphs are made up of tens of thousands of nodes. Another part of the work concerns the extension of the previous approach to the case of vector outputs defined on the graph nodes (whose dimension therefore varies as a function of the number of vertices). We propose an approach based on regularized optimal transport, the aim of which is to transfer the output fields (signals) to a reference measure and then perform a reduction dimension on it.

Joint work with: Sébastien Da Veiga (ENSAI, formerly SAFRAN), Brian Staber (SAFRAN), Josselin Garnier (CMAP, Ecole polytechnique)


Gabriel Sarazin

Title: Towards more interpretable kernel-based sensitivity analysis
Abstract: When working with a computationally-expensive simulation code involving a large number of uncertain physical parameters, it is often advisable to perform a preliminary sensitivity analysis in order to identify which input variables will really be useful for surrogate modelling. On paper, the total-order Sobol' indices fulfill this role perfectly, since they are able to detect any type of input-output dependence, while being interpretable as simple percentages of the output variance. However, in many situations, their accurate estimation remains a thorny issue, despite remarkable progress in that direction over the past few years. In this context where inference is strongly constrained, kernel methods have emerged as an excellent alternative, notably through the Hilbert-Schmidt independence criterion (HSIC). Although they offer undeniable advantages over Sobol' indices, HSIC indices are much harder to understand, and this lack of interpretability is a major obstacle to their wider dissemination. In order to marry the advantages of Sobol' and HSIC indices, an ANOVA-like decomposition allows to define HSIC-ANOVA indices at all orders, just as would be done for Sobol' indices. This recent contribution is the starting point of this presentation.
The main objective of this talk is to provide deeper insights into the HSIC-ANOVA framework. One major difference with the basic HSIC framework lies in the use of specific input kernels (like Sobolev kernels). First, a dive into the universe of cross-covariance operators will allow to better understand how sensitivity is measured by HSIC-ANOVA indices, and what type of input-output dependence is captured by each term of the HSIC-ANOVA decomposition. Then, a brief study of Sobolev kernels, focusing more particularly on their feature maps, will reveal what kind of simulators are likely to elicit HSIC-ANOVA interactions. It will also be demonstrated that Sobolev kernels are characteristic, which ensures that HSIC-ANOVA indices can be used to test input-output independence. Finally, a test procedure will be proposed for the total-order HSIC-ANOVA index, and it will be shown (numerically) that the resulting test of independence is at least as powerful as the standard test (based on two Gaussian kernels).

Joint work with: Amandine Marrel (CEA), Sébastien Da Veiga (ENSAI, formerly SAFRAN), Vincent Chabridon (EDF)


Morgane Menz

Title: Estimation of simulation failure set with active learning based on Gaussian Process classifiers and random set theory
Abstract: Numerical simulator crashes is a well known problem in uncertainty quantification and black-box optimization. These failures correspond to a hidden constraint and might be as costly as a feasible simulation. Hence, we seek to learn the feasible set of inputs in order to target areas without simulation failure. To this end, we will present a Gaussian Process classifiers active learning method based on the Stepwise Uncertainty Reduction paradigm. A strategy to address metamodeling objectives in the presence of hidden constraints based on the previous enrichment criterion is also proposed. The performances of the proposed strategies on an industrial case, concerning the simulation-based estimation of the accumulated damage of a wind turbine subject to several wind loads, will be presented.

Joint work with: Miguel Munoz-Zuniga (IFPEN), Delphine Sinoquet (IFPEN)