Estimating and Explaining Predictive Uncertainty in Machine Learning

Résumé

Although Machine Learning (ML) models have proved to be useful in key applications, they still have significant limitations, such as sensitivity to distribution shifts. Therefore, it is essential to provide trustworthy monitoring methods when ML models make predictions on new unlabeled data. For an ML model, predictive uncertainty is defined as the uncertainty of predictions, which can be caused by the data or the model. As they identify insufficient knowledge, uncertainty estimates can thus be employed to monitor ML models' predictions and make decision-making safer and more reliable in applications such as financial risk assessment or medical diagnosis. In this thesis, we investigate the problem of estimating and explaining predictive uncertainty in ML. We introduce various methods to improve the quantification and explanation of predictive uncertainty in classification or regression tasks and through different settings such as multimodality or distribution shifts. In particular, we introduce a method to produce adaptive prediction intervals in regression tasks. To address classification tasks under distribution shifts, we first propose a method to quantify and visualize predictive uncertainty with simple representations. We then extend the research to a multimodal Transformer which quantifies uncertainty by design. Also, this architecture makes it possible to explain predictive uncertainty in terms of feature values. As uncertainty quantification is closely related to unsupervised performance estimation, we then introduce a method for practitioners to assess the model's performance in out-of-distribution contexts. Lastly and as a first step toward trustworthy ML, we introduce a risk-based approach and a reporting tool which aim to produce safe machine learning lifecycles.

Citer ce document

Accès au document

Ce document est hébergé sur son dépôt institutionnel d'origine.

Auteur(s)

Statistiques

Consultations : 3

Téléchargements : 0