UNSURE2024
Uncertainty for Safe Utilization of Machine Learning in Medical Imaging

Recent Updates

Important Dates

  • June 29, 2024 Paper Submission Deadline (23:59 PST)
  • July 10, 2024 Reviews due
  • July 15, 2024 Publication of decisions
  • August 1, 2024 Camera ready submissions due
  • October 10, 2024 Workshop date

About the MICCAI UNSURE Workshop

Overview

With the rise and influence of machine learning (ML) in medical application and the need to translate newly developed techniques into clinical practice, questions about safety and uncertainty over measurements and reported quantities have gained importance. Obtaining accurate measurements is insufficient, as one needs to establish the circumstances under which these values generalize, or give appropriate error bounds for these measures. This is becoming particularly relevant to patient safety as many research groups and companies have deployed or are aiming to deploy ML technology in clinical practice.

The purpose of this workshop is to develop awareness and encourage research on uncertainty modelling to ensure safety for applications spanning both the MIC and CAI fields. In particular, this workshop invites submissions to cover different facets of this topic, including but not limited to: detection and quantification of algorithmic failures; processes of healthcare risk management (e.g. CAD systems); robustness and adaptation to domain shifts; evaluation of uncertainty estimates; defence against noise and mistakes in data (e.g. bias, label mistakes, measurement noise, inter/intra-observer variability). The workshop aims to encourage contributions in a wide range of applications and types of ML algorithms. The use or development of any relevant ML methods are welcomed, including, but not limited to, probabilistic deep learning, Bayesian nonparametric statistics, graphical models and Gaussian processes. We also aim to ensure broad coverage of applications in the context of both MIC and CAI, which are categorized into reporting problems (descriptions of image contents) such as diagnosis, measurements, segmentation, detection, and enhancement problems (addition of information) such as image synthesis, registration, reconstruction, super-resolution, harmonisation, inpainting and augmented display.

Details

In the last few years, machine learning (ML) techniques have permeated many aspects of the MICCAI community, leading to substantial progress in a wide range of applications ranging from image analysis to surgical assistance. However, in medical applications, algorithms ultimately assist life and death decisions, and translation of such innovations into practice requires a measure of safety. In practice, ML systems often face situations where the correct decision is ambiguous, and therefore principled mechanisms for quantifying uncertainty are required to envision potential practical deployment.

Safety is indeed paramount in medical imaging applications, where images inform scientific conclusions in research, and diagnostic, prognostic or interventional decisions in clinics. However, efforts have mostly focused on improving the accuracy, while systematic approaches ensuring safety of medical imaging-derived automated systems are largely lacking in the existing body of research.

Uncertainty quantification has recently attracted attention in the MICCAI community as a promising approach to provide a reliability metric of the output, and as a mechanism to communicate the knowledge boundary of such ML systems. Spurred on by this emergent interest, the workshop will encourage discussions on the topic of uncertainty modelling and alternative approaches for risk management in a wide range of medical applications. It aims thereby to highlight both fundamental and practical challenges that need to be addressed to achieve safer implementations of ML systems in the clinical world.

Format

The UNSURE workshop will be held as an in-person, half-day satellite event of MICCAI 2024 in Marrakech. This year, the workshop will be joined with the Uncertainty Tutorial this year. For more information on the tutorial, please navigate to the dedicated website

Call for Papers

Scope

We accept submissions of original, unpublished work on safety and uncertainty in medical imaging, including (but not limited to) the following areas:

  • Uncertainty quantification in any MIC or CAI applications
  • Risk management of ML systems in clinical pipelines
  • Out-of-distribution and anomaly detection
  • Defending against hallucinations in enhancement tasks (e.g. super-resolution, reconstruction, modality translation)
  • Robustness to domain shifts
  • Measurement errors
  • Modelling noise in data (e.g. labels, measurements, inter/intra-observer variability)
  • Validation of uncertainty estimates
  • Active Learning
  • Confidence bounds
  • Posterior inference over point estimates
  • Bayesian deep learning
  • Graphical models
  • Gaussian processes
  • Calibration of uncertainty measures
  • Bayesian decision theory

Submission Format

Our submission guidelines follow the guidelines of the main MICCAI conference. Submissions must be 8-page papers (excluding references) following the Springer LNCS format. Author names, affiliations and acknowledgements, as well as any obvious phrasings or clues that can identify authors must be removed to ensure anonymity. Note that the 8 page limit refers only to the main content. Including references and acknowledgements the submission may exceed 8 pages.


Analogous to the main conference we also allow supplementary materials in the form of a maximum 2-page PDF with tables, figures or proofs, and captions not exceeding 100 words. Multi-media materials such as videos are also allowed. Please refer to Section 5 of the MICCAI conference submission guidelines for more details.

How to submit?

Please submit papers using the paper submission system.

Submission Deadline: June 29, 2024 - 23:59 (PST)

Publication

We are currently enquiring to ensure we can publish the proceedings as part of an LNCS volume by Springer Nature. Accepted papers will also be invited to submit an extended version for publication in a special issue of the MELBA journal

Presentation

All accepted papers will be presented in person at the UNSURE workshop on 10th October 2024. We will select a number of papers for long oral presentation or spotlights based on the reviewers' suggestions. All authors will additionally get the opportunity to present their work as poster during the workshop poster session. To help participants in making the most of the workshop, accepted papers will be made public for comments on open-Review a week before the workshop along with 6 min presentation videos.

Resources

Proceedings

The full Springer proceedings for the UNSURE 2024 workshop are / will be available: here

Video Recordings

You can find all pre-workshop video presentations of the UNSURE 2024 accepted papers on this Youtube channel.

Presented papers

  • Uncertainty-Aware Bayesian Deep Learning with Noisy Training Labels for Epileptic Seizure Detection, Deeksha Moodasarige Shama et al, paper , video
  • Active Learning for Scribble-based Diffusion MRI Segmentation, Jonathan Lennartz et al., paper, video
  • FISHing in Uncertainty: Synthetic Contrastive Learning for Genetic Aberration Detection, Simon Gutwein et al, paper, video
  • Diagnose with Uncertainty Awareness: Diagnostic Uncertainty Encoding Framework for Radiology Report Generation, Sixing Yan et al, paper, video
  • Making Deep Learning Models Clinically Useful - Improving Diagnostic Confidence in Inherited Retinal Disease with Conformal Prediction, Biraja P Ghoshal et al, paper, video
  • GUARDIAN: Guarding Against Uncertainty and Adversarial Risks in Robot-Assisted Surgeries, Ufaq Jeelani Khan et al, paper, video
  • Quality Control for Radiology Report Generation Models via Auxiliary Auditing Components, Hermione Warr et al, paper, video
  • Conformal Performance Range Prediction for Segmentation Output Quality Control, Anna M Wundram et al, paper, video
  • Holistic Consistency for Subject-level Segmentation Quality Assessment in Medical Image Segmentation, Yizhe Zhang et al, paper, video
  • CROCODILE: Causality aids RObustness via COntrastive DIsentangled LEarning, Gianluca Carloni et al, paper, video
  • Image-conditioned Diffusion Models for Medical Anomaly Detection, Matthew M G Baugh et al, paper, video
  • Information Bottleneck-based Feature Weighting for Enhanced Medical Image Out-of-Distribution Detection, Brayden J. Schott et al, paper, video
  • Beyond Heatmaps: A Comparative Analysis of Metrics for Anomaly Localization in Medical Images, David Zimmerer et al, paper, video
  • Typicality Excels Likelihood for Unsupervised Out-of-Distribution Detection in Medical Imaging, Lemar Abdi et al, paper, video
  • Evaluating Reliability in Medical DNNs: A Critical Analysis of Feature and Confidence-Based OOD Detection, Harry E.J. Anthony et al, paper, video
  • Uncertainty-Aware Vision Transformers for Medical Image Analysis, Franciskus Xaverius Erick et al, paper, video
  • Efficient Precision control in Object Detection Models for Enhanced and Reliable Ovarian Follicle Counting, Vincent Blot et al, paper, video
  • GLANCE: Combating Label Noise using Global and Local Noise Correction for Multi-Label Chest X-ray Classification, Xianze Ai et al, paper, video
  • Conformal Prediction and Monte Carlo Inference for Addressing Uncertainty in Cervical Cancer Screening, Christopher W Clark et al, paper, video
  • INFORMER- Interpretability Founded Monitoring of Medical Image Deep Learning, Shelley Zixin Shu et al , paper, video
-->

Program

Date and Location

Date: October 10th, 2024, 08:00-12:30 Marrakesh Time.
-->

Schedule

Timezone Information: All times below are given in Marrakech Time (UTC+1). Please be mindful of time zone conversions when planning your attendance.

  • 08:00-08:05 Introduction
  • 08:05-09:00 In depth tutorial
    • 08:05-08:32
      General Introduction into uncertainty and domain shift, Vatsal Raina

      08:32-09:00
      Uncertainty for Medical Image Analysis, Natalia Molchanova

  • 09:00-09:30 Lightning tutorial
    • 09:00-09:15
      Model Calibration, Meritxell Bach Cuadra

      09:15-09:30
      Conformal Prediction, Adrian Galdran

  • 08:32-09:00 Spotlights
    • 09:30-09:36
      Making Deep Learning Models Clinically Useful - Improving Diagnostic Confidence in Inherited Retinal Disease with Conformal Prediction, Biraja P. Ghoshal et al.

      09:36-09:42
      Diagnose with Uncertainty Awareness: Diagnostic Uncertainty Encoding Framework for Radiology Report Generation Sixing Yan et al.

      09:42-09:48
      FISHing in Uncertainty: Synthetic Contrastive Learning for Genetic Aberration Detection, Simon Guttwein et al.

      09:48-09:54
      Beyond Heatmaps: A Comparative Analysis of Metrics for Anomaly Localization in Medical Images, David Zimmerer et al.

      09:54-10:00
      GUARDIAN: Guarding Against Uncertainty and Adversarial Risks in Robot-Assisted Surgeries, Ufaq Jeelani Khan et al.

  • 10:00-10:30 Coffee Break + Posters
  • 10:30-11:05 Keynote - The multi-dimensional aspect of uncertainty in automated medical image analysis Prof. Michel Dojat
  • 10:30-11:30 Long Orals Session
    • 11:05-11:25
      Conformal Performance Range Prediction for Segmentation Output Quality Control, Anna M Wundram et al.

      11:25-11:45
      Holistic Consistency for Subject-level Segmentation Quality Assessment in Medical Image Segmentation, Yizhe Zhang et al.

      11:45-12:05
      Evaluating Reliability in Medical DNNs: A Critical Analysis of Feature and Confidence-Based OOD Detection, Harry E.J Anthony et al.

  • 12:05-12:25 Panel Discussion (with Michel Dojat, William M. Wells III, Meritxell Bach Cuadra)
  • 12:25-12:30 Concluding Remarks & Best paper award

Keynote Speaker

Prof. Michel Dojat

Research Director at Inserm and Deputy Scientific Director for digital biology and health at Inria. .

UNSURE 2024 Organizing Committee

Organizers

In alphabetical order

Raghav Mehta

Imperial College London

Cheng Ouyang

University of Oxford

Chen Qin

Imperial College London

Carole Sudre

MRC Unit for Lifelong Health and Ageing, University College London / Centre for Medical Image Computing, University College London / Department of Biomedical Engineering, King's College London

William (Sandy) Wells

Radiology, BWH, Harvard Medical School

Program Committee

  • Matthew Baugh (Imperial College London, United Kingdom)
  • Ishaan Bhat (University Medical Center Utrecht, The Netherlands)
  • Liane Canas (King's College London, United Kingdom)
  • Jacob Carse (University of Dundee, United Kingdom)
  • William Consagra (Harvard Medical School, United States)
  • Herve Delingette (INRIA, France)
  • Leonhard Feiner (Technical University of Munich, Germany)
  • Paul Fischer (University of T\"ubingen, Germany)
  • Daria Frolova (AIRI, Russia)
  • Adrian Galdran (Universitat Pompeu Fabra, Spain)
  • Tristan Glatard (Concordia University, Canada)
  • Dimitri Hamzaoui (INRIA, France)
  • Chloe He (University College London, United Kingdom)
  • Yipeng Hu (University College London, United Kingdom)
  • Mobarakol Islam (University College London, United Kingdom)
  • Leo Joskowicz (Hebrew University of Jerusalem, Israel)
  • Benjamin Lambert (Institut des Neurosciences de Grenoble, France )
  • Max-Heinrich Laves (Leibniz Universitat Hannover, Germany)
  • Hongwei Li (MGH/Harvard Medical School, United States)
  • Hongxiang Lin (Zheijiang Lab, China)
  • Xinzhe Luo (Imperial College London, United Kingdom)
  • Stephen J. McKenna (University of Dundee, United Kingdom)
  • Prerak Mody (Leiden University Medical Centre, Belgium)
  • Johanna M\"uller (Friedrich-Alexander Universit\"at Erlangen-N\"urnberg, Germany)
  • Balamurali Murugesan (Ecole de technologie superieure de Montreal, Canada)
  • Jacob Peoples (Queen's University, Canada )
  • Di Qiu (The Chinese University of Hong Kong, China)
  • Lawrence Schobs (University of Sheffield, United Kingdom)
  • Amritpal Singh (Emory University, United States)
  • Fons van der Sommen (Eindhoven University of Technology, The Netherlands)
  • Prasad Sudhakar (GE Healthcare)
  • Shishuai Wang (Erasmus Medical Center, The Netherlands)
  • McKell Woodland (Rice University, United States)
  • Yiming Xiao (Concordia University, Canada)
  • Evan Yu (Cornell University, United States)
  • Jinwei Zhang (Cornell University, United States)
  • Contact

    For general inquiries please send an email to: unsure2024@ucl.ac.uk