UNSURE2020
Uncertainty for Safe Utilization of Machine Learning in Medical Imaging

Recent Updates

  • October 2020: Videos of the talks have been uploaded to our Youtube channel
  • September 2020: Program available below
  • July 2020: Deadline extended to July 12
  • April 2020: Website Up! Please find out 2019 event website here

Important Dates

  • July 12, 2020 Paper Submission Deadline (Extended from July 5)
  • August 5, 2020 Reviews due
  • August 15, 2020 Final decisions
  • August 20, 2020 Camera ready papers due
  • October 8, 2020 Workshop date

About the MICCAI UNSURE Workshop

Overview

With the rise and influence of machine learning (ML) in medical application and the need to translate newly developed techniques into clinical practice, questions about safety and uncertainty over measurements and reported quantities have gained importance. Obtaining accurate measurements is insufficient, as one needs to establish the circumstances under which these values generalize, or give appropriate error bounds for these measures. This is becoming particularly relevant to patient safety as many research groups and companies have deployed or are aiming to deploy ML technology in clinical practice.

The purpose of this workshop is to develop awareness and encourage research on uncertainty modelling to ensure safety for applications spanning both the MIC and CAI fields. In particular, this workshop invites submissions to cover different facets of this topic, including but not limited to: detection and quantification of algorithmic failures; processes of healthcare risk management (e.g. CAD systems); robustness and adaptation to domain shifts; evaluation of uncertainty estimates; defence against noise and mistakes in data (e.g. bias, label mistakes, measurement noise, inter/intra-observer variability). The workshop aims to encourage contributions in a wide range of applications and types of ML algorithms. The use or development of any relevant ML methods are welcomed, including, but not limited to, probabilistic deep learning, Bayesian nonparametric statistics, graphical models and Gaussian processes. We also aim to ensure broad coverage of applications in the context of both MIC and CAI, which are categorized into reporting problems (descriptions of image contents) such as diagnosis, measurements, segmentation, detection, and enhancement problems (addition of information) such as image synthesis, registration, reconstruction, super-resolution, harmonisation, inpainting and augmented display.

Details

In the last few years, machine learning (ML) techniques have permeated many aspects of the MICCAI community, leading to substantial progress in a wide range of applications ranging from image analysis to surgical assistance. However, in medical applications, algorithms ultimately assist life and death decisions, and translation of such innovations into practice requires a measure of safety. In practice, ML systems often face situations where the correct decision is ambiguous, and therefore principled mechanisms for quantifying uncertainty are required to envision potential practical deployment.

Safety is indeed paramount in medical imaging applications, where images inform scientific conclusions in research, and diagnostic, prognostic or interventional decisions in clinics. However, efforts have mostly focused on improving the accuracy, while systematic approaches ensuring safety of medical imaging-derived automated systems are largely lacking in the existing body of research.

Uncertainty quantification has recently attracted attention in the MICCAI community as a promising approach to provide a reliability metric of the output, and as a mechanism to communicate the knowledge boundary of such ML systems. Spurred on by this emergent interest, the workshop will encourage discussions on the topic of uncertainty modelling and alternative approaches for risk management in a wide range of medical applications. It aims thereby to highlight both fundamental and practical challenges that need to be addressed to achieve safer implementations of ML systems in the clinical world.

Call for Papers

Scope

We accept submissions of original, unpublished work on safety and uncertainty in medical imaging, including (but not limited to) the following areas:

  • Uncertainty quantification in any MIC or CAI applications
  • Risk management of ML systems in clinical pipelines
  • Defending against hallucinations in enhancement tasks (e.g. super-resolution, reconstruction, modality translation)
  • Robustness to domain shifts
  • Measurement errors
  • Modelling noise in data (e.g. labels, measurements, inter/intra-observer variability)
  • Validation of uncertainty estimates
  • Active Learning
  • Confidence bounds
  • Posterior inference over point estimates
  • Bayesian deep learning
  • Graphical models
  • Gaussian processes
  • Calibration of uncertainty measures
  • Bayesian decision theory

Submission Format

Submissions must be 8-page papers (excluding references) following the Springer LNCS format. Author names, affiliations and acknowledgements, as well as any obvious phrasings or clues that can identify authors must be removed to ensure anonymity. Note that in contrast to the requirements to the main MICCAI conference, the 8 page limit refers only to the main content. Including references and acknowledgements the submission may exceed 8 pages.

How to submit?

Please submit papers using the paper submission system.

Publication

All accepted papers will be published as part of the MICCAI Satellite Events joint LNCS proceedings to be published by Springer Nature.

Presentation

All accepted papers will be presented virtually at the UNSURE workshop on 8. October 2020. We will select a number of papers for long oral presentation based on the reviewers' suggestions. All authors will additionally get the opportunity to present their work as poster during the workshop poster session.

Resources

Best paper award

The UNSURE 2020 best paper award was presented to Max-Heinrich Laves and colleagues for their work: Uncertainty Estimation in Medical Image Denoising with Bayesian Deep Image Prior.

Best paper runner-up

The runner-up award for best paper was presented to Mark S Graham and collegues for their work: Hierarchical Brain Parcellation with Uncertainty.

Proceedings

The full Springer proceedings for the UNSURE 2020 workshop are available here: https://link.springer.com/book/10.1007/978-3-030-60365-6

Video Recordings

You can find all video recordings of the talks held at UNSURE2020 on this Youtube channel.

Program

Schedule

Timezone Information: All times below are given in UTC Time. Please be mindful of time zone conversions when planning your attendance.
  • 10:00-10:10 Welcoming Words
  • 10:10-11:00 Keynote - Yarin Gal
  • 11:00-11:15 Break
  • 11:15-11:40 Oral Session 1 (Spotlight talks)
    • 11:15-11:20
      "Image registration via stochastic gradient Markov chain Monte Carlo", D. Grezch et al.

      11:20-11:25
      "Uncertainty estimation in landmark localization abased on Gaussian heatmaps", C. Payer et al.

      11:25-11:30
      "Weight averaging impact on the uncertainty of retinal aretery-venous segmentation", A. Garifullin et al.

      11:30-11:35
      "Improving reliability of clinical models using prediction calibration", J. J. Thiagarajan et al.

      11:35-11:40
      "Uncertainty estimation for assessment of 3D US Scan adequacy and DDH metric reliability", A. Kannan et al.
  • 11:40-12:00 Break
  • 12:00-13:00 Oral Session 2
    • 12:00-12:15
      "RevPHISeg: A Memory Efficient Neural Network for Uncertatiny Quantification in Medical Image Segmentation", M. Gantenbein et al.

      12:15-12:30
      "Hierarchical brain parcellation with uncertainty", M. S. Graham et al.
      12:30-12:45
      "Quantitative comparision of Monte Carlo Dropout Uncertainty Measures for Multi-Class Segmentation", R. Camarasa et al.
      12:45-13:00
      Q&A Session
  • 13:00-14:00 Break
  • 14:00-14:40 Oral Session 3
    • 14:00-14:15
      "Improving pathological distribution measurements with Bayesian uncertainty", K. H. Tam et al.

      14:15-14:30
      "Uncertainty Estimation in Medical Image Denoising with Bayesian Deep Image Prior", M. H. Laves et al.
      14:30-14:40
      Q&A Session
  • 14:40-16:30 Poster Session
    • Poster Room A

    • Poster A1: RevPHISeg: A Memory Efficient Neural Network for Uncertatiny Quantification in Medical Image Segmentation -- M. Gantenbein et al.
    • Poster A2: Hierarchical brain parcellation with uncertainty -- M. S. Graham et al.
    • Poster A3: Quantitative comparision of Monte Carlo Dropout Uncertainty Measures for Multi-Class Segmentation -- R. Camarasa et al.
    • Poster A4: Improving pathological distribution measurements with Bayesian uncertainty -- K. H. Tam et al.
    • Poster A5: Uncertainty Estimation in Medical Image Denoising with Bayesian Deep Image Prior -- M. H. Laves et al.
    • Poster Room B

    • Poster B1: Image registration via stochastic gradient Markov chain Monte Carlo -- D. Grezch et al.
    • Poster B2: Uncertainty estimation in landmark localization based on Gaussian heatmaps -- C. Payer et al.
    • Poster B3: Weight averaging impact on the uncertainty of retinal aretery-venous segmentation -- J. J. Thiagarajan et al.
    • Poster B4: Improving reliability of clinical models using prediction calibration -- A. Garifullin et al.
    • Poster B5: Uncertainty estimation for assessment of 3D US Scan adequacy and DDH metric reliability - A. Kannan et al.
  • 16:30-17:00 Break
  • 17:00-17:50 Keynote - Herve Delingette
  • 17:50-18:00 Concluding Remarks

Keynote Speakers

Yarin Gal

Oxford

Hervé Delingette

INRIA Asclepios

UNSURE2020 Organizing Committee

Organizers

Tal Arbel

McGill Centre for Intelligent Machines, McGill University

Christian Baumgartner

Guest Researcher at Computer Vision Lab, ETH Zurich

Adrian Dalca

CSAIL, MIT and
MGH, Harvard Medical School

Carole Sudre

Department of Biomedical Engineering, King's College London

Ryutaro Tanno

Department of Computer Science, University College London

Koen Van Leemput

Technical University of Denmark and
Harvard Medical School

William (Sandy) Wells

Radiology, BWH, Harvard Medical School

Program Committee

  • Alejandro Granados (King's College London)
  • Alireza Sedghi (Queen's University)
  • Daniel Coelho (Imperial College London)
  • Dennis Madsen (University of Basel)
  • Evan Yu (Cornell University)
  • Felix Bragman (King's College London)
  • Hongxiang Lin (University College London)
  • Ivor Simpson (University of Sussex)
  • Jinwei Zhang (Cornell University)
  • Jorge Cardoso (King's College London)
  • Kerem Can Tezcan (ETH Zurich)
  • Leo Joskowicz (Hebrew University of Jerusalem)
  • Liane Canas (King's College London)
  • Lucas Fidon (King's College London)
  • Mark Graham (King's College London)
  • Max-Heinrich Laves (Leibniz Universitat Hannover)
  • Nalini Singh (Massachusetts Institute of Technology)
  • Pedro Borges (King's College London)
  • Pieter Van Molle (Ghent University)
  • Raghav Mehta (McGill University)
  • Richard Shaw (King's College London)
  • Roger Soberanis-Mukul (Technische Universitat Munchen)
  • Tanya Nair (Imagia)
  • Thomas Varsavsky (King's College London)
  • Tim Adler (DKFZ)
  • Yukun Ding (University of Notre-Dame)
-->

Contact

For general inquiries please send an email to: unsure@mit.edu