An International Challenge to Use Artificial Intelligence for Kidney and Kidney Tumour Segmentation in CT Imaging - European Medical Journal


An International Challenge to Use Artificial Intelligence for Kidney and Kidney Tumour Segmentation in CT Imaging

| Urology
*Nicholas Heller,1 Sean McSweeney,1 Matthew Peterson,1 Sarah Peterson,2 Jack Rickman,1 Bethany Stai,1 Resha Tejpaul,1 Makinna Oestreich,1 Paul Blake,1 Joel Rosenberg,1 Keenan Moore,3 Edward Walczak,1 Zachary Rengel,1 Zach Edgerton,1 Ranveer Vasdev,1 Arveen Kalapara,1 Niranjan Sathianathen,1 Nikolaos Papanikolopoulos,1 Christopher Weight1

The authors report grants from NIH R01CA225435 during the conduction of the study.


Research reported in this publication was supported by the National Cancer Institute (NCI) of the National Institutes of Health (NIH) under Award Number R01CA225435. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. Additional funding was provided by the Climb 4 Kidney Cancer foundation, and a cash prize of $5,000 USD for the challenge’s first place team was sponsored by Intuitive Surgical Inc.

EMJ Urol. ;8[Suppl 2]:25-26. Abstract Review No: AR2.

Each article is made available under the terms of the Creative Commons Attribution-Non Commercial 4.0 License.

Three dimensional (3D) semantic segmentation is a powerful tool for the radiomic characterisation of lesions in cross-sectional imaging; however, its use is limited because of the extensive manual effort required to accurately delineate regions of interest. Artificial intelligence methods based on ‘deep learning’ have had enormous success at automating semantic segmentation tasks, but they rely on large datasets with high-quality manual segmentations from which to learn.1 The scarcity of publicly available CT and MRI segmentation datasets have slowed the development of automatic segmentation systems for 3D medical imaging data. The 2019 Kidney and Kidney Tumor Segmentation challenge2 (KiTS19) was an international competition held in conjunction with the International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI) that sought to stimulate progress on this automatic segmentation frontier.

This competition was based around a first-of-its-kind dataset of 300 patients with preoperative CT imaging and corresponding high-quality 3D segmentation labels for the depicted kidneys and tumours. Of these cases, 210 were publicly released at the start of the challenge for teams to develop their systems. The imaging alone for the remaining 90 cases was released prior to a 2-week submission period, during which teams used their systems to automatically segment the kidney and tumour regions. Those automatic segmentations were then submitted to an online platform, which measured each team’s performance in terms of their Sørensen–Dice coefficient with the manually produced segmentations and ranked them on a public leader board. A total of 106 teams from five continents competed for a $5,000 USD cash prize sponsored by Intuitive Surgical Inc. Teams were required to also submit a manuscript detailing the methods they used to build and train their systems. These, in conjunction with the objective and standardised comparison of their performance, allow for rapid progress in artificial intelligence research and democratise the systems it produces.

The winning method,3 led by researchers at the German Cancer Research Center, Heidelberg, Germany, achieved a Sørensen–Dice coefficient of 0.974 for the kidney regions and 0.851 for the tumours, approaching the human inter-annotator performance on kidneys (0.983), but falling short on tumours (0.923). This challenge has now entered an ‘open leader board’ phase, in which researchers can continue to develop new systems and submit them to the online platform to be ranked on the leader board.

The results of the KiTS19 challenge show deep learning methods are fully capable of reliable segmentation of kidneys and kidney tumours. Fully segmented kidneys and tumours allow for automated extraction of all types of nephrometry scores,4 as well as tumour textural features and an opportunity to accelerate the discovery of new predictive features important for personalised medicine and accurate prediction of patient-relevant outcomes. The KiTS19 challenge attracted the highest number of submissions at MICCAI 2019 and serves as an important and challenging benchmark in automatic 3D segmentation.

Litjens G et al. A survey on deep learning in medical image analysis. Med Image Anal. 2017;42:60-88. Heller N et al. The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: results of the KiTS19 Challenge. 2019. Available at: Last accessed: 31 March 2019. Isensee F, Maier-Hein KH. An attempt at beating the 3D U-Net. 2019. Available at: Last accessed: 31 March 2019. Blake P et al. Automatic RENAL nephrometry scoring using machine learning. Eur Urol. 2019;18(1):e904-5.