CHAOS Banner
Join

CHAOS - Combined (CT-MR) Healthy Abdominal Organ Segmentation

CHAOS challenge aims the segmentation of abdominal organs (liver, kidneys and spleen)  from CT and MRI data. CHAOS was held in The IEEE International Symposium on Biomedical Imaging (ISBI) on April 11, 2019, Venice, ITALY. The results of the challenge were published: https://chaos.grand-challenge.org/Results_CHAOS/ Also detailed analyses of these results have been published in the challenge article. You may find further information on Publications and Citation page.

If you would like to attend to CHAOS but missed it, do not worry. It is possible to join CHAOS and submit results online. The up-to-date scores are being published on our Results page.  You may read the information on this website and join our Google group. https://groups.google.com/forum/#!forum/chaos-challenge

Important note: Ground truths of the test data will never be shared publicly.


Challenge Description

Understanding prerequisites of complicated medical procedures plays an important role in the success of the operations. To enrich the level of understanding, physicians use advanced tools such as three-dimensional visualization and printing, which require extraction of the object(s) of interest from DICOM images. Accordingly, the precise segmentation of abdominal organs (i.e. liver, kidney(s) and spleen) has critical importance for several clinical procedures including but not limited to pre-evaluation of liver for living donor-based transplantation surgery or detailed analysis of abdominal organs to determine the vessels arising from and entering them for correct positioning of a graft prior to abdominal aortic surgery. This motivates ongoing research to achieve better segmentation results and overcoming countless challenges originating from both highly flexible anatomical properties of abdomen and limitations of modalities reflected to image characteristics. In this context, the proposed challenge has two separate but related aims:

  1. Segmentation of liver from computed tomography (CT) data sets, which are acquired at portal phase after contrast agent injection for pre-evaluation of living donated liver transplantation donors.

  2. Segmentation of four abdominal organs (i.e. liver, spleen, right and left kidneys) from magnetic resonance imaging (MRI) data sets acquired with two different sequences (T1-DUAL and T2-SPIR).

CHAOS tasks contain combination of these organs' segmentation.

Tasks

 There are five competition categories in which the participating teams can take place and submit their result(s):

  1. Liver Segmentation (CT & MRI): This is also called "cross-modality" [1] and it is simply based on using a single system, which can segment liver from both CT and MRI. For instance, the training and test sets of a machine learning approach would have images from both modalities without explicitly feeding the model with corresponding information. A unique study about this is a reference below and this task is one of the most interesting tasks of the challenge. Keep in mind that the fusion of individual systems for different modalities (i.e. two models, one working on CT and the other on MRI ) would not be valid for this category. They can be evaluated as individual systems at Tasks 2 and 3. On the other hand, in this task, fusion of individual systems between MR sequences (i.e. two models, one working on T1-DUAL and the other on  T2-SPIR ) is allowed.

  2. Liver Segmentation (CT only): This is mostly a regular task of liver segmentation from CT, (such as SLIVER07). This task is easier than SLIVER07 as it only contains healthy livers aligned in the same direction and patient position. However, the challenging part is the enhanced vascular structures (portal phase) due to the contrast injection.

  3. Liver Segmentation (MRI only): Similar to "Task 2", this is also a regular task of liver segmentation from MRI. It includes two different pulse sequences: T1-DUAL and T2-SPIR. Moreover, T1-DUAL has two forms (in and out phase). The developed system should work on both sequences. In this task, the fusion of individual systems between MR sequences (i.e. two models, one working on T1-DUAL and the other on  T2-SPIR ) are allowed.

  4. Segmentation of abdominal organs (CT & MRI): This task is extension of Task 1 to kidneys and spleen in MRI data. In this task, the interesting part is that CT datasets have only liver, but the MRI datasets have four annotated abdominal organs (liver, kidneys, spleen). Keep in mind that fusion of individual systems for different modalities (i.e. two models, one working on CT and the other on MRI ) would not be valid for this category. On the other hand, in this task, fusion of individual systems between MR sequences (i.e. two models, one working on T1-DUAL and the other on  T2-SPIR ) are allowed.

  5. Segmentation of abdominal organs (MRI only): The same task given in "Task 3" but extended to four abdominal organs; liver, kidneys, spleen. In this task, ensemble or fusion of individual systems between MR sequences (i.e. two models, one working on T1-DUAL and the other on  T2-SPIR ) are allowed.  

    [1] Valindria, V. et al. (2018, March). Multi-modal learning from unpaired images: Application to multi-organ segmentation in CT and MRI. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) (pp. 547-556). IEEE. https://doi.ieeecomputersociety.org/10.1109/WACV.2018.00066

    Purposes of the Tasks

    Due to above mentioned significance of the problem, the challenge will serve several purposes:

    • In the last decade, not only the number of segmentation methods increased significantly, but also applicability is aimed to be extended to multiple segmentation tasks (For instance, a single deep network, which can segment multiple organs [Task 4] or more rarely, a system that can segment the same organ(s) from different modalities [Task 1 and 5]).

    • On the other hand, it is known that many new architectures significantly depend on the training data set, which yields poor generalization capability. Besides physicians, who are the final user of the medical segmentation methods, even experienced researchers still struggle at choosing appropriate techniques or tweaking optimal model parameters for a particular problem.

    • Thus, by providing data sets from two different modalities, the participants are encouraged to develop a system that would work on both.

    How to join and submit to CHAOS?

    Since the challenge contains human data, there are some critical rules for the CHAOS challenge according to ethical permissions:

    1) Please read rules of the challenge. All participants are considered to have read and accepted the rules.

    2) Create an account on grand-challenge.org website with your official mail account. Make a join request for the challenge by using the Join link.

    3) Download the data, submission manual, and template from Download page.

    4) Save your result in template file and prepare unique paper (PDF file) that is mentioned in Rules.

    5) Submit your results. After a short time, the scores will be available at Results page.

    Citation

    In your works, please give appropriate credit, provide a link to the license, and indicate if changes were made. The citation information can be found in Publications and Citation page.

    Last update: 11/12/2019