News & FAQ
You may find current updates and answers of frequently asked questions in this page.
CHAOS is featured at the December issue of Computer Vision News Magazine!
1) Is it possible to join and submit result to CHAOS?
Yes, online submission is opened. You may find how to submit your results on our download page.2) Why my "join request" is rejected or not accepted?
The challenge contains human data which need certain ethical permissions. Only educational and/or non-commercial usage is allowed. Therefore, we are trying to avoid anonymous users in order to keep a true record of interested participants. That is why using official e-mail account of the university/institution/company is strongly recommended. If the organizers cannot verify your e-mail address on a related web site, then the access to the data set is not granted.
3) Will the dataset available in only challenge day in ISBI?
No. A similar approach in previous grand challenges, such as SLIVER07 is preferred. The dataset and the challenge is still available now. New participants may submit their results to us for evaluation. Also previous participants have the option to update their new results.The actual scores is being published in our leaderboard.
4) Can I obtain the ground truths of the test data?
No. The ground truth of the test data will never be published. Only training sets include ground truths.
5) Is it possible to use CHAOS data in another academic study?
It is not possible at the moment. However, it will be possible after publication of the challenge's article. The data will be able to use via citing the article.
6) Is it possible to download evaluation code?
It will be possible after publication of the challenge's article. The code will be able to use via citing the article.
7) When can I learn result of my submission?
The evaluation is made by organizers manually. Announcement of evaluation takes less than one day in most cases. However it may take a few days depends on the availability. Please be patient and do not send several mails.
8) I want to submit my results for evaluation but I do not want them published on the leaderboard. Can you only send my scores to me via e-mail?
No it is not possible. Each evaluated submission will be published on the leaderbord regardless of score.
9) I do not like my score (or one of my score) and I want it to be cleared from the leaderboard. Is it possible?
Due to the rules, every evaluated submission will be published online so it is not possible.
10) My DICE scores are very high however my final score is lower. How is it possible? Is the evaluation fair and correct? Why do you use many metrics? I think DICE is enough, isn't it?
Segmentation of medical imaging is used for clinical operations. Hence tolerance of error is fewer for clinical aspects. There is no single metric that evaluates 3D segmented data completely and fairly in terms of clinically acceptable results. That is why combination of different metrics is used for calculating the final score. This approach was used and is still being used in many respectable segmentation challenges.