Submit

The submission site is opening now. Please click here to submit you result!

Notice:

  • Please strictly follow the latest docker-example here to containerize your algorithms, and follow this tutorial to generate direct download link.
  • Submit page will be available from 11:59 Sept. 19th to 11:59 Sept. 26th.
  • Before Oct. 1st, SEARCH button can only return the status of submitted Docker image.
  • After Oct. 1st, you could SEARCH your highest score by task.
  • After Oct. 1st, a result page will be released, where you can find top10 leaderboard. 

Reminder for submission

  • [updated 09-20] Your docker image name should be:structseg:UserName_taskN. 
  • Your tar file name should be: UserName_taskN.tar
  • The download link for GoogleDrive should be direct link, tutorial here. And also check the download permission:
  •     1. Check the share permission with 'anyone with the link';
  •     2. Get the shareable link;
  •     3. Copy the link to your browser;
  •     4. Click the Download button;
  •     5. Copy the link which ends up with '&export=download', and also with a 'uc?id=1' in the middle of the link.
  • You could submit once per hour (not per day).
  • We strongly advise to submit at least once in this week. This could help find errors of some details like download link, docker and others. This week we send emails one by one to help correct submission errors. 
  • [updated 09-23] We have an automatic system for submission if you use GoogleDrive and Dropbox. The full process including download docker, run docker, run evaluation, is all automatic and very fast. But if you use other drives, especially BaiduPan, we have to download the docker image and run it manually, which takes a lot of time to get the final evaluation scores. Please use GoogleDrive and Dropbox as mush as you can to get faster feedback. 
  • [updated 09-24] LAST 12 HOURS TO GO! 


Participants should containerize their algorithms with Docker and submit these to organizers for evaluation. The test data will not be released to the public. 

The challenge organizers will run your method on all test data. This guarantees that the test data remains secret and cannot be included in the training procedure. This workflow has proven successful for previous MICCAI challenges. (e.g. IVDM3Seg Challenge)

An easy-to-follow example to containerize your algorithms with Docker can be found here

Note: Each top-ranking team will be invited to submit a 3-page report using LNCS and an oral speech at MICCAI 2019 to describe methodology of their approach. All the top-ranking teams will be invited to jointly author a paper on describing the latest progress on automatic OAR and GTV delineation based on the challenge results.

The Submission page will be open from Sept 19th to Sept 26rd, 2019.

Apart from the award certificates and trophies, the top-ranking teams will be invited to publish full papers to describe their top-ranking methods in Neurocomputing (IF: 4.072) and also be awarded free MICCAI registrations. Below are the guidelines for the coming final submissions.

Here's the guide for submission:

How to fill the SUBMIT form:
  • a.TEAM NAME only contains English letters, numbers and underline. Each team only needs to register one ID.
  • b.Fill in the TEAM MEMBERS with all your team members' names and institution email addresses. Third-party email addresses (e.g., hotmail.com, gmail.com, 163.com, etc.) will not be considered as valid submissions.
  • c.One or multiple affiliations can be associated with your team, which could be universities, institutions, or companies that team members are affiliated with.
  • d.USERNAME is your grand challenge ID.
  • e.EMAIL is your own institution email address.
  • f.Fill in the DOWNLOAD LINK with the download link of your docker image. We only accept Google Drive link and DropBox link without any password. Please follow this tutorial. 
When you fill in the SEARCH form:
  • a.USERNAME is your grand challenge ID.
  • b.EMAIL must be the same with your grand challenge account.
Submission rules:
  • a.For each task, each team can have one or multiple members. Each participant can only be affiliated with one team. If a participant appears in multiple teams, all his/her submissions will be revoked.
  • b.A participant can join different teams for the different tasks.
  • c.For each task, the submission interval must be no less than one hour. 
  • d.For each team and each task, only its LAST THREE submissions before the deadline will be evaluated for final ranking. The one with the highest ranking among the three submissions will be used as the final rank.
  • e.Challenge time is UTC-8 Beijing time.
  • f.Submission deadline extends to Sept. 26rd, 2019
  • g.Our evaluation server is equipped with an Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz, 250GB memory and 8-NVIDIA GTX 1080Ti GPUs.
  • h.The algorithm is allowed to execute on the CPU and on a single GPU. It should occupy no more than 10GB GPU memory and no more than 16GB memory.
  • i. For each test case, the algorithm can execute for at most 60120 seconds to generate the output. Otherwise, an error will be returned.
  • j. The required cuda version is cuda 8 or cuda 9 (cuda 10 is not supported!).
Submission feedback:
  • a.For each submission, you could search the docker evaluation status after the evaluation is finished. A search result could be:
  • i.Whether the docker is valid. If it is invalid, it will return error log.
  • ii.Whether the result of the docker is valid. If it is valid (DSC > 5%), it will only return that the submission is valid
  • iii.No specific numeric result will be shown to the participants before the challenge deadline.
  • b.The submission site will be open from 11:59 Sep. 19th to 11:59 Sep. 26th.
  • c.We will inform top-10 teams of each task via emails between Sep. 26th to Oct. 1st.
  • d.On Oct. 1st, the leaderboard will be released, containing top-10 teams of each task. Other team could search their highest score by SERCH form.
  • e.The submission site will be reopened after the challenge in MICCAI2019.
To top-ranking teams:
  • a.The top-ranking teams on each task will be invited to submit full papers on describing their methods to the special issue on Deep learning for Medical Image Computing of Neurocomputing (IF= 4.072). The guest editors are
  • Hongsheng Li (SenseTime Research & CUHK)
  • Shaoting Zhang (SenseTime Research)
  • Dimitris N. Metaxas (Rutgers University)
  • b.The top-ranking teams will be invited to give talks in the challenge ceremony held in conjunct with MICCAI 2019.
  • c.The top-ranking teams will be awarded challenge certificates and challenge trophies.
  • d.The top-ranking teams will be awarded free MICCAI registrations.
  • e.The top-ranking teams will be required to submit a 4-page technical report in Springer LNCS format by Oct. 10th to briefly describe their methodology. An overview paper of the challenge dataset and challenge result will be submitted to a top-ranking journal in medical image understanding. This paper is expected to receive high citations in the future.