Dear @BraTSLighthouse2025Participants The @BraTSChallengeOrganizers are excited to announce that validation submissions queues are now **open **for the following challenges: * Task 1: Pre- and Post-Treatment Adult Glioma * Task 2: Pre-Treatment intracranial Meningioma * Task 3: Pre-Radiotherapy intracranial Meningioma * Task 4: Pre- and Post-Treatment Brain Metastases * Task 5: Brain Glioma in the underserved sub-Saharan African patient population * Task 6: Pre-Treatment Pediatric Tumor Patients in partnership with multiple related societies * Task 7: Generalizability of Segmentation Methods Across Tumors * Task 9: MRI Local Inpainting * Task 11: Predicting the Tumor Response During Therapy To submit the outputs of your solutions, please follow the [Submission Tutorial](https://www.synapse.org/Synapse:syn64153130/wiki/632674). Just a heads-up: You are allowed up to **2** scored submissions a day. This means that invalid submissions won't count against this limit. Your daily submission allowance resets at 00:00:00 UTC each day. The validation submission queues will be closed on **Sunday, July 20th, at 23:59 UTC**. Please note that the submission queues for the following two tasks will be opened soon: * Task 8: MRI Global Synthesis * Task 10: Assessing the Heterogeneous Histologic Landscape of Glioma Announcements regarding their availability will be disseminated via this threading channel soon. Furthermore, please note that eligibility for participation in the final testing phase necessitates the submission of a short paper by Sunday, July 20th, 23:59 UTC. We encourage all participants to follow the [official timeline](https://www.synapse.org/Synapse:syn64153130/wiki/) to ensure adherence to all stipulated deadlines. We extend our best wishes for your success in the challenge!

Created by Mehdi Astaraki astaraki
Hi @vchung, We followed your suggestions and it worked. We were able to successfully push the image to the synapse project.
Hi @vchung, thank you for your help. I'll try these out and let you know if I face some issues.
Hi @SatyaM, Quick update: the Synapse platform team has identified the issue and is currently working on a fix. In the meantime, they suggest the following workarounds so you can submit your Docker image now: * downgrade to an older version of docker (>= v27.1.2). For example, you can downgrade to Docker Desktop v4.31.0, which includes Docker version v26.1.4 * disable "containerd" in Docker Desktop: https://docs.docker.com/desktop/features/containerd/#enable-the-containerd-image-store After trying either of these, please rebuild your Docker image, then run this command: ``` docker manifest inspect YOUR_DOCKER_IMAGE | grep -m 1 "mediaType" ``` If the mediaType returned is `application/vnd.docker.distribution.manifest.v2+json`, then you are good to go! Push up your Docker image then you should be able to see it within your project. Hope this helps!
Hi @vchung, I've added you to my project (syn68718904). Thank You for your time. Please let me know if anything else is required from my side.
Hi @SatyaM , Thank you for sharing your commands. This definitely sounds like a bug with the platform, in that your successfully pushed Docker images will not display in the **Docker** tab. In order to issue a bug report to the platform team, do you mind sharing your Synapse Project with me as well, so that I can provide screenshots? We will work to resolve this ASAP.
Hi @Miriam_C, Yes, the submission portal for the validation phase will be open by this evening and will accept submissions until 31st of July. Best. /Mehdi
Hi, will the validation submission queues be opened again, as the submission deadline was postponed to 31st July? Best regards
Hi @vchung, sorry for the confusion, I was pushing the docker image using the codes below. All of the codes ran to completion. However, when I go to synapse within the same project (syn68718904), I cannot see any docker repo in the docker tab ! I am not able to figure out what am I missing here that is causing the problem. ``` docker build -t brats-submission:latest . # For building the docker image docker login docker.synapse.org # Logging into docker (I am using my username and the Personal Access Tokens for this) docker tag brats-submission:latest docker.synapse.org/syn68718904/submission1:latest # Tagging the docker image docker push docker.synapse.org/syn68718904/submission1:latest # Pushing the docker image to synapse. ``` I added the moderator of my challenge task to the same project, and he was able to push an image to it which was visible, but am not able to do the same myself for some reasons. Thank you for your time.
Hi @SatyaM , I can help you with the Synapse-related issues. To clarify: you were initially able to push your Docker image to Synapse, but now you are encountering problems. When you go to your Synapse project and click on the **Docker** tab, do you still see the image you successfully pushed before? What is the error message you receive when you try to push now? Let me know
Hi @astaraki, I was participating in the task 11 of the challenge. I have prepared all my codes and the necessary artifacts required by the code to get the model predictions, and containerized the same in Docker. I sucessfully pushed the created image to docker.synapse using the correct project id. I can see from my terminal output that the push was successful. However, when I open the same project after logging in, I am not able to pushed docker image. I tried multiple times with the original inference code files as well as with some dummy container, both of which were unsuccessful. Kindly help us resolve this issue.
Hi @DarylWM, as Mehdi said, you should be able to find all information on the challenge site or the accompanying [GitHub repo](https://www.synapse.org/Synapse:syn64153130/discussion/threadId=11930?replyid=34928). For task 11, you need to submit an algorithm already during validation. Let me know if there is anything left unclear. Best, Yannick
Hi @DarylWM, Please follow the instructions for the [progression challenge](https://www.synapse.org/Synapse:syn64153130/wiki/631459) regarding the structure of submission files. Best regards, /Mehdi
Hi @astaraki, there is no Submission Tutorial (Files) for Task 11. Could that be provided?
Hi @astaraki Yes. I already registered and had several submissions before. I reregistered using the link you shared and could submit to the peds challenge again. Thank you
Hi @the_heavy_shadow Have you already registered for the BraTS 2025 challenges? Please read the instructions [here](https://www.synapse.org/Synapse:syn64153130/wiki/631723). /Mehdi
On the submission dialog, I only see one option: "BraTS 2023: Segmentation - Adult Glioma". However, I need to submit to the peds challenge. I could submit my files to the challenge before. Is there a problem on the server side or am I missing sth?
Hi @lulululu , Yes, your submitted files are fine! The previous errors you received were due to some recent changes made to the access settings for previous years' BraTS data. Once those access errors were resolved, we re-ran your submissions that were processed incorrectly. Apologies for that!
Dear organizers, We are trying to submit to the "Task 9: MRI Local Inpainting" queue. To the best of our knowledge, we follow the provided submission instructions. However, we keep getting an email that the workflow failed & this error in the log file (only last part of it copied here): ``` ... STDERR: 2025-06-30T12:10:47.913890380Z SynapseUnmetAccessRestrictions: STDERR: 2025-06-30T12:10:47.913894410Z This entity has access restrictions. Please visit the web page for this entity (syn.onweb("syn51514110")). Click the downward pointing arrow next to the file's name to review and fulfill its download requirement(s). STDERR: 2025-06-30T12:10:47.913899040Z STDERR: 2025-06-30T12:10:47.913902960Z STDERR: 2025-06-30T12:10:47.913906820Z [job synapse-get] Max memory used: 22MiB STDERR: 2025-06-30T12:10:47.913910800Z INFO:cwltool:[job synapse-get] Max memory used: 22MiB STDERR: 2025-06-30T12:10:47.913914810Z [job synapse-get] Job error: STDERR: 2025-06-30T12:10:47.913918870Z ("Error collecting output for parameter 'filepath':\nhttps://raw.githubusercontent.com/Sage-Bionetworks-Workflows/cwl-tool-synapseclient/v1.4/cwl/synapse-get-tool.cwl:42:7: Did not find output file with glob pattern: '['*']'", {}) STDERR: 2025-06-30T12:10:47.913923570Z ERROR:cwltool:[job synapse-get] Job error: STDERR: 2025-06-30T12:10:47.913928200Z ("Error collecting output for parameter 'filepath':\nhttps://raw.githubusercontent.com/Sage-Bionetworks-Workflows/cwl-tool-synapseclient/v1.4/cwl/synapse-get-tool.cwl:42:7: Did not find output file with glob pattern: '['*']'", {}) STDERR: 2025-06-30T12:10:47.913932970Z [job synapse-get] completed permanentFail STDERR: 2025-06-30T12:10:47.913936930Z WARNING:cwltool:[job synapse-get] completed permanentFail STDERR: 2025-06-30T12:10:47.913940930Z Traceback (most recent call last): STDERR: 2025-06-30T12:10:47.913944931Z File "/usr/local/lib/python3.8/site-packages/toil/worker.py", line 366, in workerScript STDERR: 2025-06-30T12:10:47.913949191Z job._runner(jobGraph=jobGraph, jobStore=jobStore, fileStore=fileStore, defer=defer) STDERR: 2025-06-30T12:10:47.913953291Z File "/usr/local/lib/python3.8/site-packages/toil/job.py", line 1392, in _runner STDERR: 2025-06-30T12:10:47.913957471Z returnValues = self._run(jobGraph, fileStore) STDERR: 2025-06-30T12:10:47.913961451Z File "/usr/local/lib/python3.8/site-packages/toil/job.py", line 1329, in _run STDERR: 2025-06-30T12:10:47.913965651Z return self.run(fileStore) STDERR: 2025-06-30T12:10:47.913969591Z File "/usr/local/lib/python3.8/site-packages/toil/cwl/cwltoil.py", line 937, in run STDERR: 2025-06-30T12:10:47.913973771Z raise cwltool.errors.WorkflowException(status) STDERR: 2025-06-30T12:10:47.913977751Z cwltool.errors.WorkflowException: permanentFail STDERR: 2025-06-30T12:10:47.913981711Z ERROR:toil.worker:Exiting the worker because of a failed job on host bd3b8032533e STDERR: 2025-06-30T12:10:47.913985901Z WARNING:toil.jobGraph:Due to failure we are reducing the remaining retry count of job 'https://raw.githubusercontent.com/Sage-Bionetworks-Workflows/cwl-tool-synapseclient/v1.4/cwl/synapse-get-tool.cwl#synapse-get' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks-Workflows_cwl-tool-synapseclient_v1.4_cwl_synapse-get-tool.cwl_synapse-get/instance-wm6n99bn with ID kind-https_raw.githubusercontent.com_Sage-Bionetworks-Workflows_cwl-tool-synapseclient_v1.4_cwl_synapse-get-tool.cwl_synapse-get/instance-wm6n99bn to 0 STDERR: 2025-06-30T12:10:47.913994991Z <========= ``` Can you help us fix it? Our submission IDs are 9753584, 9753581 and 9753577 (same errors). Thank you in advance! Best regards EDIT: we now got the scores, thank you! We assume this means that there was no fundamental problem with our submitted files?
Hi @astaraki, Are there any news on the optimization regarding the metrics calculation for BraTS-MET? As of now, we are getting waiting times ~6days from when submitting predictions to the validation queue until we get a response from the platform, which hinders the development and iteration of our algorithms. Please, let us know if there are any news on this regard. Best, Daniel
Hi @Jubilo You can ignore those with the "closed" status; they indicate some sanity checks. Let us know if you have any questions, /Mehdi
Hi, What's the difference between 'closed' and 'accepted' status in submission table?
@CharlesChen Thank you for your suggestion. Please note that one of the bottlenecks is computing the NSD metric per lesion and per sub-region. Nevertheless, We are working on a highly optimized evaluation script but its integration may take a few days. I appreciate your patience, Best regards, /Mehdi
@astaraki Thank you for your response. After waiting for about two days, I finally received the evaluation results. I would like to ask if it’s possible to use multithreading to process multiple files in a folder simultaneously during the evaluation. In my experiments, I used the lesion-wise Dice evaluation script you provided last year. By using multithreading to evaluate multiple cases concurrently, I observed an improvement in speed.
Hi there, How do I resolve a 'conflicting submission' issue when making a submission? Assuming I had already made a submission as an individual, and I'm now part of a team. Thank you
@CharlesChen Thank you for reaching out. The printed error message is a general message, and we are following it to find the exact issue. As soon as we find the reason, we will update you in this thread. In the METs dataset, it is quite common to have subjects with 20+ different lesions. Calculating different metrics per lesion and per sub-region sometimes takes a bit of time. Nevertheless, we are aware of that and we will optimize it as soon as we can. Please let us know if you have any questions, /Mehdi
Hi, After I uploaded my submission (BraTS-MET), I noticed that the evaluation process had been running for more than 9 hours without returning any results. Upon checking the logs, I found the following messages: [job score.cwl] Job error: ("Error collecting output for parameter 'results':\nbrats-infra-main/wf-segmentation/steps/score.cwl:31:5: Did not find output file with glob pattern: '['results.json']'", {}) [job score.cwl] completed permanentFail ... INFO toil.leader: 1 jobs are running, 0 jobs are issued and waiting to run It appears that the score.cwl job either failed to generate the expected results.json file or was stuck in execution for an extended period. Could you please advise on the cause of this issue? Thank you very much for your assistance.
I missed that. Thank you very much.
@CALADAN Thank you for reaching out. Yes, the evaluated metrics over the validation phase can be seen via the [unranked tables](https://www.synapse.org/Synapse:syn64153130/wiki/632724).
Hi, Will there be a results leaderboard similar to last year's challenge?

Submissions for the validation phase are now OPEN! page is loading…