On behalf of the MICCAI Brain Tumor Segmentation - Metastases (BraTS-METS) Challenge Team, we want to kindly invite you and the members of your team to participate in the critically acclaimed challenge for benchmarking algorithms for the automatic segmentation of brain metastases. In this thread we will announce important matters and will answer all your questions and concerns. We will be uploading tutorials for short paper writing, containerizing your algorithm, etc. Feel free to reach out here on every issue that you may encounter! Best, Nikolay Yordanov /BraTS-METS 2026 Co-Organizer/

Created by Nikolay Yordanov Niki_Yordanov_99
@lcw Dear Chengwei Li, thank you very much for your question! Yes, you are free to implement both of the strategies you mentioned. We appreciate it if you contribute new (fine-tuning) approaches and/or methods and it is especially nice if these surpass the baseline models. **Please make sure that you detail all your methods and strategies in the short paper that needs to be submitted together with the docker! ** It is important that all employed approaches and methods, as well as fine-tuning strategies, are explained clearly and in detail in order to ensure reproducibility of the proposed algorithms and to clearly distinguish them from existing models. Please note that the paper submission (in additional to a valid docker) is mandatory for invitation to the conference. Let me know if you have any further questions! :) Good luck! Paula
Dear BraTS Organization Team Regarding Task 4 of BraTS 2026, I would like to consult a question: Am I allowed to first pre-train the model using the official baseline code released on GitHub, then fine-tune the obtained weights with my self-designed algorithm? Additionally, could I take the weights trained by my Method A and conduct fine-tuning on my Method B subsequently? Looking forward to your reply. Sincerely, IntelliSense team
Hi, I'd like to reinforce @nittifra's point: since methods are ranked per case x region, this creates an implicit bias because 1) large lesions are more common, so lesion-wise DSC is weighted more heavily than small-lesion F1, and 2) common regions are weighted more heavily, mainly downweighting RC. It would also influence my final submission whether the final scoring corrects for this or simply accepts it as part of the ranking. Best, Luc
Hi @lcw, Thanks for reaching out. Please note that both of the Normalized Surface Distance/Dice terms stand for the same definition. NSD varies in the range of 0 to 1, with closer to 1 meaning better performance. For a more detailed description, you can read the [MetricsReloaded reference](https://arxiv.org/abs/2206.01653) Best regards, /Mehdi
Dear organizers, I apologize if this is a basic question, as I am still getting familiar with the challenge evaluation metrics. I have a question about the NSD metric used in Task 2. On the evaluation page, it is referred to as Normalized Surface Distance (NSD), but the description after the colon says that it assesses boundary-level overlap between predicted and reference segmentations. This wording is a bit confusing to me, since “distance” usually suggests that lower values are better, while “overlap” usually suggests that higher values are better. Could you please clarify how NSD should be interpreted in Task 2? Specifically: 1. Is a higher NSD score better, or is a lower NSD score better? 2. What is the value range of this metric? 3. Is this NSD equivalent to a normalized surface Dice / surface overlap score, or is it a distance-type error metric? Thank you very much for your clarification. Best, IntelliSense Team
Dear @astaraki, Following up on the weighting question from 16 June, where you noted it would be announced soon whether there would be an explicit weight between segmentation and detection metrics. Has the decision been made on whether a segmentation vs detection weight will be applied, or whether the three metrics enter the rank aggregation unweighted? With the ranking deadline getting closer, this now directly affects which model I select as my final entry, so I'd be grateful for an update. Thank you again for your time. Best, Francesco
Hi @mts027, Thanks for reaching out. The answers to your questions are: 1./2 Yes, the segmentation metrics `lesionwise_dsc/nsd_mean_{region}` as well as `small_instance_f1_{region}` will be used for the ranking. The other computed metrics, of course, can be valuable for participants to tune their solutions. 3. Please note that only a subset of the data contains `RC` label. If the size of the `RC` region is smaller than 27mm**3 then it will be used as a detection metric; otherwise, the segmentation metrics will be evaluated. 4. In many cases, the tiny tumor subregions are distributed like islands. If their sizes are smaller than the thteshold, only Detection metric will be considered. Best regards, /Mehdi
Dear organizers, Thank you for organizing this challenge and for providing the BraTS_evaluation package. I have a few clarification questions about the final BraTS-METS ranking and evaluation. 1. The evaluation description says DSC and NSD are subject-wise metrics, but also that segmentation metrics are applied only to lesions larger than 27 mm3. In the provided METS parser, this appears to correspond to the `lesionwise_dsc_mean_{region}` and `lesionwise_nsd_mean_{region}` columns, computed over large reference lesions with false-positive penalties. Is this the intended DSC/NSD used for ranking? 2. Which exact parsed F1 column is used for the detection ranking: `all_instance_f1_{region}`, `large_instance_f1_{region}`, `small_instance_f1_{region}`? 3. Are the same segmentation and detection metrics used for RC=[4] as for the tumor regions ET, TC, and WT? In particular, is RC included in the lesion-wise F1 ranking, or is F1 only used for tumor/ metastasis regions? 4. In some training cases, for example BraTS-MET-00404-000, ET=[3], TC=[1,3], and WT=[1,2,3] contain very small disconnected components. In that case, there are 4 WT connected components with size <= 2 voxels. Are such tiny components, which I assume are likely annotation artifacts, excluded from the F1 score, or are they counted as separate lesions during instance evaluation? Best regards, Luc
Dear Maria, We still haven't made the non-annotated data public. We plan on making it publicly available on Monday, 07/06/26. Keep in mind that it is a very small non-annotated dataset - around 275 cases, so it is not expected to bring any significant gains in performance. Using this dataset will be optional. The main purpose is to foster the development of semi-supervised methods. Before making it public we will reach out to all the people who have filled out the following feedback form and intend to submit to the challenge. We want to coordinate any updates with them. Please, if you intend to submit to the ranking phase of the challenge (deadline: 23rd of July), fill out the following form: [https://forms.office.com/r/YtZL5JdfzK](url) It will help us tremendously with the planning and support! Thank you! - Nikolay
Dear organizers, I have a question about the data available. On the challenge page it reads: _...In addition to that, we will also be offering to the participants non-annotated cases to facilitate innovations in semi-supervised learning approaches._ However, when accessing the data files I only obtain last years training and validation sets. Is the non-annotated data available somewhere? Thanks in advance. Cheers, Maria
Hi @nittifra, The structure of the dataset in the testing phase is identical to the training/validation sets in terms of naming convention, number of sequences etc. Best regards, /Mehdi
Dear organizers, A question about input modalities for the testing phase. The challenge description says that _T2W is no longer mandatory, and that some cases have native T2W, some synthetic, and some none._ For the hidden test set mounted at /input, I'd like to confirm how this is handled per case: Will every test case provide all four channels (T1n, T1c, T2w, FLAIR), with synthetic T2W supplied where native is unavailable, so a container can always expect four files? Or can the T2W file be genuinely absent for some cases (a missing file, zero/empty volume)? I'd like to make sure my container handles the different input modalities correctly and doesn't fail Thank you for your help! Best, Francesco
Dear @astaraki, This is my first time participating in a challenge, so please forgive me if I've missed something in the rules. I'd like to clarify the multi-task participation policy before proceeding. I am currently participating in Task 1 (Brain Metastases). I am considering submitting to Task 2 and Task 3 as well. To be clear, I would use the same model architecture and training pipeline across all three tasks, but retrain it separately on each task's own training data. My questions: 1. Is it permitted to use the same architecture/pipeline across Task 1, 2, and 3 as long as each model is trained on its respective task's data? 2. Are there any documentation requirements I should be aware of (e.g., disclosing in the algorithm description that the same architecture is used across tasks)? I want to make sure I remain eligible and compliant across all three tasks. Thank you very much for your help. Best regards, Junho Kim
Hi @Junho , The Synapse team is aware of the error, and a fix has already been proposed and will be merged soon! Back to your main question, you are still eligible for the standard track, as your first submission attempt did not go through. You're good to go for the standard track!
Dear @astaraki, This is my first time participating in a challenge, so please forgive me if I've missed something. I'd like to clarify my track status before proceeding. I do NOT require early results for visa purposes, so I intend to use the STANDARD Docker track that opens on July 3rd, not the EARLY track. However, earlier today I attempted a submission to the "Task 1: Brain Metastases – Docker (EARLY)" queue (before fully reading the EARLY/standard track policy), but it FAILED with the following error and did not go through: "Cannot find a node with id syn75859423 and version -1" My questions: 1. Since this submission failed with an error and did not complete, am I correct that my team has NOT been opted into the EARLY track, and that I can still submit to the standard Docker queues on July 3rd? 2. Is this error a known platform/queue issue that I should be aware of for the standard queues as well? I want to make sure I remain eligible for the standard track. Thank you very much for your help. Best regards, Junho Kim
Dear @msadi002, Thanks for reaching out. Please note that **Early Submission** is for those who need extra time for VISA processing. For those who don't need to apply for a visa, then the **Standard track** can be followed. Please note that teams cannot submit Docker images in both tracks. Best regards, /Mehdi
Dear astaraki Could you please clarify the exact submission deadline? The timeline shows both an Early Submission track (July 2, 2026) and a Standard track (July 23, 2026). I would appreciate confirmation of which deadline applies and whether participants must select one track before submission. https://www.synapse.org/Synapse:syn74274097/wiki/639587 Thank you.
Dear @nittifra Thank you for raising this important question. In the current implementation, size is measured relative to the reference mask; therefore, FPs are recognized but not measured with respect to their size (for that, different types of lesion-matching should be used). It is true that FPs penalize the segmentation metrics. For this, we can refer you to the following references regarding the Panoptic metrics: [Panoptica](https://arxiv.org/abs/2312.02608) and [Redefining Instance Matching](https://arxiv.org/abs/2605.31094) As for the weighting, please note that the majority of the lesions have a size larger than the Detection threshold. We will announce it soon whether there would be a weight between Segmentation (DSC,NSD) and Detection (F1) metrics. Best regards, /Mehdi
Dear @astaraki, Thank you for the earlier clarification on the ranking, and for pointing us to the evaluation pipeline, it's been very useful to better understand the challenge. Following your note that the METs ranking uses DSC and NSD for large lesions and the F1 score for small lesions, I'd like to confirm one detail about false positives, as the released pipeline seems to diverge slightly from that description. In the lesion parsing, false positive instances are appended to the large-lesion DSC/NSD list as zeros regardless of size (num_fp is not size-attributed). So a small false positive lowers the large-lesion DSC and NSD rather than affecting only the small-lesion F1. Could you kindly clarify: - whether false positives are intended to contribute to the large-lesion DSC/NSD, or whether they should be size-attributed so that small false positives affect only the small-lesion detection metric; - how detection and segmentation are weighted relative to each other in the final ranking. From the challenge description, detection appears to carry considerable weight, but in the rank aggregation it enters through a single metric (F1) while segmentation contributes two (DSC, NSD), so in practice segmentation appears to drive the placement. Is this balance intended? I'd like to be sure I'm interpreting the metrics correctly before selecting my final submissions Thank you again for your time Best, Francesco
Hi @Junho Thank you for your question. Please review the general instructions about [Containerization](https://challenges.synapse.org/Challenges/DetailsPage/Instructions?id=syn74274097) We will update this page in a few days. We highly recommend first trying out your Docker image locally, once you ensure it works smoothly proceed with pulling to the registry. Best regards, /Mehdi
Dear organizers, Quick question on team structure: can a single person participate across more than one team? For example, can I both lead my own team and join a colleague's, or be a member of two? Trying to figure out whether to consolidate everyone into one team or whether separate teams with overlapping members is permitted. Thanks! Best, Francesco
Dear @astaraki, I am currently preparing my submission and would like to make sure I am following the official containerization procedure correctly. I have looked through the Overview and Instructions tabs, but I want to confirm that I am not missing anything. Could you kindly point me to where the baseline Docker container information and guidelines are documented? Thank you. Junho Kim
Dear @Junho, Thanks for reaching out. Final ranking will be done as follows: (i) a ranking score, based on case-wise cumulative rankings aggregated across multiple metrics and testing cases, and (ii) a rigorous statistical significance analysis via pairwise permutation testing across the ranked order of the AI models. Please read the [PermRanker ](https://link.springer.com/chapter/10.1007/978-3-032-16365-3_1) that describes the evaluation method in detail. The METs evaluation is based on the DSC and NSD metric for large lesions and the F1 score for small lesions. Best regards, /Mehdi
Dear @astaraki, Thank you very much for organizing this challenge and for offering to address participants' questions here. I have a question regarding how the final ranking is determined from the evaluation metrics, and I would be grateful for your clarification. The leaderboard reports several metrics — lesion-wise DSC, lesion-wise NSD, and small-instance F1 — across multiple regions. In analyzing my own validation submissions, I noticed that the relative ordering of entries is quite sensitive to how these metrics are combined: a submission with the highest DSC/NSD but a lower F1 ranks very differently under a simple average of metric values than under a rank-based aggregation. Since this directly affects which submission I should select as my final entry, I would like to make sure I understand the official procedure correctly. Specifically, could you kindly clarify: 1. Which metrics enter the final ranking? 2. How the metrics are aggregated — for example, by averaging the metric values directly, or by a rank-then-aggregate scheme? Thank you again for your time and for the effort you and the team are putting into this challenge. Best regards, Junho Kim
Dear @vchung Problem solved with your proposed solution. Thank you a lot for your help! Best, Francesco
Hi @Junho, Thanks for reaching out. The validation leaderboard for the METs challenge has been updated, and now it shows the Detection metrics for small lesions as well. You should receive an email containing a CSV file showing the detailed evaluation metrics from your submitted jobs. The evaluation pipeline is available so you can use it for assessing your training procedure: [BraTS Evaluation](https://github.com/BraTS/BraTS_evaluation) Best regards, /Mehdi
Hi @msadi002 , Sorry if this wasn't made clear. The instructions are available under "File Predictions" in the Instructions tab here: https://challenges.synapse.org/Challenges/DetailsPage/Instructions?id=syn74274097 If it helps, I downloaded your submission and your output filenames include an extra "-seg" at the end, which is causing the invalid errors. We are expecting the filenames to end with the subject ID (5-digit case ID and 3-digit timepoint), followed immediately by `.nii.gz`. Something like this: ``` -xxxxx-xxx.nii.gz ```
Dear, I submitted the validation files, but getting an error: Your submission (id: 9767111) is invalid. Below are the invalid reasons: Not all filenames in the archive follow the expected naming format. Please check the Submission Tutorial of the task you're submitting to for more details. Then also could not find the submission instructions in the challenge portal Best Shibly
@nittifra , Oh! Let us continue diagnosing here then. I think I may have found your issue. When you view your [profile page](https://www.synapse.org/Profile:v/profile), do you see “Challenge Submission Project\_syn74274097\_3587408” listed among your Projects? If not, can you try going to the [Trash Can](https://www.synapse.org/Trash:0) and seeing if the project is there? What I suspect is happening is that the widget is attempting to locate your submission project, failing to find it, and then trying to create a new one. However, it's getting blocked because the unique alias (`challenge_submission_project_syn74274097_3587408`) is still technically taken by the project sitting in the trash. If the project is in your Trash Can, please try restoring it to see if that clears up the error. Let me know how it goes! EDIT: markdown fix
Dear @vchung, Unfortunately I am not able to access support ticket on the link that you sent me, even after creating the account with the same email that I have here on Synapse. I am thinking if it may be a viable solution to recreate an account here and join again the challenge, let me know how you would advice me to proceed. Thank you again for your help Best, Francesco
Hi @nittifra, Thank you for trying that! To keep this thread focused on general MET announcements, I have moved our conversation over to a dedicated support ticket that I just opened on your behalf: https://sagebionetworks.jira.com/servicedesk/customer/portal/18/CBSD-129 We will continue troubleshooting there and hope to get this resolved for you soon!
Dear Nikolay and the BraTS-METS 2026 Organizing Team, Thank you very much for organizing this challenge and for offering to address participants' questions here. I have two requests/questions regarding the evaluation metrics, and I would be grateful for your clarification. First, the current leaderboard reports only lesion-wise DSC and NSD. Would it be possible to additionally include the lesion-wise F1-score, as well as subject-wise DSC and NSD? Having these complementary metrics would give participants a more complete picture of detection and segmentation performance, particularly for cases with multiple small lesions. Second, I would like to confirm my understanding of the lesion-wise DSC and NSD computation. Based on the evaluation announcement, my understanding is that lesions smaller than 275 mm³ are excluded from the DSC and NSD calculation. Could you kindly confirm whether this is correct? Thank you again for your time and for the effort you and the team are putting into this challenge. I look forward to your response. Best regards, Junho Kim
Hi @vchung, thanks for confirming my registration! I tried relogging and refreshing across several browsers and machines, but I still get the same errors, so it doesn't seem to be on my end. The one that seems to be blocking things is this: "The friendly url name (alias): challenge_submission_project_syn74274097_3587408 is already taken." My guess is that a submission project with that alias already exists from when I staged a submission before the phase opened, so the page can't create it again and the submit widget never loads. Happy to run any test that might be helpful Thanks a lot for your help Best, Francesco
Hi @nittifra , As a first diagnostic step, can you try relogging in, then refreshing the page?
Hi @nittifra , Apologies for this issue! I can confirm that you and your submission team are fully registered for the challenge, so this appears to be an unexpected UI error. Thank you for bringing this to our attention, I am looking into the issue now and will update you as soon as possible.
Hi Mehdi, thanks! I can see the METs queue is live now and submissions are coming through on the leaderboard. On my end though, the upload/submit option isn't showing anymore. I'm registered and had staged some submissions before the phase opened, but the widget now seems disabled. Could you confirm my team registration is complete on your side? Thank you for your help! Best, Francesco EDIT. Maybe It could be useful, I am getting this error from the server side: repo-prod.prod.sagebase.org/repo/v1/team/3379097/member/273950:1 Failed to load resource: the server responded with a status of 404 () repo-prod.prod.sagebase.org/repo/v1/team/3584866/member/273950:1 Failed to load resource: the server responded with a status of 404 () repo-prod.prod.sagebase.org/repo/v1/team/3379097/member/3587400:1 Failed to load resource: the server responded with a status of 404 () repo-prod.prod.sagebase.org/repo/v1/entity:1 Failed to load resource: the server responded with a status of 409 () index-08ZqUM07.js:238 Uncaught (in promise) sT: The friendly url name (alias): challenge_submission_project_syn74274097_3587408 is already taken. Please select another. at aJ (index-08ZqUM07.js:238:810253) at async e (ChallengeSubmission-D11DsK2L.js:117:102037)
Hi @nittifra, The validation queues for the Pathology and inpainting have been opened since yesterday, and for the segmentation tasks, including METs, will become available by tomorrow. Best regards, /Mehdi
Dear Nikolay, quick question on Task 1. I don't see a way to submit yet. Was the opening date moved from June 1, or is there a separate step I need to do to access submissions? Thanks for any clarification! Best, Francesco

BraTS-METS Announcements page is loading…