Hi organizers, I would like to clarify the evaluation protocol for the Task 2 Pediatric Segmentation task. From the challenge description, my understanding is that the **final evaluation** is based on **lesion-wise metrics**, whereas the public validation leaderboard reports **global metrics**. Is this correct, or will the final ranking also be based on the global metrics? Also, the detailed evaluation email currently includes global metrics (DSC/NSD/HD95) and instance detection metrics (TP/FP/FN/F1), but I could not find the lesion-wise DSC/NSD. Will these lesion-wise metrics be available for validation submissions? Thank you!

Created by Yong Chen chenyo
Hello @astaraki, thank you for the clarification, and thanks @chenyo for raising this. I understand that PEDs and GoAT are ranked on global segmentation metrics only, even though lesion-wise and detection metrics are already computed and reported through the evaluation pipeline. I'd like to gently raise one data-driven consideration for the organizers to weigh. Looking at the small/large lesion distributions for these tasks, a large fraction of lesions fall below the small-lesion threshold (27mm³). For Task 2, small lesions outnumber large ones in most regions (e.g., NET: 1296 small vs. 425 large; CC: 721 vs. 222), and Task 3 shows a similar pattern across ET/TC/WT. The contrast becomes striking when looking at volume: while these small lesions make up a large share of the lesion count, they account for less than ~0.5% of the total lesion volume in nearly every region (e.g., Task 2 NET: 0.04%; WT: 0.005% — and even the highest, CC, is only 0.51%). **This means that under a volumetric global DSC, a model could miss the small lesions almost entirely with negligible impact on its score, so performance on the many small lesions is essentially invisible to the current ranking** — which is the same concern that motivated the dual-metric ranking in the Metastasis task. Since the lesion-wise DSC and F1-based detection metrics are already being computed, would the organizers consider incorporating them into the final ranking for Tasks 2 and 3 — even with a small weight, or as a secondary/tie-breaking signal — rather than reporting them for reference only? I completely understand if the protocol is already finalized; I just wanted to surface the distribution data in case it's useful. Thank you for your work organizing this challenge! Best regards,
Hi @chenyo, Thanks for reaching out. In the segmentation track, the Metastasis task is evaluated for both segmentation and detection metrics due to the fact that some lesions are too small to be fitted for overlapping-based metric evaluations. The other segmentation tasks, including PEDs and GoAT, however, will be quantified for only the segmentation metrics (global metrics). You can use the original [evaluation pipeline](https://github.com/BraTS/BraTS_evaluation) to compute all the metrics, such as lesion-wise, subject-wise, segmentation, and detection. Best regards, /Mehdi

Task 2 Final Ranking: Global or Lesion-wise Metrics page is loading…