Hi, I have three questions to the submissions of the predictions on validation data concerning task 2. 1) The submissions are not yet available since the validation phase starts at 25th of June, right? Because right now when trying to submit I get the rejection, saying that I am not participating in "Evaluation/Challenge", even though I have registered already. I want to be sure that this is not an issue with the registration but with the timeline. 2) I would like to be sure about the format of the submitted files. While on the submission platform it is already explained in great detail, I was wondering about the meaning behind the cursive parts of the file-/folder-names "LISA_HF_12345segprediction.nii.gz" and "LISA_LF_SEG_predictions.zip". Since the labels **LF** and **HF** do not match with each other and they were written in cursive I wanted to make sure that this is just the formatting error and not a detail about the naming conventions that I have missed about the structuring of the submitted files. 3) The last question is regarding the team contributions. Is it allowed to participate as an individual up until a certain point and then apply to some team, or is there any deadline for team creation there? Thank you in advance for your help!

Created by Kostiantyn Lavronenko mackostya
Dear @mahbodissaiy, We have not specified the server to work at specific time intervals. Your submission should be evaluated within a few minutes after submission. We strive for this level of immediate feedback, but there can sometimes be difficulties with communication between Synapse's backend workflow orchestrator and our machine that performs the automatic scoring. We appreciate your notifications when there is an issue so we can address it. Best regards, The LISA 2025 Challenge Organizers
Dear @LISA_mri_challenge, When submitting files for evaluation, sometimes it takes several hours for the status to progress from _RECEIVED_ to _ACCEPTED_. I wonder if this is a backend issue or the server works at some specific intervals. Thanks for organizing such great challenge.
For 12 hours it seemed stuck but it finally got evaluated. Thank you @LISA_mri_challenge !
Hi @mahbodissaiy, Has your submission been scored? If not, please let us know. Thanks, The LISA 2025 Challenge Organizers
Dear @LISA_mri_challenge , I submitted some other files to the challenge, they are _RECEIVED_ but don't seem to progress any further. Is this an issue with the backend or do the evaluations take place at certain times? Thank you in advance!
Dear @mackostya @mahbodissaiy Thanks for your patience and feedback throughout the process. We're happy to confirm that the submission pipelines for both Task 2A and 2B are now fully operational, as you have noted. The validation scoring is running as expected and your results can be found here: https://www.synapse.org/Synapse:syn65670170/wiki/631796. Regarding previous submissions, these were not scored due to a backend configuration issue in the evaluation queue that has since been corrected. You're welcome to resubmit those predictions, and they will now be processed normally. Best regards, The LISA 2025 Challenge Organizers
It seems to work for me too, thanks @LISA_mri_challenge!
I think the submission pipeline is working properly now. Thank you @LISA_mri_challenge for the quick fix.
Dear @LISA_mri_challenge, it seems like the submissions that I have done yesterday, on the 03.07.2025 were removed from the table without any scoring or further explanations. Could you please disclose the reason for it? Thank you for your help.
Thank you for your fast response @LISA_mri_challenge , However as @mackostya pointed out, Task 2A seems to have problems. Thanks for the assistance.
Dear LISA 2025Challenge Organizers, yes, now it seems to be possible to submit for the task 2b. The submission for the task 2a though seems to still not work, but due to a different issue. Here is the error that pops up: "It is currently outside of the time range allowed for submissions". Thank you very much for your help in advance!
Dear @mahbodissaiy and @mackostya, Thank you both for your messages, and apologies for the inconvenience. We’ve reviewed the issue and believe it has now been resolved. You should now be able to submit your predictions for Task 2 without encountering the “You are not currently participating in an Evaluation/Challenge” error. Please try again, and let us know right away if the issue persists. We’re actively monitoring the system to ensure everyone has smooth access during the validation phase. Best regards, The LISA 2025 Challenge Organizers
Hi, Right now, I am not able to submit as well. Same window pos up.
Hello there! I can't seem to be able to submit to the challenge task 2. I have created a project and uploaded the zip file as instructed. However, I get the message "_You are not currently participating in an Evaluation/Challenge. Please join one and try again._" I am pretty sure that I am participating in the challenge... Since it's July 2nd, I was wondering if I am the only one unable to submit or others have the same problem as well. Thank you for your assistance.
Dear @mackostya, Please see our answers below, in the order you asked. 1) That is correct. You can begin making submissions and receiving validation scores starting in July. Right now, the validation queue is not available. 2) Please follow the naming conventions described on the submission platform. While your predictions are made on low-field data and should therefore be submitted as LISALFSEG_predictions.zip, the predictions are compared to a high-field segmentation (the ground truth). Appropriate naming (i.e., using HF in the individual segmentation file) ensures that the file names from your prediction and our ground truth match exactly, minimizing potential scoring issues. 3) There is no deadline for team creation. Best regards, The LISA 2025 Challenge Organizers

Validation prediction submissions task2 page is loading…