Hi, I submitted predictions for Task 4 and some files were successfully evaluated (submission ID 9771038). However, the all_scores.csv file shows empty scores for many predictions. Looking at the submission logs, the Toil worker is throwing a disk space warning during the scoring step (4.3 GB [4567248896B] used, 2.0 GB [2147483648B] requested). I double-checked the .zip file I submitted, and all predictions are present with the correct names. Could you please take a look at this issue? Thanks!

Created by Arnela Hadzic arnelahadzic
@arnelahadzic @NVT , Apologies for the inconvenience! We confirmed this was not an isolated issue, and we have deployed a fix to the root cause. If you have past submissions you need re-evaluated, please reply with your submission IDs (but please keep in mind we will be running these sequentially). Otherwise, feel free to keep submitting as usual, as all new entries will use the updated environment.
I saw the same pattern in my previous submission as well.
Hi, Just to note: in my last Task 4 submission, 61 out of 219 predictions were evaluated in all_scores.csv, and the remaining ones are empty. All files are present in the .zip with the correct names. Happy to share my submission ID if helpful. Thanks!
Thank you for your response! Unfortunately, it looks like the issue occurred again, many scores are still empty and the final results remain unchanged. Is this happening to other submissions as well, or is there something specific about my submission format that I should change on my end?
@arnelahadzic , Thanks for bringing this to our attention. I have looked into the issue and a fix has been implemented. I've restarted your submission (ID 9771038), and updated results should be returned soon.

Task 4: Incomplete Scores page is loading…