I was looking at the test-phase submission guidelines on Synapse and it seems like the runtime requirements has been changed from 1 hour for _all_ 20 cases per task to 1 hour _per case_ per task, for a total of 20 hours per task. This is a very sharp deviation from what was advertised initially, which has been conveniently removed from the challenge website seen in [this GitHub Diff](https://github.com/MRIxFields/mrixfields.github.io/commit/c38bb2df4d00a97a9fd3f503b91543ae3ac2b44c#diff-768f5d880a7914e66e927bcd06fed91da9d79b46bd8fcfc572df5b4d10372b69): you can clearly see that in faq.html that the sentence 'However, the total running time for all data in each task cannot exceed **1 hour**.' was removed only five days ago. In fact, this was clarified [here](https://www.synapse.org/Synapse:syn72060672/discussion/threadId=13179) that the strict runtime requirements was intended to encourage creative problem solving on fast and accurate generation of complex 3D volumes. We, as a team, designed our solution with the strict runtime in mind and forewent expensive solutions (and hence we are much faster than the 1 hour total time limit), and we believe that this fully upends many approaches to innovative problem solving. Please could you clarify whether the 20 hours is here to stay or just a relic from a previous version of the test-phase submission that will be changed.

Created by siddsrivastava
Thank you for the follow-up. The original one-hour total limit was set before we had completed full end-to-end testing of the released data and evaluation pipeline. Our later internal tests showed that each case contains multiple imaging contrasts and requires multiple source-to-target field generations, making the original limit impractical. The runtime limit was intended as an operational constraint rather than a ranking metric. We apologize again for the late clarification and understand the concern this has caused.
Could you please clarify two things: 1) why and how the runtime limit was changed so drastically from around 3 minutes per case to 60 minutes per case and 2) if "MRIxFields2026 has **always primarily focused** on reconstruction/generation performance rather than computational efficiency", why was the 1 hour time limit there in the first place? I think anyone would look at the 1 hour limit and quickly realise that generating that many high-quality volumes in that time is challenging, and that some trade-offs would have to be made between generation quality and staying under the time limit, so clearly model efficiency has been an implicit metric.
Thank you very much for your feedback. First of all, we sincerely apologize that the runtime requirement was not clearly described in the earlier version of the challenge documentation, which understandably caused confusion. After carefully considering the overall dataset size and the complexity of the tasks, as well as conducting our own internal testing, we concluded that a runtime limit of up to 1 hour per case (per task) is a more practical and reasonable requirement for the final evaluation. It is also worth noting that MRIxFields2026 has always primarily focused on reconstruction/generation performance rather than computational efficiency. We do not include model efficiency as an additional evaluation metric, and therefore the runtime limit is intended mainly to ensure that submitted methods can be evaluated reliably within a reasonable amount of time. The recent update to the website was made because we received feedback that the previous wording was ambiguous, and we wanted to clarify the intended requirement. We apologize for any inconvenience this clarification may have caused, especially for teams that designed their methods based on the earlier interpretation.

Test phase submission runtime page is loading…