I suggest
a. multiple submissions are allowed.
b. no score feedback.
c. no log feedback.
d. last submission taken as scoring.
The reason is that some participants may make some mistakes, then they will ask to resubmit. Some of them could be reasonable requests, such as accidentally hit scoring lane instead of express lane. If then you allow one that means you must allow all resubmission. Then the score and log feedback has to be turned off for obvious reasons.
Thank you for your consideration.
Yuanfang
Created by Yuanfang Guan 小黄药师 yuanfang.guan Great, thank you. great! we look forward to the community phase. Hi,
The deadline remains November 20st at 11:59PM PDT. Indeed in the main page it was stated Nov 29 which was an error, apologies for this, it has been updated.
That said, we would be very keen to explore further rounds beyond the deadline, as you suggest. We could do a follow up community phase.
Unclear deadline on the synapse website:
- Sections 1 and 2.4: Nov 20.
- Main page (https://www.synapse.org/#!Synapse:syn8228304/wiki/): Nov 29.
Which is correct?
my student just suggested his twitter (or NCI's twiiter) might be hacked... i wish that is the case, as nothing is changed on the website, can you confirm? Thank you, Julio.
through an automatic email sent through twiiter @gustavo 's twitter, I saw the deadline got extended.
As I clearly communicated with you through several emails and you agreed, that to reach our current performance, teams only need to make 1-3 line of code modification. It will be extremely unfair to us to extend the deadline as we are going to explicitly tell how to write this line in our talk, while the other teams clearly have the advantage of knowing everything that we don't know.
However, I understand that now to ensure full participation, it is preferred that the deadline is extended. This situation has happened twice before. I would like to mention the eventual solution for your reference to make a proper going-foward. In the ENCODE challenge, both the winners from the original round and the extended period were considered as official winners, and shared the incentives equally. In that case, my team benefited from the extended round by clearly taking some ideas from the 1st round winners. The follow-up community phase was not disrupted by this poorly planned timeline, but rather built tight relationships between the 4 winning teams.
In the recent PDDB challenge, the final solution is that teams are allowed to submit after the initial deadline, which is 20th in our case, but after the initial deadline, no cash prize (and in that case additionally talk/travel award) is given, as this case is very close to our case that the winning solution from my team and another team are already released on the deadline. But, in that case, even if giving the rest teams another month, they won't be able to produce my result as there is hardware/software constraint; while in this case, half of the teams are able to reproduce in 5 minutes. We feel our intellectual property is completely deprived.
Hi Yuanfang,
This is a good suggestion, and we have implemented the system now so that the last submission will be the one scored.
Thanks
Julio Hi Yuanfang,
I've forwarded these to the organizers.
-Andrew @andrewelamb @MI_YANG @thomas.yu
the reason i suggest this is if only one shot final submission, people will wait until the last day to submit, in the hope that some magic improvement might happen at last minute. The queue will be very tight and there will be many failed submissions, eventually lead to some people asking for extension. Then, if you extend, there will be people complaining about extension, and ask for additional quota.... I see a lot of troubles coming up if not that last submission is taken (of course no score/log feedback).
thank you for your consideration.