Dear Organizers,   For more than 10 times, the data has been secretly updated without notifying the participants. I wonder if it is possible, that in the future, when leaderboard/final board or training data gets updated for any reason, can you send a global email to all of us?   Your help will be deeply appreciated.   Yuanfang Guan

Created by Yuanfang Guan 小黄药师 yuanfang.guan
that makes sense. thank you. i think it is a great promise to us that if test data get updated by any means we will be notified. thank you.
Dear Yuanfang, Thank you so much for your interest in the challenges. I am sorry for the late reply. We work closely with CPTAC CDAP group to generate the most updated version of the training and testing data. We announced the update of the training data in our webinar at Sep 1. While at the same time, the testing data were still under production by the CDAP group. That is why we had complete training data on the dream challenge webpage as placeholder. We then got the testing data and updated the training data with subsetting genes shared by both collections to make all participants' model more applicable to the testing data. That is why you see the changes on the training data. Again the changes on the unpdated training data were only keeping common genes, control NA value rates and harmonize format for the purpose of making your model building process easier and more generalizable to the testing data. We hope we address your concerns. Best, Zhi
Dear Yuanfang, Sorry about the delay in response. We will definitely notify you when there is an update on the testing data. There has not been any updates to the testing data since before we opened up the submission queues. I will have the other collaborators talk about the changes to the training data. Best, Tom
@saezrodriguez   I think it is absolutely necessary that all participants are notified when there is a change in test data. The reason is that training data was secretly changed for more than 10 times, and thus I have good reason to believe that testing/leaderboard data has been consistently changed all the time and will be changed multiple times in the future.   I don't think that is an excessive request, because other wise, we cannot compare among multiple submissions. I asked via email if it is possible to thomas yu, he said I must ask here, then I ask here. I never got an answer. To be frank, I don't get what an absence of answer implies. I think it is a serious non-commitment to our participants, that the errors on the organizers's side are never reported, especially taking advantage of the docker mechanism.   Can you please check in? Thanks.
sub2 data and Sub3 was replaced for about 10 times respectively, as I download about every two times when they update, as i really cannot tolerate downloading every other day and retrain the model. and I have 6 dirs all with different data: sub2 sub2_Aug11 sub2_Aug14 sub2_Aug31 sub2_July5 sub2_Sep7 I am more of requesting no change on leaderboard and final test data now.
Just curious what has been updated? Should I reload the training data for sub challenge 2? Edward

Requests to notify all participants when data get updated page is loading…