We want to make sure our use of the validation split stays within its intended purpose. We use N_hits / N_clusters from validation submissions to compare methods (for example, two neighbourhood definitions applied to the same earlier selection). Is that acceptable? For transparency: in August some of our validation submissions (e.g. 9775004) were calibration sets assembled from our own earlier validation results rather than model outputs. They were never used on the test split, and we have stopped building them. If this falls outside intended use we are happy to annotate them however you prefer. Thank you.

Created by Vũ Mai VBDI_Bio_FEx_TeamVNG
Hello @VBDI_Bio_FEx_TeamVNG, we intended the validation split to be a toy dataset that participants could use to test their virtual screening pipelines. Your assumption on the use of the validation split was correct. In general there are no limits on how the validation split data, and evaluation results, could be used, as long as any used data is explicitly indicated in the submitted writeup.

Intended use of validation feedback, with a disclosure page is loading…