According to the challenge guidelines: "Teams will be ranked based on the number of hits, their chemical novelty, and their chemical diversity." However, the current leaderboard appears to prioritize the number of hit clusters (N_clusters), with the total number of hits (N_hits) considered only as a secondary criterion. I have three concerns about this ranking approach: 1. The number of clusters depends heavily on the clustering method. Different clustering algorithms, molecular representations, or similarity thresholds can produce substantially different numbers of clusters from the same set of compounds. In contrast, the total number of experimentally validated hits remains unchanged. Therefore, I believe N_hits should serve as the primary ranking criterion, as it provides a more direct and objective measure of predictive performance. 2. Multiple hits within the same chemical cluster can also be scientifically valuable. If a chemical cluster contains many validated hits, this may indicate a promising chemical scaffold with a high hit rate and strong potential for further lead optimization. Conversely, a cluster represented by only one hit provides less evidence of reproducible structure–activity relationships or broader optimization potential. Therefore, prioritizing N_clusters over N_hits may undervalue teams that successfully identify multiple active compounds within promising chemical families. 3. Why not use a weighted composite score? Since the challenge guidelines explicitly mention three criteria—hit count, chemical novelty, and chemical diversity—why not combine them into a transparent weighted score? For example: Overall Score = w₁ × Hit Score + w₂ × Novelty Score + w₃ × Diversity Score This would allow all three criteria to contribute to the final ranking, rather than having one metric dominate the others. If the ranking is ultimately determined primarily by N_clusters, could the organizers clarify why N_hits is reported on the leaderboard but does not appear to carry comparable weight in the ranking? I would appreciate clarification on the rationale behind the current ranking methodology and whether a more balanced evaluation could be considered.

Created by Lei Zheng lz2826
@LucaChiesa
@LucaChiesa

Concerns Regarding the Ranking Criteria for the Prospective Phase page is loading…