Hello organisers, Thank you for the release and the clarification about DEL use in the Active Learning Step. To make sure our workflow is eligible before we spend a submission, could you confirm whether the following counts as "significant use of DEL data" with the DEL-trained model as "a key component"? 1. Our ML model is trained on the DEL screening data and scores every compound in the Active Learning test set. This DEL score is the primary ranking. 2. We then use the released Blind Test labels (actives and inactives) as a secondary signal, for example by up-weighting compounds structurally similar to released actives and down-weighting those closer to released inactives, and/or by fine-tuning the DEL-trained model with the released labels. 3. The final 50 are selected from this combined ranking. Is there a minimum share of the final ranking that must come from the DEL model, or is it enough that the DEL model is used as the primary ranker as described? Thank you, Arnav Agarwal

Created by Arnav Agarwal ArnavAgarwal
Hello @ArnavAgarwal, 1. The method you are describing is acceptable for the challenge. 2. In this case we would consider it acceptable, since the same result is obtained with both selections. If identical results would be obtained with a strict similarity threshold, and a more inclusive threshold, the lower threshold is considered. 3. The more correct answer is "Yes", since we consider the use of external data in any shape or form. It is not a big problem, the incorrect labelling will be fixed if the method will be featured in the overview article.
Hi Luca, a gentle follow-up on my reply 36811 above (14 Oct), in case it got buried. The main question is point 2: whether you read our effective analogue pool as 690 or as the 10,924 that gives the identical 50 picks. Your answer decides how we build our second Active Learning submission before the 31 Oct close. Thank you. Arnav Agarwal
Hello Luca, Thank you for the reply of 9 October. You said the answer would depend on the contribution of the DEL-ML score relative to the similarity term, and on how many analogues are selected. Our Active Learning submission has since been scored, so I can give you those two numbers measured rather than described. **1. The contribution of the DEL-ML score.** The final ranking key is (maximum ECFP4 Tanimoto similarity to the 39 released actives - inactive penalty) x DEL-ML percentile. Two DEL-ML models trained on the provided AIRCHECK data are combined by average-rank. Removing the DEL percentile entirely, while keeping the same similarity floor, the same cap of two picks per anchor and the same tie-break, **changes 19 of the 50 picks**. Removing only the second DEL-ML model changes 12 of the 50. **2. How many analogues are ranked.** You wrote that 100 analogues would probably not qualify and 10,000 probably would. Our similarity floor of 0.45 admits **690** of the 178,052 compounds. Lowering that floor to 0.30 admits **10,924**, and leaves all 50 picks unchanged, pick for pick and in the same order. That follows from the arithmetic: the percentile is at most 1, so a compound's key can never exceed its own similarity term, and compounds far from every released active cannot outrank the near analogues however well the DEL models score them. So the shipped code ranks 690 analogues, and ranking the 10,924-compound pool returns an identical selection. My question is therefore narrow: **do you read the effective analogue pool here as 690, or as the 10,924 that produces the same answer?** If 690 is too few for the requirement as you intend it, I would rather know now and change the approach for our second submission than find out afterwards. **3. A separate, practical question about declaring external data.** We use compound 47 from CACHE Challenge #7, a published PGK2 inhibitor, only as a known-answer control: it is appended to the validation frame to confirm that the consensus ranks a known binder highly. It is not in any training set and it is not among the 50 picks. It is declared in the write-up text of every submission we have made. On the submission form we set `external_data_used` = Yes for this Active Learning write-up. Our two Blind Test write-ups declared the same compound in their text but were submitted with `external_data_used` = No. **Is Yes the correct answer for a published compound used only as a control, and if so, should those two earlier rows be corrected?** I did not want to edit previously submitted rows without asking. Thank you for the time, and for the detailed answers on this forum throughout the challenge. Arnav Agarwal
Hello @ArnavAgarwal, unfortunately I cannot give you a clear answer on this, since the description is quite vague. Based on what you describe your workflow is eligible, but it would depend on the relevance of point 2 in your list. If you fine-tune your DEL-model on the released label that is the ideal use case we imagined for the challenge. If you use structure similarity to weight compounds it will depend on how it is done. A blending of similarity and DEL-ML score is acceptable (it would depend on the contribution between DEL score and active learning score). Selecting analogs of the revealed active molecules and then scoring them by DEL-ML would depend on how many analogs you select (100 analogs probably not, 10,000 analogs probably yes).

Active Learning Step: does a DEL-trained model re-ranked with the released labels meet the "significant use of DEL data" requirement? page is loading…