Reproducing TraitGym results

Help & Support
1

Thank you for your great contribution to the field! I’m trying to reproduce the TraitGym results:

  • I was able to score variants using score_variant and recommended scorers, but did not get close to the reported performance.
  • I then tried to reproduce the TraitGym protocol described in the supplement, using the lower-level predict_sequence . However, it was very notably slow and throwing errors related to data transfer quotas.

It would be amazing if you could support the “TraitGym” protocol (e.g. L2 norm, reverse-complement averaging) in a manner similar to the score_variant API.

2

Hi @Gonzalo_Benegas,

Thanks for the question! Yes we will add support for L2_DIFF_LOG1P aggregation type, which should be sufficient to reproduce the paper’s traitgym results (reverse complement should already be supported by passing an interval with negative strand).

3

Great to hear, thank you very much!

4

Hello!

Thanks for the recent addition of L2_DIFF_LOG1P aggregation! I still have some questions about the scorer:

Specifically, for each track predicted by a model, we first computed the predicted log-fold change in activity per position (or bin) due to the variant and then calculated the L2 norm across the sequence.

Does this mean you took L2 norm across the entire 1Mb for all assay types? Or did you use a center mask for some assays?

I’m guessing the protocol cannot be reproduced with CenterMaskScorer as it seems to support a max width of 200kb, which would probably lead to underperformance for RNA-seq.

Thank you for your help!