VALID/TEST records in the released TFRecords have a different layout from TRAIN

The model card says intervals were extended by 2,048 bp on each side, and its example shapes match the TRAIN records. VALID and TEST are laid out differently, and some bundles need the metadata to tell padding from data.

Layout:

  • TRAIN: 1,052,672 bp records with full-length targets. 1 bp tracks, splice_sites and splice_site_usage have 1,052,672 rows, ChIP 8,224, contact maps 514 x 514.
  • VALID and TEST: 1,048,576 bp records. The 1 bp and 128 bp targets, including splice_sites and splice_site_usage, cover only the centred 196,608 bp (offset 425,984), so ChIP has 1,536 rows. Contact maps still cover the whole interval: 512 x 512.
  • splice_site_positions are never cropped. They are coordinates over the full interval, and in VALID/TEST many fall outside the 196,608 bp window.

Same for mouse and human. evals/track_prediction.py copes by centre-cropping predictions, but this isn’t written down.

Padding in bundles without a _mask feature (contact_maps, splice_sites, splice_site_usage, splice_junctions) can only be found from the Padding names in the metadata textprotos. The value differs per bundle. In mouse:

  • contact_maps: 0, 20 of 28 columns. Real entries can also be NaN.
  • splice_site_usage: -1, 554 of 734 columns.
  • splice_junctions: 0, 277 of 367 tissues, which looks the same as a real tissue with no junctions in the interval.

Could the model card’s data section describe the VALID/TEST layout and these padding values?