Benchmark validity in graph neural network scoring of metabolic reaction activity on Recon3D: detecting label leakage, memorized noise and input-invariant models

Context-specific genome-scale metabolic modeling begins with scoring which of the approximately 10,600 human reactions are active in a patient's tumor. Methods in this literature are routinely benchmarked against activity labels obtained by thresholding the same transcriptomic matrix that is supplied to the model as input. We report a self-audit of our own graph attention scorer, MetaGNN, evaluated on TCGA colorectal (n=624), breast (n=1,095) and lung adenocarcinoma (n=517) cohorts, in which two independent failure modes produced a near-ceiling benchmark score and a positive architectural result, neither of which survived inspection. First, under expression-thresholded supervision the framework reaches AUROC 0.9864 +/- 0.0008 on TCGA-BRCA.

That figure partitions into 5,925 reactions whose labels are a deterministic threshold of the model's own input, where ranking by the cohort-mean input alone gives AUROC 1.000; and 4,675 reactions whose stored labels we reproduce bit for bit from a seeded pseudo-random number generator, where the model nonetheless reaches 0.9291 +/- 0.0030 by memorizing a patient-invariant label vector that patient-level splitting leaves fully visible during training. Second, on the cohort supervised independently of the input, the archived models never received patient data at all. Their released feature tensors are uniformly zero, and independently trained models show no agreement on which patient deviates where (|r|