Limitations of genomic language models for realistic sequence generation

Genomic language models (gLMs) are increasingly explored for synthetic genome generation, but it is still unclear how well-generated sequences resemble natural genomes. We evaluated Evo 2 across 210 prokaryotic, eukaryotic, and viral genomes and megaDNA in bacteriophages. Generated sequences reproduced local sequence statistics but differed from natural DNA in k-mer spectra, chaos game representations, nullomer patterns, non-B DNA motifs, and transcription factor binding site organization. In most cases, these differences exceeded natural within-species variation. Classifiers readily distinguished generated from natural sequences, with AUROC values reaching 0.98 in eukaryotes and 0.88 in prokaryotes, and this distinction increased with distance from the conditioning seed. Simple k-mer approaches performed similarly to the neural classifier. Moreover, correcting individual artifacts did not resolve the overall differences but shifted them to other genomic features. Thus, reproducing local sequence properties does not necessarily result in a sequence that reflects the broader organization of a natural genome.