Biological Foundation Models in Veterinary Virology: From ESMFold to Genomic Surveillance
Introduction
The application of deep learning to biological sequence data has undergone a paradigm shift with the emergence of foundation models. These large-scale neural networks, pre-trained on vast corpora of protein and nucleotide sequences, learn distributed representations that capture evolutionary, structural, and functional constraints. In veterinary virology, where pathogen diversity is immense and experimental structural data are scarce, these models offer transformative capabilities for protein structure prediction, variant effect prediction, and genomic surveillance. This article reviews the technical underpinnings of biological foundation models, with emphasis on ESMFold and related architectures, and examines their integration into surveillance pipelines for animal viruses.
Biological Foundation Models: Architecture and Pre-training
Foundation models in biology are typically transformer-based architectures trained using self-supervised objectives on sequence databases. For proteins, the masked language modeling objective (similar to BERT) is common: a fraction of amino acid residues are randomly masked, and the model learns to predict the original residues from context. The resulting embeddings encode information about secondary structure, solvent accessibility, contact maps, and evolutionary conservation.
The Evolutionary Scale Modeling (ESM) family, developed by Meta AI, exemplifies this approach. ESM-1b and ESM-2 are transformer models trained on UniRef and related databases containing hundreds of millions of protein sequences. ESM-2, with up to 15 billion parameters, achieves state-of-the-art performance on structure prediction tasks without explicit multiple sequence alignment (MSA) input. This is a critical advantage for viral proteins, which often lack deep sequence homologs due to rapid evolution and host adaptation.
Other foundation models include ProtBERT, ProtT5, and Ankh, each with different tokenization strategies and training regimes. For nucleotide sequences, models such as DNABERT and Enformer have been applied to regulatory element prediction, though their use in virology remains nascent.
ESMFold and Protein Structure Prediction
ESMFold is a structure prediction module built on top of ESM-2 embeddings. Unlike AlphaFold2, which requires an MSA generated from homologous sequences, ESMFold directly predicts atomic coordinates from the single sequence representation learned by the language model. The architecture consists of a folding trunk that processes the ESM-2 embeddings through a series of attention and equivariant transformer layers to produce backbone coordinates and side-chain angles.
For veterinary virology, ESMFold enables rapid structure prediction for viral proteins that are poorly represented in sequence databases. Examples include the hemagglutinin of avian influenza viruses, the spike protein of feline coronavirus, and the capsid proteins of canine parvovirus variants. The ability to generate high-confidence models in minutes rather than days facilitates structural analysis of emerging variants during outbreaks.
The accuracy of ESMFold is comparable to AlphaFold2 for proteins with high sequence coverage but degrades for very long or disordered proteins. However, for typical viral proteins (200-600 residues), ESMFold provides reliable backbone predictions that can be used for docking studies, epitope mapping, and rational vaccine design.
Genomic Surveillance Applications
Genomic surveillance of animal viruses relies on rapid sequencing and phylogenetic analysis to track emergence, spread, and antigenic drift. Foundation models enhance this pipeline in several ways.
Variant Effect Prediction
When a novel mutation is detected in a viral genome, its functional impact is often unknown. ESMFold can be used to predict the structural consequences of amino acid substitutions. For example, a mutation in the receptor binding domain of avian influenza H5N1 hemagglutinin can be modeled to assess changes in binding affinity to avian or mammalian sialic acid receptors. Similarly, mutations in the capsid of canine parvovirus (CPV-2a, CPV-2b, CPV-2c) can be evaluated for altered host range or immune escape.
Phylogenetic Contextualization
Embeddings from ESM models can serve as features for phylogenetic inference. Traditional methods use sequence alignment and substitution models, but embedding-based distances capture structural and functional similarity beyond primary sequence identity. This is particularly useful for highly divergent viruses where alignment is ambiguous, such as in the Coronaviridae family.
Host Range Prediction
Foundation models can be fine-tuned to predict host tropism from viral protein sequences. By training on known host-virus associations, the model learns to recognize sequence signatures associated with replication in specific hosts. This approach has been applied to influenza A virus to distinguish avian, swine, and human isolates.
Integration with Diagnostic Workflows
The incorporation of foundation models into routine veterinary diagnostics requires computational infrastructure and standardized pipelines. A typical workflow is illustrated in Figure 1.
graph TD
A[Clinical Sample], > B[Nucleic Acid Extraction]
B, > C[High-Throughput Sequencing]
C, > D[Genome Assembly]
D, > E[Variant Calling]
E, > F[ESMFold Structure Prediction]
F, > G[Structural Impact Assessment]
G, > H[Phylogenetic Analysis]
H, > I[Surveillance Report]
E, > J[Embedding Extraction]
J, > K[Host Range Prediction]
K, > I
F, > L[Docking Studies]
L, > M[Vaccine Antigen Design]
M, > I
Figure 1. Integrated genomic surveillance pipeline incorporating biological foundation models. Sequence data from clinical samples are processed through assembly and variant calling, then fed into ESMFold for structure prediction and into embedding-based classifiers for host range prediction. Outputs inform surveillance reports and vaccine design.
This pipeline is applicable to a wide range of veterinary pathogens. For example, in the context of Highly Pathogenic Avian Influenza (H5N1) in Poultry and Wild Birds, rapid structural assessment of hemagglutinin mutations can guide decisions on vaccine strain selection. Similarly, for Feline Leukemia Virus, structural modeling of envelope glycoprotein variants can predict immune evasion and inform diagnostic antigen design.
Challenges and Limitations
Despite their promise, biological foundation models face several challenges in veterinary virology.
Data Scarcity for Rare Pathogens
Foundation models are pre-trained on general protein databases that are dominated by model organisms and human pathogens. Viral proteins from understudied hosts (e.g., poultry ectoparasites like Dermanyssus gallinae or aquatic pathogens such as Streptococcus agalactiae in tilapia) may be poorly represented, leading to lower prediction accuracy.
Computational Cost
Running ESMFold or similar models requires GPU acceleration. While inference is faster than AlphaFold2, it still demands resources that may not be available in field laboratories. Cloud-based solutions or edge deployment on portable devices are areas of active development.
Interpretability
Deep learning models are often black boxes. Understanding why a particular mutation is predicted to be destabilizing or why a sequence is classified as avian-adapted remains difficult. Explainability methods (e.g., attention maps, integrated gradients) are improving but are not yet routine.
Validation
Structural predictions must be validated against experimental data. For many veterinary viruses, crystallographic or cryo-EM structures are unavailable. Cross-validation using homologous structures or molecular dynamics simulations can provide confidence, but experimental confirmation remains the gold standard.
Future Directions
The next generation of foundation models will likely incorporate multimodal data, including protein-protein interaction networks, metabolomic profiles, and clinical metadata. For veterinary virology, models that jointly encode host and pathogen sequences could predict cross-species transmission risk. Additionally, few-shot learning techniques may enable adaptation to novel viruses with minimal sequence data.
Integration with point-of-care diagnostics is another frontier. Portable sequencers combined with on-device inference of foundation models could provide real-time structural and functional analysis during outbreaks. This would be particularly valuable for diseases such as African Swine Fever, where rapid genotyping and virulence assessment are critical for containment.
Conclusion
Biological foundation models, exemplified by ESMFold, represent a major advance in computational virology. Their ability to predict protein structure and function directly from sequence, without reliance on homologous alignments, makes them uniquely suited to the diverse and rapidly evolving landscape of animal viruses. When integrated into genomic surveillance pipelines, these models enhance variant interpretation, host range prediction, and vaccine design. Continued development of accessible, validated, and interpretable models will solidify their role in veterinary diagnostics and outbreak response.
References
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C.L., Ma, J., Fergus, R. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proceedings of the National Academy of Sciences.
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., Bridgland, A. Highly accurate protein structure prediction with AlphaFold. Nature.
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., dos Santos Costa, A. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science.
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., Bhowmik, D. ProtTrans: Toward understanding the language of life through self-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence.
Rao, R., Bhattacharya, N., Thomas, N., Duan, Y., Chen, X., Canny, J., Abbeel, P., Song, Y.S. Evaluating protein transfer learning with TAPE. Advances in Neural Information Processing Systems.
Ji, Y., Zhou, Z., Liu, H., Davuluri, R.V. DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome. Bioinformatics.
Avsec, Ž., Agarwal, V., Visentin, D., Ledsam, J.R., Grabska-Barwinska, A., Taylor, K.R., Assael, Y., Jumper, J., Kohli, P., Jones, D.T. Effective gene expression prediction from sequence by integrating long-range interactions. Nature Methods.