NV-JEPA-DNA-NTv3

Model Overview JEPA-DNA-NTv3-100M is the Nucleotide Transformer v3 (NTv3) 100M backbone of the JEPA-DNA family, a model-agnostic continual pre-training framework that grounds genomic foundation models by coupling standard token-level DNA language modeling with a Joint-Embedding Predictive Architecture (JEPA). Where conventional genomic models rely purely on masked language modeling and prioritize local token reconstruction, JEPA-DNA additionally supervises the model's global sequence embedding in a latent space, forcing it to predict the functional representation of masked genomic segments rather than their literal tokens. This shifts the learning signal from token recovery toward semantic alignment, yielding more linearly-separable and biologically relevant features while preserving the generative precision of the underlying backbone. This checkpoint is an exact parameter match to the original research work described in the JEPA-DNA paper, and the code for using it can be found in the official JEPA-DNA Github repository. Description: JEPA-DNA-NTv3-100M is the NTv3 variant of JEPA-DNA, a genomic foundation model training framework that adds a Joint-Embedding Predictive Architecture (JEPA) objective to standard DNA language-model pretraining. It applies a continual pre-training stage on top of the NTv3 100M backbone — a 100M-parameter hybrid U-Net / Transformer model trained with a masked language modeling (MLM) objective over single nucleotides — combining token-level learning with latent-space prediction to improve functional genomic representations. This model is ready for non-commercial use. License/Terms of Use: Governing Terms: Use of this model is governed by the NVIDIA non-commercial license, subject to completion of NVIDIA's applicable internal legal and open source release review process. Deployment Geography: Global Use Case: JEPA-DNA checkpoints are intended for genomics research, representation learning, and downstream sequence modeling workflows. Typica