Single Cell and DNA Foundation Models

DeepChem supports transformer-based foundation models for chemistry (ChemBERTa, MolFormer) but has no equivalent for genomic data. This project fills that gap by integrating DNABERT-2 into DeepChem, following the same design pattern used by ChemBERTa and MolFormer. The implementation consists of two components: (1) a DNABert model class that wraps HuggingFace's DNABERT-2 inside DeepChem's HuggingFaceModel/TorchModel hierarchy, supporting masked language modeling pretraining and fine-tuning for classification, regression, and multitask regression; and (2) a DNABertTokenizer that serves as both a HuggingFace tokenizer and a DeepChem featurizer, making it compatible with DeepChem's data loading pipeline. Deliverables include the fully integrated model and tokenizer classes, tests across all task modes, and benchmarks on a suitable genomic dataset.