End to End Open Verifiable LLM: Deterministic Training & Verification

Open-weight LLMs are reproducible in principle but not verifiable in practice. You can download weights, but you cannot prove what data they were trained on or whether they were modified later. This project closes that gap by building a training pipeline where identical configurations produce identical model states, and any deviation is cryptographically detectable. A key insight from my experiments is that while training computation can be deterministic, PyTorch’s .pt serialization breaks verification: identical weights can produce different checkpoint files. This project resolves that by verifying tensor-level state using byte-stable formats rather than file-level artifacts. The deliverable is a 1B–3B model trained on a pinned Wikipedia dataset, along with a public manifest that enables independent reproduction and audit.