Support for Logarithmic Number Systems in Large Language Models
LLMs rely on floating-point arithmetic for inference, but the Logarithmic Number System (LNS) offers a compelling alternative: multiplication becomes integer addition in the log domain, and 16-bit LNS (xlns16) uses only 2 bytes per element, half the memory bandwidth of float32. This project integrates the open-source xlnscpp C++ LNS library into ggml (the compute backend of llama.cpp) as a proper ggml_backend, enabling LLM inference to run entirely in xlns16 arithmetic as a proof of concept. The implementation introduces GGML_TYPE_LNS16 as a native ggml datatype and implements the 13 operations required for a complete transformer inference pass (matrix multiply, softmax, RMS norm, RoPE, SiLU, residual add, along with some others), all using xlns16 kernels. Activations stay in xlns16 format throughout inference; weights are converted dynamically from their stored quantised formats (Q4_K_M, etc.) per kernel invocation using O(1) lookup tables. The backend is validated at two levels: a unit test confirms sub-0.6% relative error on matrix multiply, and an end-to-end run on SmolLM2-135M-Instruct confirms the full transformer executes without crashing. Deliverables: (1) GGML_TYPE_LNS16 datatype registered in ggml; (2) a complete ggml_backend for xlns16 arithmetic covering all ops needed for LLM inference; (3) integration with llama.cpp; (4) a validation test suite; (5) an accuracy analysis comparing LNS and FP output across model layers
Project details
Technologies
Not listed in the archive