Parallel HNSW Index Construction for MariaDB Vector Search
MariaDB's HNSW vector index currently builds graphs sequentially, one row at a time. This creates two problems: ALTER TABLE on large tables stalls for hours because every vector insertion runs a CPU-intensive graph traversal on a single thread, and OPTIMIZE TABLE provides no quality improvement because it blindly rebuilds with the same hardcoded parameters, also leaving dead edges from deleted rows in the graph. This proposal addresses both by hooking into MariaDB's bulk insert API to defer all graph construction until after the base table is fully populated, then building the graph in parallel across multiple worker threads. Since OPTIMIZE TABLE already falls back to a table rebuild internally, both operations share the same unified path. The result is significantly faster index construction through parallelization, and meaningfully better graph quality through configurable rebuild parameters, with no changes to the normal single-row insert path.
Project details
Technologies
Not listed in the archive