Back to Python Software Foundation
GSoC 2026

borgbackup: Borg Pack Files for BorgBackup 2.x Proposal

BorgBackup 2.x stores each deduplicated chunk as a separate borgstore object, resulting in millions of tiny files and severe performance regressions on remote/cloud backends. This project implements client-side pack files that bundle multiple chunks into larger files (~50 MiB default) before transfer, reducing API calls by 100-1000x. Deliverables: pack file binary format with per-pack index, extended ChunkIndex with pack location tracking (borghash HashTableNT), PackWriter/PackReader I/O modules, pack-aware garbage collection in borg compact, pack validation in borg check, ChunkIndex rebuild from packs, and backend-specific benchmarking. All phases produce working, mergeable code targeting local backends first with remote validation following.

Project details

Contributor

ebuzerdrmz44

Mentors

Not available

Technologies

Not listed in the archive