Back to EROFS filesystem
GSoC 2026

Multi-Threaded Decompression Support in fsck.erofs

fsck.erofs currently relies on a strictly synchronous execution path for image unpacking and verification. This design causes the main thread to become heavily bottlenecked by synchronous I/O during file extraction, which subsequently blocks the CPU-intensive decompression phase. As EROFS images scale in size, this sequential pipeline creates a significant performance barrier across container environments, embedded systems, and developer workflows. This project resolves the synchronous I/O bottleneck by introducing a multi-threaded decompression pipeline built on a producer-consumer architecture. A background workqueue will enable the main thread to handle directory walks and queue compressed blocks, while a dedicated worker thread pool processes the decompression payloads. To eliminate heavy scheduling overhead and lock contention, the project introduces dynamic pcluster batching, which groups sequential blocks into a single work unit and scales the batch size based on the specific compression algorithm in use. Additionally, the extraction process will be further accelerated by a thread-safe, LRU-bounded metadata cache protected by read-write semaphores to prevent redundant I/O and strictly bound memory usage. The deliverables for this project include a scalable, multi-threaded extraction pipeline for fsck.erofs integrated with erofs_workqueue , alongside dynamic pcluster batching heuristics optimized for LZ4, LZMA, DEFLATE, and Zstandard to minimize futex overhead. Additionally, the project will provide a thread-safe, LRU-bounded metadata cache designed to significantly reduce redundant I/O , and the implementation will be validated through comprehensive regression tests, edge-case validations for corrupted images, and Flamegraph-backed performance benchmarks.

Project details

Contributor

Nithurshen

Mentors

Not available

Technologies

Not listed in the archive