Back to The Linux Foundation
GSoC 2026

Real-Time Audio Capture and Playback Pipeline for Zephyr with Optional ML Integration

Zephyr provides low-level I2S and DMIC driver APIs along with a few narrow samples (echo, codec), but lacks a reusable application-layer audio pipeline that handles real-world complexity. The closest prior effort (GSoC 2025, PR #96657) attempted to integrate TensorFlow Lite Micro Micro Speech with a Linux host streaming setup, but the audio pipeline was tightly coupled with ML logic, limiting extensibility and resulting in fragile real-time behavior. A clean abstraction to support multiple backends is still absent. This proposal addresses these gaps by introducing a modular, backend-agnostic audio pipeline, designed around fixed-size PCM blocks and explicit ownership semantics. The system will support continuous audio streaming at 16 - 48 kHz, 16-bit PCM, targeting 4 - 10 ms frame latency with zero TX underruns and zero RX overruns under sustained load. Key deliverables include a pipeline API, support for multiple backends (I2S, PDM, optional USB), and reusable processing stages such as format conversion, and ML inference as an upstream sample.

Project details

Contributor

mohittalwar23

Mentors

Not available

Technologies

Not listed in the archive