Back to VideoLAN
GSoC 2026

Auto-synchronise downloaded subtitle with audio using whisper.cpp

External subtitles are one of the basic and most used features of VLC. VLC already has functionality to allow auto downloading of subtitles. But sometimes these subtitles go out of sync mainly due to frame rate mismatches between the video and subtitle file, different video versions (e.g., Director's Cut), or technical issues like faulty encoding or laggy streaming. This causes subtitles to be consistently early/late or to gradually drift further off-sync over time. Currently vlc doesn’t have any inbuilt auto sync feature. Due to this users have to manually sync (by setting offset and frame drift) or rely on the external program to sync the subtitle. This project aims to solve this issue by using light VAD (Voice Activity Detection) and the lightest whisper model to match subtitles. The main goal would be to make it as efficient as possible so it can even run on mobile devices. Deliverables: 1. Implement a dummy solution using python. 2. Implement native vlc module to match subtitles. 3. Optimize and test the module on various platforms. 4. Update documentation to reflect project contributions.

Project details

Contributor

Mangal Kushwah

Mentors

Not available

Technologies

Not listed in the archive