GPU Accelerated Motion Compensation-based Video Frame Interpolation in libplacebo

Video frame interpolation (VFI) refers to the process of increasing the framerate or frames-per-second (FPS) of digital media beyond the initial framerate that is explicitly available (e.g., interpolating 24fps video to 60fps video), and is desirable to be done in real-time using GPU-accelerated algorithms to allow for use in media players such as VLC Media Player. This technique is not available to a broad audience through open source software, with implementations either being proprietary, unsuitable for real-time use, or unsuitable for use in a media player setting. We will solve this problem in libplacebo, VideoLAN's GPU-accelerated video processing library, by implementing a motion compensation-based VFI algorithm, a class of VFI algorithms well suited for real-time performance. This algorithm will require two stages: motion estimation (predicting the motion of pixels from one frame to the next), and motion interpolation (using motion estimates to estimate an intermediate frame between the two). This algorithm will be robust to common VFI challenges, using an image pyramid for large motion handling, and bidirectional motion estimation for occlusion handling. We will clearly document VFI in libplacebo to enable easy integration into software projects, such as media players. We will document the performance, the speed and accuracy of our VFI implementation, using common benchmarks from literature such as Adobe240FPS, or the Xiph dataset, using metrics such as SSIM as well as visually identifying artifacts, to better understand the computational efficiency of our algorithm (performance on 1080p vs 4k resolution, performance when interpolating multiple frames 24->120fps) and accuracy under varying input (large motion caused by high resolution).