GPU-Accelerated VLM/VLA infrastructure on Bubbaloop
This project builds GPU-accelerated image processing kernels for kornia-rs and integrates them into a real-time vision-language model (VLM) pipeline on the Bubbaloop framework, targeting the Jetson Orin Nano. The work has two pillars. First, implementing GPU kernels (resize, normalize, color conversion, cast/layout reordering) using CubeCL, eliminating CPU-GPU synchronization bottlenecks in VLM preprocessing. Second, integrating Qwen2.5-VL-3B via ONNX/TensorRT into a live Bubbaloop camera pipeline, handling dynamic resolution tiling, vision token packing, and memory-constrained inference within the Jetson's 8GB limit. Time permitting, V-JEPA 2 will also be evaluated as a complementary world model for latent scene prediction. The end result positions kornia-rs as a practical foundation for deploying vision-language models on edge devices in Rust.
Project details
Technologies
Not listed in the archive