Benchmarking and Performance Evaluation of Inference Routing Extensions in kgateway

kgateway routes AI inference traffic to self-hosted model servers. It picks the best server using EPP scoring — based on KV cache usage, queue depth, and LoRA affinity. That scoring costs more than simple round-robin routing, but there's no data on how much latency it actually adds or what a normal regression looks like for this setup. This project adds the benchmarking layer kgateway is missing. I'll build a reproducible test environment using the upstream GIE methodology, run scenarios comparing agentgateway with inference extension against a plain Kubernetes Service, set up a two-tier CI pipeline (mock server for regular runs, real Azure GPU for releases), store results in Azure Blob, and add regression detection that blocks CI when p99 latency crosses a stored threshold. The regression detection piece was what was missing from the issue discussion.