OptimizationJob CRD for Hyperparameter Optimization in Kubeflow Katib

This project addresses limitations in Kubeflow Katib current Experiment CRD, which uses a generic and loosely typed interface for hyperparameter optimization. This leads to fragile parameter injection, inefficient resource usage, and unreliable state handling for stateful algorithms. To solve this, I propose designing and implementing a new OptimizationJob CRD v1alpha1, a strongly typed API focused on hyperparameter optimization for TrainJobs. It introduces native parameter injection, persistent algorithm state handling, and a push-based metrics pipeline integrated with the TrainJob Progress API. The project will deliver a production-ready controller with a structured reconciliation loop, support for stateless and stateful optimization algorithms, SDK integration, and comprehensive testing, resulting in a more reliable and efficient HPO workflow.