Using Next-Gen Transformers to Seed Generative Models for Symbolic Regression

Symbolic regression (SR) aims to discover human-interpretable mathematical expressions from numerical data. While recent transformer models achieve high token-level accuracy on physics datasets, they struggle to recover exact, functionally correct equations. Conversely, generative techniques like Genetic Programming (GP) excel at numeric fitting but explore expression space blindly.

This project solves this structural gap by combining modern transformer priors with generative search. I will build upon previous ML4SCI projects to implement a hybrid pipeline: an xVal-encoded seq2seq transformer generates structurally sound candidate expressions, which seed a GP population. The GP outputs are then ranked and used to iteratively fine-tune the transformer via Direct Preference Optimization (DPO).