Event Classification With Masked Transformer Autoencoders

The proposal titled "Event Classification With Masked Transformer Autoencoders" aims to enhance particle jet tagging by developing a Lorentz-equivariant Particle Transformer (Lorentz ParT) that adheres to the symmetries of special relativity. This hybrid architecture combines Lorentz Geometric Algebra Transformers (L-GATr) with Particle Transformers (ParT) to compute physical invariants, such as invariant mass, while tracking geometric orientations between particles. A key innovation is the introduction of Gated Particle Attention, which addresses residual bottlenecks in standard transformers by dynamically weighting contextual information against original features. The model is designed for a dual-head functionality, performing both self-supervised classification across ten unique jet classes and continuous mass regression using the JetClass dataset. To optimize learning, a Masked Autoencoder (MAE) approach is utilized to understand underlying jet symmetries before label-based training begins. Preliminary results indicate that the gated prototype can achieve a lower eta loss than original methodologies, although it currently faces challenges with "hallucinating noise" in reconstruction maps due to increased parameter counts. The proposed 12-week execution plan focuses on establishing mass regression, mitigating ROC curve issues, and validating the architecture against a scratch model to ensure stable performance