Back to German Center for Open Source AI
GSoC 2026

Correcting Statistical Inference for Smooth Terms in pyGAM - Accurate p values

pyGAM currently produces anti-conservative p-values for smooth terms due to several deviations from the statistical theory outlined in Wood (2013b). In particular, it assumes a simple chi-squared reference distribution, ignores effective degrees of freedom corrections, and does not properly handle penalization-induced rank deficiency. This project will correct the statistical inference pipeline in pyGAM by implementing five key fixes: (1) proper computation of effective degrees of freedom, (2) stable pseudoinverse truncation, (3) replacement of the chi-squared approximation with a weighted mixture using the Liu-Tang-Zhang method, (4) QR-based covariance handling, and (5) correct treatment of unknown-scale families. The result will be a refactored inference module that produces well-calibrated p-values, along with a comprehensive validation suite comparing type-I error and power against mgcv. Stretch goals include incorporating smoothing parameter uncertainty corrections and exploring REML-based smoothing. Deliverables include a refactored statistics() pipeline, extensive tests, and improved documentation.

Project details

Contributor

vsl366

Mentors

Not available

Technologies

Not listed in the archive