Mitra

Empirical evaluation finds tabular foundation models degrade systematically under distribution shift

A new arXiv preprint evaluates nine tabular foundation models (TFMs) — including TabPFNv2 through v3, TabICL, Mitra, LimiX, and TabFM — on out-of-distribution performance using three real-world datasets from the TableShift benchmark covering label, socioeconomic, and geographic shifts. All evaluated TFMs show systematic performance degradation under distribution shift regardless of pre-training strategy, with shift gaps ranging from 0.003 to 0.060. The study also identifies a scalability gap, noting that high-performing TFMs require computational resources beyond standard deployment infrastructure. Results suggest TFMs inherit the same OOD fragility documented in classical tabular models, raising concerns for high-stakes deployment.