5arXiv cs.LG (Machine Learning)·
Empirical evaluation finds tabular foundation models degrade systematically under distribution shift
A new arXiv preprint evaluates nine tabular foundation models (TFMs) — including TabPFNv2 through v3, TabICL, Mitra, LimiX, and TabFM — on out-of-distribution performance using three real-world datasets from the TableShift benchmark covering label, socioeconomic, and geographic shifts. All evaluated TFMs show systematic performance degradation under distribution shift regardless of pre-training strategy, with shift gaps ranging from 0.003 to 0.060. The study also identifies a scalability gap, noting that high-performing TFMs require computational resources beyond standard deployment infrastructure. Results suggest TFMs inherit the same OOD fragility documented in classical tabular models, raising concerns for high-stakes deployment.
Related guides (2)
Related events (8)
KnowsTFM: Knowledge graph-informed fine-tuning of small tabular foundation models
A new arXiv preprint introduces KnowsTFM, a method for fine-tuning small tabular foundation models (nanoscale TabPFN and TabICL variants) using structural attention priors derived from knowledge graphs and parameter-efficient low-rank updates. The approach targets niche domains with scarce, high-dimensional data shifted from pretraining distributions, showing meaningful gains in specialist settings but marginal gains on general tasks. The paper also reports that continual fine-tuning of frontier tabular models can trigger collapse of pretrained knowledge, a notable failure mode.
Distilling Tabular Foundation Models for Structured Health Data
This paper investigates knowledge distillation from tabular foundation models (TFMs) to lightweight student models for healthcare applications. The authors address context leakage in in-context TFMs via stratified out-of-fold teacher labeling, evaluating across 19 healthcare datasets, 6 TFM teachers, and 4 student families. Distilled students retain at least 90% of teacher AUC while running 26× faster on CPU, with preserved calibration and fairness properties. Multi-teacher ensembles do not consistently outperform the best single teacher.
Ensembling Tabular Foundation Models: A Diversity Ceiling and a Calibration Trap
This paper benchmarks six ensemble strategies across six tabular foundation models (TFMs) on 153 OpenML classification tasks, finding that ensembling provides minimal gains over the best single TFM. The best ensemble strategy (two-level cascade stacking) achieves only +0.18% accuracy improvement at 253× the compute cost. A key finding is that logistic-regression meta-learner stacking improves accuracy while severely degrading calibration (log-loss), because sharpening class boundaries destroys probability estimates. The authors recommend greedy ensemble selection as the practical default.
TABVERSE benchmark isolates table representation effects across formats in LLMs and VLMs
TABVERSE is a new controlled multimodal benchmark that evaluates LLMs and VLMs on table understanding by holding table content fixed while varying representation format (HTML, Markdown, LaTeX, rendered images). Evaluation across three tasks—Question Answering, Structural Understanding, and Structure Reconstruction—shows that representation choice substantially affects performance, with structured text generally outperforming rendered images and HTML being the most robust text format. The benchmark addresses a gap in existing evaluations where content, format, and modality vary simultaneously, making it impossible to isolate representation effects.
On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters
This paper reframes parameter-efficient fine-tuning (PEFT) not merely as a cheaper alternative to full fine-tuning, but as a substrate for persistent, instance-specific personal models layered atop shared foundation models. The authors analyze three scaling axes: Scale Up (stronger base models amplifying adapter utility), Scale Down (minimum viable adapter size), and Scale Out (managing millions of concurrent adapted instances). They introduce MinT as an infrastructure reference for adapter identity, versioning, provenance, evaluation, and serving at scale.
Toto 2.0: Open-Weights Time Series Foundation Models Demonstrate Scaling Laws from 4M to 2.5B Parameters
Datadog releases Toto 2.0, a family of five open-weights time series forecasting models ranging from 4M to 2.5B parameters, demonstrating consistent forecast quality improvements with scale. The models achieve state-of-the-art results on three benchmarks: BOOM (observability), GIFT-Eval (general-purpose), and TIME (contamination-resistant). The release includes architectural details, a u-muP hyperparameter transfer pipeline, and all base checkpoints under Apache 2.0 license.
ODTQA-FoRe: Open-Domain Tabular QA Dataset for Future Data Forecasting and Reasoning
The paper introduces ODTQA-FoRe, a new benchmark dataset for open-domain tabular question answering focused on time-series forecasting and forecast-based reasoning using real estate data. The authors also propose TimeFore, an LLM agent framework that decomposes the task into three roles: a SQL-generating Retriever, a Forecaster that calls external time-series models, and an Analyzer that synthesizes results. The work targets a gap in existing tabular QA systems, which typically cannot perform future-oriented numerical prediction. Experiments demonstrate TimeFore's effectiveness on the new benchmark.
Benchmark of deep learning architectures for multi-horizon behavioural forecasting in mobile health
A new arXiv preprint benchmarks six deep learning architectures, two zero-shot foundation models, and statistical baselines on multi-horizon behavioural forecasting from wearable and smartphone data across 800+ participants. Key findings include: no single architecture dominates (PatchTST leads among trained models), TimesFM matches or exceeds trained models zero-shot especially in low-data regimes, and participant-level fine-tuning reduces per-feature RMSE by 16–60%. The study is the first to jointly evaluate modern deep learning, foundation models, and personalisation for this domain.