TabICL

KnowsTFM: Knowledge graph-informed fine-tuning of small tabular foundation models

A new arXiv preprint introduces KnowsTFM, a method for fine-tuning small tabular foundation models (nanoscale TabPFN and TabICL variants) using structural attention priors derived from knowledge graphs and parameter-efficient low-rank updates. The approach targets niche domains with scarce, high-dimensional data shifted from pretraining distributions, showing meaningful gains in specialist settings but marginal gains on general tasks. The paper also reports that continual fine-tuning of frontier tabular models can trigger collapse of pretrained knowledge, a notable failure mode.