technique

Stratified Out-of-Fold Teacher Labeling

techniqueactivestratified-out-of-fold-teacher-labeling-6be65b21·1 events·first seen 29d ago

Aliases: Stratified Out-of-Fold Teacher Labeling

Co-occurring entities

knowledge distillation AUC Tabular Foundation Models (TFMs)

More like this (12)

RL-Teacher Positional Encoding Soft Label Supervision soft-label learning Federated Learning Selective Classification Structured Outputs OWL Structured Interactive Learning Instruction Hierarchy Reversible Residual Layers OFA (One-For-All)

Recent events (1)

5arXiv · cs.AI·29d ago·source ↗

Distilling Tabular Foundation Models for Structured Health Data

This paper investigates knowledge distillation from tabular foundation models (TFMs) to lightweight student models for healthcare applications. The authors address context leakage in in-context TFMs via stratified out-of-fold teacher labeling, evaluating across 19 healthcare datasets, 6 TFM teachers, and 4 student families. Distilled students retain at least 90% of teacher AUC while running 26× faster on CPU, with preserved calibration and fairness properties. Multi-teacher ensembles do not consistently outperform the best single teacher.

Evaluation and Benchmarking Inference Economics knowledge distillation Stratified Out-of-Fold Teacher Labeling AUC +2 more