conformal-prediction-0dc5b0a7·2 events·first seen Aliases: Conformal Prediction
A new arXiv preprint proposes and evaluates uncertainty-aware decision-making algorithms for LLMs grounded in Bayesian decision theory and risk-averse decision making, applied to tutoring and automatic peer review tasks. The authors incorporate conformal prediction to provide formal guarantees over strategy and score outputs. Empirical results show Bayesian methods outperform risk-averse rules, which can degrade to generic outputs under high ambiguity. The work highlights a gap in decision-making algorithm research relative to model training advances.
This paper proposes an algorithmic framework to certify high-probability safety of belief-space safety filters (BeliefSF) in interactive robotics, addressing the challenge that neural approximations and runtime inference errors make formal guarantees difficult. The approach uses conformal prediction focused on regions where inference is reliable, preserving standard sample complexity while certifying a less conservative filter. Evaluation on a simulated human-vehicle interaction benchmark demonstrates the method produces significantly more permissive safety guarantees than a standard conformal prediction baseline.