Entity · paper

Cost-Sensitive Conformal Prediction and Human-in-the-Loop Abstention for Imbalanced High-Stakes Decision Support: A Multi-Domain Benchmark

paperactiveprovisional

cost-sensitive-conformal-prediction-and-human-in-the-loop-abstention-for-imbalanced-high-stakes-decision-support-a-multi-domain-benchmark-ade62d8c

·1 events·first seen 19h ago

Aliases: Cost-Sensitive Conformal Prediction and Human-in-the-Loop Abstention for Imbalanced High-Stakes Decision Support: A Multi-Domain Benchmark

Co-occurring entities

Mondrian Conformal Prediction

More like this (12)

Conformal Prediction Conformal Decision Theory Mondrian Conformal Prediction Bias Benchmark for Question Answering Sparse Subspace-to-Expert Sharing for Task-Agnostic Continual Learning Beyond Aggregate Risk: Role-Stratified Conformal Risk Control for LLM Tool Calls Bayesian Inference and Decision Audits for Public Archives of Frontier AI Evaluations OptimismBench: Forecasting Bias and the Alignment Effect in Language Model Judgment Towards Explainable Adjudicative Variance: Quantifying Judicial Discretion via Gated Multi-Task Learning distributionally robust optimization Optimal Deterministic Multicalibration and Omniprediction A Continual Validation, Updating, and Decision-Making Framework for Self-Adaptive Digital Twins via Robust Model Predictive Control

Recent events (1)

4arXiv · cs.LG·19h ago·source ↗

Benchmark shows Mondrian conformal prediction restores minority-class coverage in high-stakes imbalanced classification

A new arXiv preprint benchmarks cost-sensitive conformal prediction methods across 15 real-world imbalanced tabular datasets, 7 classifiers, and 3 calibration techniques (3,150 total runs), finding that standard marginal conformal prediction drops minority-class coverage to as low as 0.5%. Class-conditional (Mondrian) conformal prediction recovers valid minority coverage with an average 61.7 percentage-point improvement, and combining it with cost-controlled abstention reduces expected decision cost under realistic human review budgets. The paper also derives dataset-specific break-even thresholds for when deferring to human experts becomes cost-effective, offering practical deployment guidance for credit scoring, fraud detection, healthcare, and industrial safety applications.

Evaluation and Benchmarking AI Safety Research Mondrian Conformal Prediction Cost-Sensitive Conformal Prediction and Human-in-the-Loop Abstention for Imbalanced High-Stakes Decision Support: A Multi-Domain Benchmark +1 more