Abeba Birhane on Bias in Web-Scraped Training Datasets
Researcher Abeba Birhane examines how large-scale web-scraped datasets used to train trillion-parameter NLP and vision models propagate bias and antisocial content. The commentary highlights that performance gains in deep neural networks come alongside inherited societal biases from web training data. Two posts from The Batch summarize her work on cleaning up web datasets and the specific mechanisms by which NLP models absorb web-sourced biases.
Related guides (2)
Related events (8)
Ethics and Society Newsletter #4: Bias in Text-to-Image Models
Hugging Face's Ethics and Society team publishes their fourth newsletter focusing on bias in text-to-image generative models. The piece examines how these models encode and reproduce societal biases in visual outputs, likely covering evaluation methods, documented failure modes, and mitigation approaches. As a Tier 2 commentary piece from a major ML platform, it contributes to ongoing discourse around fairness and safety in multimodal AI systems.
Computational audit finds ClinicalBERT amplifies demographic bias beyond training data distributions
Researchers present a systematic audit of representational bias in ClinicalBERT, a BERT-based model pretrained on MIMIC-III clinical discharge summaries, using two probing methodologies: Log Probability Bias Analysis and Masked Language Model probing across 98 clinical sentence templates and eight intersectional race-gender combinations. Of 32 statistically significant findings, 65.6% contradict observed corpus distributions, rising to 80% for Black patients and 87.5% for agency attribution under MLM probing. The key finding is that bias in ClinicalBERT operates predominantly through model-internal amplification rather than simple inheritance from training data, which has direct implications for clinical AI safety and deployment. This challenges the assumption that auditing training corpora is sufficient to characterize model bias.
Automated Benchmark Auditing for AI Agents and Large Language Models (ABA)
The paper introduces Auto Benchmark Audit (ABA), an agentic framework that systematically audits AI benchmark tasks for issues such as ambiguous specifications, environment conflicts, and incorrect ground truths. Applied to 168 benchmarks across nine domains including NeurIPS publications, ABA identifies critical issues in over 25.7% of evaluated tasks. The authors demonstrate that filtering out flawed tasks materially shifts model rankings and improves average performance on SWE-bench Verified and Terminal-Bench 2 by 9.9% and 9.6% respectively, indicating that current benchmark scores are significantly distorted by task quality problems. The agentic tool and annotations are released publicly.
Data Points: NeurIPS-China Standoff, Anthropic Emotion Vectors, Gemma 4, Cursor 3, Microsoft MAI Models
This edition of The Batch covers five significant AI developments: NeurIPS reversed a sanctions-related submission policy after China's largest tech federation announced a boycott; Anthropic's interpretability team identified 171 emotion-related representations in Claude Sonnet 4.5 that causally influence model behavior including unsafe actions; Google released Gemma 4, a family of Apache 2.0-licensed open-weights models up to 31B parameters with strong benchmark performance; Cursor released version 3 with a redesigned multi-agent interface; and Microsoft announced three specialized MAI models for transcription, voice synthesis, and image generation. The NeurIPS incident highlights growing friction in international AI research access, while the Anthropic findings have direct implications for AI safety and interpretability research.
Reducing bias and improving safety in DALL·E 2
OpenAI announced a new technique applied to DALL·E 2 that adjusts image generation of people to better reflect global demographic diversity. The intervention targets representational bias in the model's outputs when generating human subjects. This is an early public example of a major lab deploying a post-training bias mitigation technique in a production image generation system.
Lessons learned on language model safety and misuse
OpenAI published a post summarizing their evolving thinking on language model safety and misuse in deployed systems. The piece is intended to share lessons with other AI developers facing similar challenges. It covers OpenAI's internal approaches to mitigating harmful outputs and misuse patterns observed in production.
Audit of 39 deepfake speech datasets reveals fairness and generalization gaps
A dataset-level audit of 39 deepfake speech datasets examines accessibility, documentation, demographic coverage, scale, and source corpora. The study finds that fairness assessment is largely infeasible due to missing demographic metadata, and that substantial overlap in underlying speech corpora across datasets undermines cross-dataset evaluation and inflates generalization claims. The findings challenge the credibility of robustness and fairness claims made for deepfake speech detectors.
Blind Users Can Use AI Models As Virtual Mirrors, But Don't Always Like What They See
Blind and visually impaired users are increasingly relying on vision-language models (notably GPT-4 Vision via Be My Eyes) to assess their own appearance, gaining independence but also encountering AI outputs that reflect conventional beauty standards and may be factually inaccurate. A BBC article by blind journalist Milagros Costabel documents cases where AI feedback was psychologically harmful, including unsolicited critical commentary on facial features. Psychologists warn that blind users are especially vulnerable because they cannot independently verify AI visual judgments. The piece raises broader questions about accuracy, trust calibration, and empathy in AI products designed for accessibility.