AI Snake Oil
ai-snake-oil-f4533355·12 events·first seen 1mo agoAliases: AI Snake Oil
Co-occurring entities
More like this (12)
Recent events (12)
New paper: AI agents that matter
A paper from the AI Snake Oil / Normal Tech group critiques current AI agent benchmarking and evaluation practices. The work argues that existing agent benchmarks are poorly designed for assessing real-world utility, and calls for rethinking how agent performance is measured. The commentary targets the gap between benchmark scores and practical deployment value.
AI as Normal Technology
A paper by the AI Snake Oil authors argues that AI should be understood as 'normal technology' rather than as something categorically unprecedented, a framing they plan to expand into a book. The piece appears to challenge dominant narratives about AI exceptionalism. The body is minimal, suggesting this is a teaser or announcement for forthcoming work.
Do AI Risks Require Extraordinary Government Intervention?
A commentary piece from the AI Snake Oil newsletter (published via normaltech.ai) examines whether AI risks justify extraordinary government intervention. The piece appears to argue against shortcuts in AI governance, emphasizing the importance of rigorous policy work. The article engages with ongoing debates about the appropriate scope and urgency of regulatory responses to AI.
Why AI hasn't replaced software engineers, and won't
A commentary piece from the AI Snake Oil / Normal Tech newsletter argues that coding agents should be understood as normal technology rather than transformative replacements for software engineers. The piece examines why AI has not displaced software engineering roles despite significant capability advances. This is a skeptical industry analysis relevant to ongoing debates about AI's impact on software development labor.
Is AI Progress Slowing Down?
A commentary piece from the AI Snake Oil newsletter examines recent claims and trends around whether AI progress is decelerating. The article appears to analyze the evidence for and against a slowdown in frontier AI development. As a tier-2 commentary source, it likely synthesizes public signals rather than presenting original research.
Could AI Slow Science? Confronting the Production-Progress Paradox
A commentary piece from AI Snake Oil explores the potential paradox whereby AI tools increase scientific output volume while simultaneously slowing genuine scientific progress. The piece examines how AI-assisted research production may prioritize quantity over quality, potentially crowding out deeper, slower-moving inquiry. This raises structural concerns about how AI integration into research workflows could reshape the incentive landscape of science.
Did Google's AI agents really build an operating system for $916?
This commentary piece from AI Snake Oil examines a Google claim that AI agents built an operating system for $916, emphasizing the need for independent evaluation of such capability announcements. The piece appears to scrutinize the methodology and framing behind the claim rather than accepting it at face value. It raises questions about how AI agent productivity claims are measured and verified.
We Looked at 78 Election Deepfakes. Political Misinformation is not an AI Problem.
An analysis of 78 election-related deepfakes argues that political misinformation is fundamentally not an AI problem, challenging the prevailing narrative that AI-generated content is the primary driver of electoral disinformation. The piece contends that technology is neither the root cause nor the solution to political misinformation. Published on the AI Snake Oil / Normal Tech platform, this represents a data-informed commentary pushing back on AI-centric framings of election integrity concerns.
Scientists should use AI as a tool, not an oracle
This commentary critiques the feedback loop between AI hype and scientific research, arguing that scientists who treat AI systems as oracles rather than tools produce flawed research that in turn amplifies further hype. The piece examines how uncritical adoption of AI in scientific workflows can compromise research integrity. It calls for a more epistemically disciplined approach to AI use in science.
Open-world evaluations for measuring frontier AI capabilities: Introducing CRUX
This commentary introduces CRUX, a new evaluation project designed to assess frontier AI systems on long-horizon, open-ended, and messy real-world tasks. The piece argues that existing benchmarks are insufficient for capturing the full range of capabilities exhibited by frontier models in complex settings. CRUX aims to fill this gap by providing evaluations that better reflect deployment-relevant performance.
New Paper: Towards a Science of AI Agent Reliability
A new paper proposes a framework for quantifying the gap between AI agent capability and reliability, aiming to establish a more rigorous science of agent dependability. The work addresses the observation that agents may demonstrate high capability on benchmarks while failing unpredictably in deployment. The piece is published via the normaltech.ai newsletter, associated with the AI Snake Oil research commentary tradition.
Does the UK's liver transplant matching algorithm systematically exclude younger patients?
This commentary examines whether the UK's liver transplant matching algorithm contains technical design choices that systematically disadvantage younger patients. The piece argues that seemingly minor algorithmic decisions can have life-or-death consequences in high-stakes medical AI systems. It falls within the broader discourse on algorithmic fairness and unintended bias in deployed AI/ML systems.