AI and Efficiency: Algorithmic Progress Halving Training Compute Every 16 Months Since 2012
OpenAI released an analysis showing that compute required to match AlexNet-level ImageNet performance has decreased 44x since 2012, with algorithmic efficiency doubling every 16 months. This outpaces Moore's Law, which would have yielded only an 11x improvement over the same period. The findings suggest that for heavily-invested AI tasks, algorithmic progress is a larger driver of efficiency gains than hardware improvements alone.
Related guides (4)
Related events (8)
AI and Compute: OpenAI Analysis of Exponential Growth in Training Compute Since 2012
OpenAI published an analysis in May 2018 showing that compute used in the largest AI training runs has been doubling every 3.4 months since 2012, far outpacing Moore's Law's 2-year doubling period. Over the 2012–2018 period, this metric grew by more than 300,000x. The analysis frames compute scaling as a key driver of AI progress and argues for preparing for systems with capabilities well beyond those of the time.
Is AI Progress Slowing Down?
A commentary piece from the AI Snake Oil newsletter examines recent claims and trends around whether AI progress is decelerating. The article appears to analyze the evidence for and against a slowdown in frontier AI development. As a tier-2 commentary source, it likely synthesizes public signals rather than presenting original research.
Anthropic Growing 10x/Year While Competitors Cut Workforce
A Latent Space newsletter item highlights a notable divergence in the AI industry: Anthropic is reportedly growing at roughly 10x per year while other AI/tech companies are conducting layoffs exceeding 10% of their workforces. The piece frames this as a significant economic dichotomy within the AI sector. The body is brief and reflective, offering limited technical detail.
Decade-long analysis of 56,800 AI conference papers finds sixfold increase in code/data sharing
A new arXiv preprint analyzes documentation and reproducibility practices across 56,800 papers from five leading AI conferences between 2014 and 2024. Code and data sharing rose nearly sixfold from 11% to 64%, with estimated reproducibility increasing from 28% to 64% over the same period. Notably, improvements in documentation practices predate the introduction of formal reproducibility checklists, suggesting the shift reflects a broader open-science movement rather than compliance with venue requirements.
One Year Since the "DeepSeek Moment"
A Hugging Face retrospective marking one year since the DeepSeek moment, which shook assumptions about AI development costs and open-weights competitiveness. The piece likely reflects on how DeepSeek's efficient training approach influenced the broader AI landscape, open-weights progress, and inference economics over the past year. Published on the anniversary of the original release, it offers industry analysis from a major open-source AI platform perspective.
How countries can end the capability overhang
OpenAI has published a report examining disparities in advanced AI adoption across countries and proposing initiatives to help nations capture productivity gains from AI. The report focuses on the gap between AI capabilities that exist and their actual deployment at scale—termed the 'capability overhang.' OpenAI frames this as a strategic and economic issue requiring coordinated national action. The report appears to be part of OpenAI's broader policy and international engagement efforts.
AI Scaling Myths
A commentary piece from normaltech.ai argues that AI scaling will eventually hit limits, framing the debate as a question of timing rather than whether limits exist. The piece appears to challenge prevailing optimism around continued scaling returns. Given the minimal body text, the depth of argument is unclear, but the topic directly engages the scaling laws debate central to frontier AI development.
Deep Double Descent: Universal Phenomenon in CNNs, ResNets, and Transformers
OpenAI researchers demonstrate that the double descent phenomenon—where model performance improves, degrades, then improves again—occurs universally across CNNs, ResNets, and transformers as a function of model size, data size, or training time. The effect can often be masked by careful regularization, which may explain why it has been underappreciated. The underlying mechanism remains poorly understood, and the authors identify it as an important open research direction.