5Simon Willison's Weblog·
Simon Willison: Reverse-engineering is cheap now with LLMs
Simon Willison argues that LLMs have dramatically lowered the cost and skill barrier for reverse-engineering tasks. The post likely covers practical workflows where AI assistance makes previously expert-level analysis accessible to generalist developers. This is a practitioner-level observation about a meaningful capability shift in software analysis work.
Related guides (2)
Related events (8)
Google Study Shows LLM-Generated Malware Is Getting Harder to Track and Stop
A Google security report catalogs emerging LLM-enabled cyberattack techniques including morphing malware with mutation engines, logical-flaw discovery in code, and AI-directed obfuscation networks. The report was prompted in part by a real incident where hackers used an LLM to find a zero-day in a widely used web administration tool. Separately, the UK AI Security Institute found that Claude Mythos Preview and GPT-5.5 can reliably execute attacks expected to take humans 3 hours, up from earlier 1-hour benchmarks, with performance scaling further when token limits are relaxed. The findings suggest an accelerating gap between LLM offensive capability and conventional defensive tooling.
Software engineer describes LLM-driven career erosion in high-engagement HN post
A software engineer's personal blog post describing how LLMs are eroding their career prospects attracted 722 upvotes and 681 comments on Hacker News. The post reflects growing practitioner anxiety about AI displacement in software engineering roles. High engagement signals this as a meaningful community sentiment data point about how developers perceive LLMs affecting their livelihoods.
Practitioner spends $1,500 testing LLM offensive security capabilities against a purpose-built vulnerable app
A developer built a deliberately vulnerable application and ran LLMs against it as automated penetration testers, spending $1,500 on API costs across the experiment. The post evaluates how well current LLMs can identify and exploit real vulnerabilities in a controlled setting. Results provide practical signal on the current state of LLM-assisted offensive security, a capability area with both red-team and safety implications.
The last six months in LLMs in five minutes
Simon Willison publishes a rapid-fire retrospective covering the major LLM developments of the past six months. As a tier-2 commentary source, the piece synthesizes frontier model releases, tooling shifts, and ecosystem trends into a condensed overview. The body content was not provided, so specific claims cannot be assessed, but the framing suggests a broad industry-analysis sweep rather than a single technical finding.
LLMs automate reproducibility assessments in social and behavioral sciences, outperforming human reanalysts
A preprint from arXiv demonstrates that an LLM pipeline can automate reproducibility assessments of published social and behavioral science studies, recovering original effect sizes in 41% of cases (vs. 34% for human reanalysts) and reaching the same qualitative conclusion in 96% of cases (vs. 74% for humans). The study evaluated 76 published studies with predefined claims. The results suggest LLMs could serve as a scalable tool for systematic auditing of empirical research, addressing the resource-intensive nature of traditional reproducibility efforts.
Simon Willison builds LLM-powered cliché highlighter tool
Simon Willison describes building an LLM-based tool that highlights clichéd phrases in text. The post is a practical demonstration of using language models for stylistic text analysis. It represents a lightweight capability demo from a well-known AI practitioner.
LLMs can extract and reuse experiential abstractions to improve reasoning, study finds
Researchers investigate whether LLMs can distill solution traces into reusable natural-language abstractions (strategies, reminders) and apply them to future problems, analogous to human experiential learning. Using the MATH training set, abstractions are extracted by either a stronger teacher model or the LLMs themselves into a retrievable library, then applied via inference-time retrieval or RL with abstraction-augmented prompts. Both modes improve performance on mathematical and logical reasoning benchmarks, with self-extracted abstractions matching teacher-extracted ones. The framework generalizes across datasets and models, suggesting a promising direction for continual self-improvement in LLMs.
ImportAI 449: LLMs training other LLMs; 72B distributed training run; computer vision is harder than generative text
Import AI issue 449 covers several AI/ML developments including LLMs being used to train other LLMs, a 72B parameter distributed training run, and analysis of why computer vision remains harder than generative text. The newsletter also touches on potential political implications of AI progress. As a tier-2 commentary source, this aggregates and contextualizes multiple technical developments across the AI landscape.