4Simon Willison's Weblog·
Simon Willison on Kimi K3 and the pelican benchmark as a model evaluation tool
Simon Willison writes about Kimi K3, a new model from Moonshot AI, using his informal 'pelican benchmark' as a lens for evaluation. The post reflects on what idiosyncratic, qualitative benchmarks can still reveal about model behavior that formal evals miss. As a tier-2 commentary piece, it offers practitioner-level perspective on a new open-weights or API-accessible model.
Related guides (3)
Related events (8)
Simon Willison quotes Kimi K3
Simon Willison shares a brief quote or observation about Kimi K3, a model from Moonshot AI. The content body is empty, suggesting this is a short link or quote post. The item signals community attention to the Kimi K3 model release or capability.
Kimi K2.6: Moonshot AI's 1T-Parameter Vision-Language Model Matches Open-Weights Peers, Trails Top Closed Models
Moonshot AI released Kimi K2.6, a 1 trillion-parameter mixture-of-experts vision-language model with 32B active parameters, designed for long-horizon autonomous coding sessions lasting multiple days and multi-agent orchestration scaling to 300 parallel subagents executing up to 4,000 steps. The model matches Qwen3.6 Max Preview and DeepSeek-V4-Pro on the Artificial Analysis Intelligence Index (scoring 54 vs. their 52) while trailing closed models like GPT-5.5 and Claude Opus 4.7. Weights are freely downloadable from Hugging Face under a modified MIT license permitting commercial use, with API access priced at $0.95/$0.16/$4.00 per million input/cached/output tokens. Notable features include a 256K token context window, native INT4 quantization, a 'preserve thinking' mode for multi-turn reasoning continuity, and a research preview 'claw groups' feature enabling cross-developer agent collaboration.
Kimi K3 2.8T-A50B released: largest open model ever, Opus 4.8-class performance at Sonnet 5 pricing
Moonshot AI has released Kimi K3, a 2.8 trillion total parameter MoE model with 50 billion active parameters, described as the largest open model ever released. The model is reported to achieve performance comparable to Claude Opus 4.8 while being priced at the level of Sonnet 5, representing a significant cost-performance advance. This release continues a strong week for open-weights models and raises the ceiling for publicly available model scale.
Simon Willison argues better AI models may produce worse tooling ecosystems
Simon Willison publishes a commentary piece titled 'Better Models: Worse Tools,' suggesting a potential inverse relationship between frontier model capability improvements and the quality or utility of the surrounding tool ecosystem. The piece appears to examine how advances in model capability may reduce incentives or change the design space for tooling built around those models. As a widely-read practitioner voice, Willison's framing could influence how developers think about the agent and tooling landscape.
Kimi Code CLI: MoonshotAI releases command-line agent tool
MoonshotAI has published kimi-cli, a Python-based command-line agent tool branded as 'Kimi Code CLI'. The repository has accumulated 9,341 GitHub stars with 48 added today, indicating meaningful developer interest. This is a CLI agent harness built around MoonshotAI's Kimi models.
GPT-5.5 Outperforms Benchmarks but Leads in Hallucination Rate; Kimi K2.6 Tops Open LLMs
GPT-5.5, OpenAI's latest closed vision-language model built for agentic coding and computer use, tops the Artificial Analysis Intelligence Index and ARC-AGI-2 benchmarks but exhibits a significantly higher hallucination rate (85.53%) compared to Claude Opus 4.7 (36.18%) and Gemini 3.1 Pro Preview (49.87%) on the AA-Omniscience benchmark. GPT-5.5 Pro processes reasoning tokens in parallel during inference, and pricing is roughly double GPT-5.4 rates. The model ranks lower on subjective Arena.ai leaderboards, where Claude Opus models dominate. The issue also notes Kimi K2.6 leading open-weight LLMs, though details on that item are truncated.
Gemma 4 and what makes an open model succeed
A commentary piece from Interconnects analyzing Google's Gemma 4 release and the broader question of what drives success for open-weight models. The piece argues that benchmark scores are not the primary determinant of open model adoption or impact. This is a tier-2 analytical take on the open-weights ecosystem and the strategic dynamics around model releases.
Simon Willison introduces 'The AI Compass' framework
Simon Willison published a piece titled 'The AI Compass,' likely presenting a conceptual framework or mental model for navigating AI decisions, use cases, or risks. The body content was not provided, but given Willison's track record, this is likely a substantive analytical or strategic piece aimed at practitioners. As a tier-2 commentary source, it represents informed independent analysis rather than a primary lab announcement.