vals-ai-9a0750f7·2 events·first seen Aliases: Vals AI
Multiple independent organizations found they could not fully evaluate Claude Fable 5 (the public-facing safeguarded version of Claude Mythos 5) because Anthropic's classifiers silently rerouted flagged prompts to the weaker Claude Opus 4.8 or refused them outright. Evaluators including Artificial Analysis, Vals AI, and ARC Prize Foundation each adopted different scoring strategies — blended, pure, or abstaining entirely — producing widely divergent rankings depending on how refusals were handled. On GPQA Diamond, Claude Fable 5's score swung from 93.18% (2nd place) to 55.56% (94th place) depending on whether refusals were counted as failures. The episode surfaces a structural tension between safety-oriented deployment constraints and the ability of the field to independently measure frontier model capabilities.
Anthropic is expanding its Claude for Financial Services offering with a beta Excel add-in (Claude for Excel), seven new real-time data connectors (including LSEG, Moody's, Aiera, and Chronograph), and six new pre-built Agent Skills covering tasks like DCF modeling, comparable company analysis, and initiating coverage reports. The updates build on Claude Sonnet 4.5's performance on the Finance Agent benchmark from Vals AI, where it scored 55.3% accuracy. Claude for Excel allows users to read, analyze, modify, and create Excel workbooks directly from a sidebar, with transparency into cell-level changes. These features are rolling out in preview to Max, Enterprise, and Teams users, with Citi cited as a notable enterprise adopter.