olmo-1f5d874f·3 events·first seen Aliases: OLMo
Researchers introduce LACUNA, the first unlearning testbed with ground-truth parameter-level localization, designed to evaluate whether LLM unlearning methods truly erase knowledge from model weights or merely suppress it at the output level. The testbed injects PII of synthetic individuals into predefined parameters of 1B and 7B OLMo-based models via masked continual pretraining, enabling direct measurement of localization precision. Benchmarking current SOTA unlearning methods reveals they are highly imprecise and vulnerable to resurfacing attacks despite strong output-level performance, while successful localization enables even simple gradient-based methods to achieve robust erasure. The work addresses a critical gap in unlearning evaluation methodology relevant to privacy compliance and AI safety.
AllenAI published a blog post on Hugging Face introducing olmo-eval, an evaluation workbench designed to integrate into the model development loop. The tool appears aimed at streamlining evaluation workflows for researchers iterating on open-weights models. This is relevant to the OLMo model family ecosystem and the broader open-weights evaluation infrastructure space.
Interconnects covers the latest OLMo hybrid model release and discusses emerging trends in open-source post-training tooling. The piece examines architectural directions for future large language models. As a tier-2 commentary source, it provides analysis rather than primary research findings.