HalfLife

Pretraining data poisoning via public web discussion interfaces shown feasible at scale

A new arXiv paper demonstrates that LLM pretraining data can be poisoned through public discussion interfaces (e.g., comment sections, forums) at web scale, going beyond prior work that focused on controlled sources like Wikipedia. The authors introduce HalfLife, a novel analysis method for estimating how much adversarially injected content survives web crawling and data curation pipelines. The work establishes third-party webpage content as a viable attack vector and highlights the inadequacy of current curation pipelines as a defense against such poisoning.