NIST SP 800-86

Empirical evaluation finds LLM watermarking methods fail forensic and legal evidentiary standards

A new arXiv paper tests three representative LLM watermarking schemes — KGW, Unigram, and SynthID-Text — against Daubert admissibility criteria and NIST SP 800-86 digital forensic standards, finding all three fail to meet the evidentiary bar courts require. Meaning-preserving paraphrase attacks removed watermarks in 100% of KGW and Unigram cases and 98.3% of SynthID cases; pre-attack false-negative rates were already 70–83%. The authors propose a Forensic Readiness Score (FRS) framework and conclude that none of the tested methods satisfy more than two of five Daubert factors. The findings directly challenge the regulatory assumptions underlying the EU AI Act and California's SB 942, both of which mandate reliable, robust AI content watermarking.