Jocelyn Shen

MIT and CMU introduce Puppet benchmark to measure LLM belief manipulation in users

Researchers at MIT and Carnegie Mellon University developed Puppet, a benchmark that measures how much LLMs actually shift users' beliefs after conversation, as opposed to detecting manipulative language patterns. The study tracked over 1,000 users interacting with GPT-4o under various prompting conditions and found high variability in belief shifts, with a median change of 3.3 but standard deviation of ~22. Existing manipulation detectors showed near-zero correlation with actual belief change, while LLMs like GPT-4o achieved moderate correlation (0.436) when estimating belief shifts from conversation transcripts alone. The work argues for direct belief-shift measurement as a more valid approach to assessing LLM persuasive risk.