conversational-agents-7e0cc221·2 events·first seen Aliases: conversational agents, AV2AV conversational agents
VideoFDB is introduced as the first benchmark targeting full-duplex audio-visual-to-audio-visual (AV2AV) conversational agents, filling a gap where existing full-duplex benchmarks evaluate only speech. It provides 237 dyadic video-call clips covering 11 nonverbal conversational dynamics, a perception/generation taxonomy, and an LM-as-judge rubric framework. Evaluation across open- and closed-source vision-speech agents reveals systematic failure modes including captioning collapse and visual-stream ignorance, and shows current systems cannot perform the streaming joint audiovisual grounding required for natural conversation. Cascaded speech-to-avatar architectures are found to be architecturally incapable of producing full-duplex nonverbal cues.
Hugging Face published a blog post introducing Ecom-RLVE, a framework for training e-commerce conversational agents using reinforcement learning with verifiable environments. The approach creates adaptive environments that can verify agent actions and outcomes in e-commerce contexts, enabling RL-based training signals. This represents an application of the RLVR (Reinforcement Learning with Verifiable Rewards) paradigm to a specific commercial domain.