4arXiv cs.AI (Artificial Intelligence)·
PRISM-AH: Multimodal video framework for ambivalence and hesitancy recognition in health behaviour contexts
A new arXiv preprint introduces PRISM-AH, a multimodal video understanding framework designed to detect ambivalence and hesitancy (A/H) — conflicting affective states relevant to health behaviour change. The system fuses frozen vision, audio, and text encoders with a streaming model that scores cross-modal dissonance and uses a knowledge-guided LLM for late fusion reasoning. On a 525-video public test set, PRISM-AH achieves macro F1 of 0.6133 versus a zero-shot baseline of 0.2827. The work is a domain-specific application of multimodal reasoning to affective computing in a clinical/health context.
Related guides (1)
Related events (8)
ClinPRISM: Cost-effective multimodal LLM framework for clinical time series question answering
Researchers introduce ClinPRISM, a multimodal LLM reasoning framework designed for question answering over irregular clinical time series (ICTS). The system combines an irregularity-aware multi-scale encoder, a temporal evidence distiller, and a progressive alignment strategy to handle the sparsity and asynchrony typical of clinical observations. Built on a 4B-parameter LLM backbone, it achieves state-of-the-art results on a held-out clinical benchmark using only 16 time-series tokens with 0.15-second inference latency. The authors also release a training corpus of 30,000 clinical time series with multi-scale descriptions and 41,000 instruction-tuning instances across 11 tasks.
Survey: Human-View Video Understanding with MLLMs — Watch, Remember, Reason Framework
A new arXiv survey paper proposes a unified 'human-view' framework for analyzing multimodal LLM-based video understanding, organized around three functional abilities: watching (perception), remembering (memory), and reasoning. The authors introduce a formulation characterizing video understanding systems by perceptual representations, memory states, reasoning traces, and predictions, then survey methods, datasets, and benchmarks across these dimensions. The work covers challenges including spatio-temporal perception, long-video processing, streaming understanding, and faithful reasoning, with application domains spanning egocentric, sports, medical, and narrative video.
PRISM probe separates physical danger from text-level safety in LLM hidden states for embodied agents
Researchers introduce PRISM, a single-layer logistic probe over LLM hidden states that distinguishes physically grounded danger (e.g., unsafe robot actions) from ordinary text-level content danger, showing these are separable signals in representations across multiple model families. PRISM achieves 86.2–87.7% accuracy on SafeAgentBench with substantially lower false-positive rates than same-scale LLM judges, which over-block safe tasks at 24.7–39.0% FPR. The authors also release PhysicalSafetyBench-1K (PSB-1K), a 1,000-pair contrastive benchmark for evaluating physical-risk detection without relying on explicit harm keywords. The work is relevant to safety in embodied AI and agentic systems where linguistic safety filters are insufficient.
Multimodal Voice Activity Projection framework for turn-taking prediction in social robots
Researchers introduce MM-VAP, a Multimodal Voice Activity Projection framework that extends audio-only turn-taking prediction to synchronized audio-visual inputs for use in social robots acting as conversation mediators. The approach uses pretrained audio-visual backbones adapted via Low-Rank Adaptation, an inter-speaker attention stage, and a semantic consistency loss to regularize the output space. Experiments on the NoXi, NoXi+J, and Haru EDR corpora show improvements over baselines on several turn-taking events. The work targets human-robot interaction scenarios where anticipating conversational dynamics is required rather than simply reacting to pauses.
HarmVideoBench: Multi-layered benchmark for harmful video understanding in large multimodal models
Researchers introduce HarmVideoBench, a diagnostic benchmark of 1,379 videos paired with 4,137 multiple-choice questions designed to evaluate harmful video understanding across three hierarchical dimensions: Observable Evidence, Clip-Internal Meaning, and Beyond-Clip Reasoning. The benchmark addresses limitations in existing work by moving beyond binary classification and requiring explanatory rationales. The authors evaluate 19 leading models and introduce BCR, a method that dynamically retrieves context based on predicted reasoning boundaries, improving macro average performance from 61.7% to 84.4%.
OmniAgent: POMDP-based active perception agent for long video understanding with test-time scaling
Researchers introduce OmniAgent, a multimodal agent that reformulates long video understanding as a POMDP-based iterative Observation-Thought-Action cycle, selectively distilling audio-visual cues into persistent textual memory rather than processing all frames uniformly. The system uses Agentic Supervised Fine-Tuning and a novel reinforcement learning method (TAURA) with turn-level entropy for credit assignment. OmniAgent demonstrates positive test-time scaling and achieves state-of-the-art open-source results across ten benchmarks, with its 7B model outperforming Qwen2.5-VL-72B on LVBench (50.5% vs. 47.3%).
Prism: Plug-in Infrastructure for Multimodal Continual Instruction Tuning Research
Prism is an open-source codebase designed to address engineering bottlenecks in Multimodal Continual Instruction Tuning (MCIT) research. It introduces a plugin registration mechanism that separates algorithmic development from backbone MLLM implementation, allowing new continual learning strategies to be integrated without modifying the underlying model codebase. This design aims to eliminate structural fragmentation across method-specific implementations and enable fair, reproducible comparisons at scale.
PsyBridge: Hybrid AI framework for multi-dimensional mental health assessment
Researchers propose PsyBridge, a hybrid decision-support framework that integrates PHQ-9, GAD-7, cognitive evaluation, and personality profiling into a unified architecture for mental health risk classification. The system uses a weighted aggregation mechanism to produce interpretable outputs and was evaluated on a semi-synthetic dataset of 500 patient profiles. PsyBridge achieves 0.84 accuracy, outperforming standalone screening tools, with ablation studies confirming the value of multi-dimensional integration. The work targets digital healthcare and telehealth deployment contexts.