4arXiv cs.CL (Computation and Language)·
MyMentorLLM: Multimodal LLM simulation environment for CBT psychotherapy training
Researchers present MyMentorLLM, a multimodal (voice and text) simulation environment for deliberate practice in Cognitive Behavioural Therapy training, generating 2,100 complete CBT sessions linking simulated DSM-5-TR-grounded patients, trainee therapists, and expert supervisors. Simulated patients expressed disorder-congruent emotional profiles and trainees mirrored these as in real counselling, while supervisor quality varied across LLMs with most overestimating trainee competence. Native speech-to-speech models scored closest to human supervisors, and symptom identification accuracy scaled with model size. The work demonstrates feasibility of LLM-based deliberate practice simulation for clinical training while flagging calibration and harmful feedback as open risks.
Related guides (1)
Related events (8)
Systematic Evaluation of LLM Safety Failures on Eating Disorder Queries with Clinician Feedback
This paper investigates how LLMs respond to queries from users with eating disorders, finding that specific linguistic cues in prompts increase the likelihood of unsafe model responses. Working with clinical ED experts, the authors systematically vary risk levels in user prompts to measure the extent to which LLMs uncritically adapt to potentially dangerous inputs. The study highlights a gap between perceived model safety and actual harm facilitation in sensitive health contexts.
Self-evolving LLM annotation framework for DSM-5-TR-aligned depression symptom labeling
A new arXiv preprint proposes a human-in-the-loop annotation framework for Major Depressive Disorder (MDD) that combines LLM-assisted labeling with expert verification across three stages: evidence selection, DSM-5-TR criterion analysis, and case-level synthesis. A dual-memory architecture (Example Memory and Reflection Memory) internalizes expert feedback to iteratively improve annotations without model retraining. The framework targets dataset construction for explainable AI in mental health rather than clinical diagnosis, and exports reasoning traces and edit histories for auditability. A pilot study reports improved annotation consistency and reduced manual revision effort.
Dep-LLM: Training-free depression diagnosis framework using structured multi-factor LLM reasoning
Dep-LLM is a training-free framework for automatic depression detection from clinical interviews that uses frozen foundation LLMs without fine-tuning. The system decomposes long clinical dialogues into five thematic factors via Chain-of-Thought analysis, applies token-level entropy-based confidence modulation, and integrates multi-factor signals for final diagnosis. Evaluated on DAIC-WOZ and E-DAIC datasets, it outperforms zero-shot baselines across 21 foundation LLMs and surpasses supervised domain-specific and commercial LLMs on multiple metrics.
LLUMI: Fine-Tuning Open-Source LLMs for Mental Health Writing Assistance Using Reddit Community Feedback
LLUMI is a two-component system (a generation model and an improvement model) designed to provide mental health writing assistance using smaller open-source LLMs hosted in privacy-preserving, on-premise environments. The system leverages Reddit community endorsement signals (upvotes/downvotes) to construct preference pairs for SFT and DPO training, then further aligns outputs via human evaluation across readability, empathy, connection, actionability, and safety dimensions. Results show LLUMI achieves performance comparable to proprietary GPT-based models on linguistic and human evaluations, suggesting community-derived preference signals can substitute for expensive expert labeling in sensitive domains.
Study finds LLM health chatbot triage outcomes vary significantly with patient communication style
Researchers analyzed 2,053 real patient-chatbot conversations and found wide variation in communication patterns and emotional expression, revealing that idealized patient simulations used in chatbot development are inadequate. They built a patient simulator modeling clinical content, emotional state, conversational strategy, and communication style, achieving near-indistinguishability from real conversations (human graders 55% accurate). Evaluating four LLMs across 1,164 clinician-graded cases with five patient personae, the study found that communication style significantly alters triage urgency assessments. The authors warn that systems optimized for cooperative, articulate users risk underperforming and amplifying health disparities in real-world deployment.
Counterfactual context revision framework for auditing LLM-based stance simulation in online discussions
Researchers introduce a counterfactual context revision framework to audit how LLMs simulate individual users' stances in online discussions. By applying controlled text-only and multimodal (meme-based) revisions to conversational contexts, they measure how readily simulated stances shift in response to semantically independent changes. Results show effective and robust stance transitions across both revision types and polarization-preference mechanisms, raising concerns about whether LLM simulations reflect genuine user-specific beliefs or are highly context-sensitive artifacts. The work contributes an evaluation framework and highlights risks of using LLMs to model online opinion dynamics.
Reason-Mediated Behavioral Models for Auditing LLM Social Simulators
A new arXiv paper proposes an evaluation framework for LLMs used as social simulators, arguing that matching final outcomes is insufficient — simulators must also replicate the underlying rationale-derived reasoning paths. Using a 94-person sunscreen concept test, the authors map open-ended human rationales into signed 'reason states' and show that LLM-simulated reasons, while superficially plausible, frequently mirror the stimulus material rather than recovering the respondent's actual acceptance or rejection logic. The framework provides an interpretable audit for whether LLM social simulators are reasoning correctly, not just producing correct-looking outputs.
Adaptive LLM tutoring system with subject-aware prompt routing improves high-school student engagement
Researchers develop and evaluate an LLM-based tutoring system that uses a learned prompt routing model to dynamically select pedagogical strategies based on 14 features extracted from conversation transcripts. The system was trained in simulation and deployed in an A/B test with 359 high-school students (656 conversations), showing sim-to-real transfer and reducing required interactions by ~3 turns. A stochastic routing strategy achieved a notably higher exercise conversion rate (28.1%) compared to a greedy router (19.1%) and static baseline (19.6%).