4arXiv cs.LG (Machine Learning)·
BioModule: plug-in transformer bridges 3D pose estimation and biomechanical attribute prediction
Researchers introduce BioModule, a lightweight temporal transformer that attaches downstream of any 3D human pose estimator to predict biomechanical quantities (joint loads, muscle activations, etc.) from standard 17-joint skeletons without modifying the upstream model. The work constructs a large-scale aligned dataset pairing Human3.6M video/keypoints with biomechanical labels from Human3.6Mplus, and benchmarks the module across seven state-of-the-art pose estimators. The paper provides the first systematic analysis of how upstream pose estimation quality propagates to downstream biomechanical prediction fidelity, targeting rehabilitation, sports science, and clinical movement analysis applications.
Related guides (1)
Related events (8)
Humanoid-GPT: GPT-style Transformer trained on 2B-frame motion corpus for zero-shot humanoid control
Researchers introduce Humanoid-GPT, a causal Transformer pre-trained on a 2-billion-frame retargeted motion corpus that unifies major mocap datasets with large-scale in-house recordings for whole-body humanoid control. The model achieves zero-shot generalization to unseen motions and control tasks, overcoming the agility-generalization trade-off seen in prior MLP-based trackers. Scaling analyses demonstrate a new performance frontier for dynamic motion tracking without task-specific fine-tuning.
Domain-specific transformer embeddings for detecting dosing errors in clinical trial protocols
Researchers from CaresAI evaluated biomedical transformer models (ClinicalBERT, PubMedBERT, BioBERT, MedCPT) for detecting dosing errors in clinical trial protocols, combining text embeddings with structured metadata and classical ML classifiers. BioBERT achieved the best single-encoder performance at ROC-AUC 0.794, while gradient boosting and SVM ensembles reached 0.821–0.853. The study finds that domain alignment of the encoder matters more than stacking multiple embeddings, and demonstrates a practical NLP pipeline for clinical trial safety monitoring.
PEVA: Whole-Body Conditioned Egocentric Video Prediction for Embodied World Models
Researchers from BAIR introduce PEVA (Predicting Ego-centric Video from human Actions), a model that generates first-person video frames conditioned on 48-dimensional whole-body kinematic pose trajectories. The model uses an autoregressive conditional diffusion transformer trained on the Nymeria dataset, which pairs real-world egocentric video with body pose capture. PEVA can generate atomic action videos, simulate counterfactuals, and support long video generation, representing a step toward world models grounded in physically embodied human agents.
BODHI: Contrastive embedding training for causal discovery in Large Behavioural Models
Researchers identify a critical failure mode in biomedical language model embeddings: off-the-shelf encoders (BioBERT, PubMedBERT, BioM-ELECTRA) assign high cosine similarity (0.76–0.92) to causally unrelated cross-domain pairs, achieving 0% accuracy on cross-domain discrimination. The paper introduces BODHI, a contrastive training approach using hard negatives mined from a biomedical knowledge graph, which improves within-vs-across-domain separation from 1.05x to 2.30x and raises discrimination gap by +0.392. The work targets Large Behavioural Models (LBMs)—foundation models that reason over personal life graphs—where false embedding proximity directly produces false causal edges. Additional contributions include an OpenVINO inference optimization achieving 133x latency reduction (1367ms to 10ms) on Intel AMX hardware, plus a counterintuitive finding that FP16 outperforms INT8 on this silicon.
Illumination-robust rPPG heart-rate estimation via spatial-temporal transformer for robot-mounted cameras
A new arXiv paper presents an end-to-end spatial-temporal transformer framework for remote photoplethysmography (rPPG) heart-rate estimation that is robust to illumination variation, targeting robot-mounted RGB cameras. The system integrates 3D face alignment, illumination augmentation, a Residual Temporal Standardization Module, and a hybrid waveform-plus-spectral loss. On a new dataset spanning three illumination levels, the method achieves 0.79 bpm MAE and 0.982 HR correlation, reducing error by 93.6% relative to the PhysFormer baseline. The work is relevant to physiological sensing in service and assistive robotics.
AnyMo: Geometry-Aware Setup-Agnostic Framework for Wearable IMU Human Motion Understanding
AnyMo is a geometry-aware framework that addresses the setup-dependence problem in wearable IMU-based human motion modeling by using physics-grounded simulation over dense body-surface placements to generate synthetic training signals. It pre-trains a graph encoder from synthetic placement views and masked partial observations, then tokenizes multi-position IMU data into full-body motion tokens aligned with an LLM for motion-language understanding. Evaluated across zero-shot activity recognition (14 unseen datasets), cross-modal retrieval, and motion captioning, AnyMo improves average Accuracy/F1 by ~11.7%/11.6%, zero-shot retrieval MRR by 15.9–28.6%, and captioning BERT-F1 by 18.8%. The work positions itself as a generalist model for wearable motion understanding transferable across devices and sensing configurations.
DynaFLIP: Dynamics-Aware Multimodal Pre-Training for Robot Manipulation Perception
DynaFLIP is a pre-training framework that injects motion understanding into visual encoders for robot manipulation by constructing image-language-3D flow triplets from human and robot videos. The method encourages tri-modal alignment via simplex-volume minimization in a shared hyperspherical space, combined with cosine regularization and contrastive objectives. The resulting dynamics-aware visual backbone consistently outperforms baselines across diverse downstream policies including VLAs, with gains up to +22.5% in out-of-distribution scenarios. The work argues that robot generalization requires encoding how the world changes under action, not just static scene content.
Foundation Model for Wearable Health Data Pretrained on 1 Trillion Minutes from 5 Million Participants
Researchers propose a large-scale foundation model for wearable health data, pretrained on over one trillion minutes of unlabeled sensor signals from five million participants. The model demonstrates systematic performance improvements across 35 health prediction tasks spanning cardiovascular, metabolic, sleep, and mental health domains, with joint scaling of model capacity and data volume. A 'classroom' of LLM agents autonomously searches downstream predictive head configurations, and the resulting embeddings are integrated into a Personal Health Agent validated by 1,860 clinician ratings. The work establishes label-efficient few-shot learning and generative capabilities for daily health metric estimation.