5arXiv cs.AI (Artificial Intelligence)·
CHARM: Multimodal graph foundation model with hierarchical context for zero-shot transfer
Researchers introduce CHARM, a graph foundation model designed for zero-shot transfer across multimodal graphs containing text, images, and other node-associated data. The model replaces isolated node representations with hierarchical graph contexts that capture cross-modal relations and map domain-specific patterns to shared high-level concepts, reducing the need for target-domain fine-tuning. A modality-aware encoder converts these representations into graph tokens fed to a large language model backbone. Experiments show consistent improvements on zero-shot multimodal graph tasks, addressing a gap where existing GNN-based GFMs require downstream adaptation and LLM-based methods are largely unimodal.
Related guides (2)
Related events (8)
CHARM: Multimodal JEPA for Semantic Time-Series Embeddings via Channel-Aware Representation Learning
CHARM (Channel-Aware Representation Model) is a new Transformer-based architecture for general-purpose representation learning over heterogeneous multivariate time series. It integrates channel-level textual descriptions into a permutation-equivariant encoder trained with a Joint Embedding Predictive Architecture (JEPA) and a novel temporally stable embedding loss. The model achieves strong performance across anomaly detection, classification, and forecasting tasks using only a linear probe, with text descriptions primarily serving as channel identifiers enabling cross-dataset generalization.
DualG-MRAG: Dual-tier graph architecture decouples macro-reasoning and micro-matching for multimodal RAG
A new arXiv preprint introduces DualG-MRAG, a framework for Multimodal Retrieval-Augmented Generation that separates global structural reasoning (Macro Graph) from fine-grained local evidence matching (Micro Graph) to address retrieval noise in multi-hop reasoning tasks. The system uses a GNN-based retriever with query-driven message passing and a dynamic programming decoding mechanism that extracts explicit reasoning paths from the GNN's forward pass rather than feeding isolated document chunks to the generative model. Experiments show improvements over baselines on evidence recall and complex QA accuracy. The work targets a known failure mode of graph-enhanced MM-RAG systems where fine-grained visual features cause graph explosion and noise.
GLM-RAG: Graph Language Model retrievers outperform GNNs and vector search on out-of-domain multi-hop reasoning
A new arXiv preprint introduces GLM-RAG, a retrieval-augmented generation framework using graph language model (GLM)-based retrievers over knowledge graphs. The authors compare GLM-based, GNN-based, and vector-search retrievers across single- and multi-hop QA settings, finding that finetuned GLM retrievers generalize better to unseen domains and achieve state-of-the-art on two multi-hop benchmarks. GNN-based retrievers offer better graph coverage with efficient training, while vector search excels at single-hop tasks. The work provides a systematic comparison of retriever architectures for knowledge-graph RAG with practical implications for domain transfer.
HCIG: Hierarchical Cross-Modal Incongruity Graph Network for Multimodal Sarcasm and Cyberbullying Detection
Researchers propose HCIG, a graph attention network framework that models cross-modal incongruity between text and images at token, phrase, and global levels for detecting sarcasm and cyberbullying in social media. A complementary architecture, GCCN, uses contradiction-aware pooling for efficient multimodal reasoning. HCIG achieves 85.74% accuracy on the MMSD sarcasm benchmark and 69.62% accuracy on the MultiBully cyberbullying dataset, outperforming conventional fusion-based approaches. The work addresses a narrow but practically relevant NLP task in content moderation.
CHERRY: Compressed Hierarchical Experts with Recurrent Representational Yield — three compute-efficient LM training techniques
A preprint from arXiv introduces CHERRY, a suite of three complementary techniques for compute-efficient language model training: Selective Ground Truth Token Training (SGT) that concentrates supervision on ~15% of semantically loaded tokens while recovering ~67% of full-sequence loss reduction; depth compression that shrinks a 48-layer 1B-parameter model to 6 layers (227M) via layer averaging and recurrent unrolling, matching a 566M dense model's loss; and a Mixture of Efficient Experts (MoEE) assembly that outperforms individual compressed models at comparable active parameters. The techniques are validated on CHERRY-1.8B, a Korean-language foundation model trained entirely from scratch using these methods. Authors are transparent about scope limitations: one model family, Korean data, and loss-based metrics only.
FM-CGM: Foundation Model Framework for Zero-Shot Visual Causal Generative Modeling
FM-CGM is a modular framework that decomposes visual causal reasoning into three components—concept extractor, concept manipulator, and counterfactual generator—using pretrained foundation models without task-specific causal training. The approach combines a large reasoning model for causal inference with a text-to-image diffusion model for generation, enabling zero-shot causal discovery and counterfactual image synthesis. A novel cross-attention mechanism called Causal Semantic Guidance (CSG) ensures that semantic interventions propagate correctly through causal descendants while preserving unaffected image regions. Empirical results show the framework can identify plausible causal structures and generate faithful counterfactual images.
ContextRL: Context-aware reinforcement learning improves grounding in agentic and multimodal LLMs
Researchers introduce ContextRL, a reinforcement learning method that trains LLMs to select the context that supports a given query-answer pair from two highly similar candidates, rather than supervising only final answers. The approach constructs contrastive context pairs in two domains: coding agent trajectories (1k pairs) and multimodal image pairs (7k pairs). ContextRL achieves +2.2% average gains over standard GRPO on 5 long-horizon benchmarks and +1.8% across 12 visual QA benchmarks, with ablations showing the gains stem from the context-selection objective rather than the contrastive data alone.
FedLAB: Traceable semantic codebooks for federated multimodal graph foundation learning
FedLAB is a new federated learning framework for multimodal graph foundation models that organizes knowledge into typed hierarchical codebooks covering modality evidence, node semantics, and topology context. The system enables semantic traceability under strict data isolation, addressing a gap where existing methods exchange knowledge through parameters or embeddings without exposing how evidence jointly supports predictions. Evaluated on 10 benchmarks and 6 downstream tasks, FedLAB improves over state-of-the-art baselines by up to 7.53% while keeping raw data local.