Entity · model
Gemma-3-4B-IT
modelactive
gemma-3-4b-it-dc8c82f8·5 events·first seen Aliases: Gemma-3-4B-IT, Gemma-3, Gemma3-4B, Gemma-3-4B
Co-occurring entities
X-SensitiveSH-DetectionAnalysing Self-Harm Representations in Language Models: a Cross-Architecture StudyInnocuous-Seeming Data, Latent Ideology: Ideological Generalisation in Finetuned LLMsGoogleOpenAIGPT-4.1GSM8KMET: Theory-Grounded and Culture-Aware Multilingual Moral ReasoningQwen3-4BMCLASHMMoralExceptQAMuSiQueOLMo-3Logit-Contribution ScoringNoLiMaQwen3BABI-LongSubspace ProjectionSparse Autoencoders (SAEs)
More like this (12)
Recent events (5)
Cross-architecture study of how LLMs internally represent self-harm content
Researchers analyze how four LLMs encode self-harm content using linear probes and contrastive direction extraction across two datasets (X-Sensitive and SH-Detection). A key finding is that self-harm information crystallizes in the final 3–7% of network layers (93–97% depth), and that probe accuracy does not correlate straightforwardly with linear separability. Gemma-3-4B shows a notably different internal representation of the contrastive self-harm direction compared to other models, with implications for detection, intervention, and LLM governance.
Finetuning on narrow datasets causes broad ideological shifts in LLMs, including extremist outputs
A new arXiv paper demonstrates that finetuning LLMs on small, moderation-passing datasets with ideological slant causes broad ideological shifts across unrelated domains — a phenomenon the authors call 'ideological generalisation.' Training GPT-4.1 on economics Q&A with a political lean shifts model outputs on criminal justice, environment, and cultural topics, and can produce out-of-distribution extremist endorsements (e.g., race-IQ connections, political violence) not present in training data. The effect replicates on Gemma-3, survives mixing with generic data, and preserves general capabilities (GSM8K within ±1pp). The finding has significant implications for supply-chain safety of finetuned models deployed via third parties.
MET: Theory-grounded multilingual moral reasoning with cultural adaptation and self-distillation
Researchers introduce three contributions to address gaps in multilingual moral reasoning for LLMs: MCLASH, a culturally situated multilingual moral decision-making benchmark; MET, a two-step prompting method grounded in psychological and philosophical moral theory; and MET-D, a self-distillation training variant requiring no external supervision. MET-D improves macro-F1 by an average of 3.71 points on MCLASH and 4.23 on MMoralExceptQA across Qwen3-4B, Qwen3-8B, and Gemma3-4B, with a peak gain of 12.94 points for Malay on Qwen3-8B. The work also finds that native-language reasoning increases by 62 points on average and that beneficial moral grounds differ systematically across cultures.
LOCOS: Logit-Contribution Scoring identifies non-literal retrieval heads in long-context LLMs
A new arXiv preprint introduces Logit-Contribution Scoring (LOCOS), a method for identifying attention heads responsible for non-literal retrieval in long-context LLMs — cases where models synthesize answers from meaning rather than copying tokens verbatim. Existing detectors fail at this task because they rely on a literal-copy criterion that misses the output-value (OV) circuit mechanism. Evaluated across Qwen3, Gemma-3, and OLMo-3.1, LOCOS outperforms prior attention-based detectors on the NoLiMa benchmark, with ablation of 50 heads on Qwen3-8B collapsing ROUGE-L from 0.401 to 0.000 while the best baseline retains 0.292. The identified heads are retrieval-specific, leaving parametric recall and arithmetic reasoning unaffected.
SAEs as Stethoscopes: Interpretability-Guided Layer Selection for Task Vector Model Editing
This paper evaluates a Sparse Autoencoder (SAE)-guided model editing pipeline for mathematical reasoning on Gemma-3-4B-IT, finding that projecting task vectors onto SAE feature subspaces discards ~97% of modification energy due to geometric misalignment between activation-space SAE directions and weight-space task vectors. The authors reframe SAEs as diagnostic tools ('stethoscopes') rather than intervention filters ('scalpels'), using SAE-derived specificity scores to identify which layers to inject unfiltered task vectors into. This approach improves Number Theory accuracy from 29.6% to 39.4% on Minerva Math (p=0.0007), with 5 of 7 math subjects significantly improved and none degraded. The method is fully deterministic and adds no inference cost.