Entity · model
Llama 3.2
modelactive
llama-3-2-535757d5·11 events·first seen Aliases: Llama 3.2, Llama 3.2 1B, Llama 3.2 3B, Llama 3.2-3B, Llama-3.2-3B, Llama-3B, Llama-3.2-1B
Co-occurring entities
MetaLlama-3.1-8BLoRAQwen 2.5-7BGemma 2Qwen2.5-1.5BMinistral 3BHugging FaceMinimizing Targeted Activations: Input-Only Suppression of Evaluation-Awareness Latents in Large Language ModelsLlama ScopeFluent DreamingGCGThe PileSuper-Tuning: From Activation-Aware Pruning to Sparse Fine-TuningMeta-Llama-3-8BPaFiWandaDoomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe CascadeTextCraftTraitFactory
More like this (12)
Recent events (11)
Input-only prompt optimization can suppress evaluation-awareness latents in LLMs, but activation readability ≠ behavioral control
Researchers study the input-side dual of activation steering: optimizing fluent prompts to drive a chosen internal latent toward zero without inference-time model access. The target is an 'evaluation-awareness' latent whose suppression would threaten safety evaluation validity if models behave differently when detecting they are being tested. Experiments on Llama-3.2-3B and Llama-3.1-8B across five latent constructions (CAA direction, subspace norm, SAE feature, MLP neuron, behavioral logit) find the latent is robustly suppressible, but a key cautionary result emerges: a placebo random direction is suppressed just as hard and shifts behavior just as far, and suppressing the eval-direction in context fails to reduce behavioral eval judgment. The paper concludes that activation-readability does not imply behavioral controllability, with implications for how safety evaluations should be designed and interpreted.
Super-Tuning: Pruning saliency signals repurposed for sparse parameter-efficient fine-tuning
Researchers propose Super and Supra, two sparse PEFT methods that reuse activation-weighted magnitude scores (Wanda-style) originally developed for pruning to select which parameters to update during fine-tuning. Supra combines this sparse update with LoRA under a fixed parameter budget via a budget-splitting rule. Experiments on Llama-3.2-1B and Llama-3-8B on a Math17K arithmetic task show the best Super/Supra variants outperform other tested adapter configurations. The work suggests pruning-inspired orderings are a useful, low-cost signal for identifying effective sparse fine-tuning supports.
Early abort cascade for LLM agents predicts episode failure from hidden activations, saving ~47% inference compute
A new arXiv paper introduces a method to detect doomed LLM agent episodes early by probing internal hidden-state activations, rather than waiting for observable failure. The approach uses a cascade of calibrated per-round gates with recall budgets, guaranteeing that eventually-successful episodes survive at a user-specified rate. On TextCraft with Qwen-2.5-7B and Llama-3.2-3B, the cascade saves 37–47% of inference compute at a 90% recall target, outperforming behavior-only baselines by roughly 2x. The work provides both a practical deployment mechanism and theoretical guidance on sample complexity for certifying high recall targets.
ORBIT: Training-free multi-attribute behavioral steering via orthogonal subspace rotation
Researchers introduce ORBIT (Orthogonal Rotation-Based Intervention Technique), a training-free activation steering method that simultaneously controls multiple behavioral attributes in language models. The approach constructs a joint subspace from per-attribute steering planes via SVD and applies a single norm-preserving rotation, avoiding the norm imbalance and directional cancellation problems of naive vector summation. The authors also release TraitFactory, a new multi-attribute behavioral benchmark, and evaluate across Llama-3.2-3B, Qwen-2.5-7B, and Llama-3.1-8B. ORBIT outperforms existing training-free baselines on multi-attribute steering while better preserving output coherence.
Activation-space directions for detecting and mitigating emergent misalignment across LLM families
Researchers fine-tuned four small instruction-tuned model families (Qwen2.5-1.5B, Gemma-2-2B, Llama-3.2-1B, Ministral-3B) on insecure code to induce emergent misalignment, then investigated whether a shared activation-space direction could detect and correct it. A difference-in-means direction achieves 99.6% separation of aligned vs. misaligned activations within each model, and causal steering by subtracting this direction reduces misaligned behavior by 21–51 points. Cross-architecture transfer via ridge regression yields large behavioral suppression but fails specificity controls, revealing a two-tier structure: within-model directions are causally specific and actionable, while cross-model directions are real but non-specific. The findings bound the utility of linear cross-architecture correction and recommend within-model probing for safety auditing.
Program synthesis used to reverse-engineer transformer attention heads with executable Python surrogates
Researchers propose a pipeline that approximates transformer attention heads with executable Python programs generated by a language model, then re-ranked by held-out predictive accuracy. Applied to GPT-2, TinyLlama-1.1B, and Llama-3B, fewer than 1,000 programs reproduce attention patterns with >75% average IoU similarity on TinyStories. Replacing 25% of attention heads with programmatic surrogates incurs only a 16% average perplexity increase while preserving downstream QA performance, demonstrating a path toward symbolic transparency in neural models.
Multi-source cybersecurity log dataset with ATT&CK labels and SLM fine-tuning evaluation
Researchers introduce a new multi-source cybersecurity log dataset of 870 sessions (~2.3M events) capturing system, network, and browser activity on Windows endpoints, with per-entry MITRE ATT&CK technique labels across 12 tactics and 53 techniques. The dataset addresses gaps in existing public datasets (CICIDS, UNSW-NB15, ATLAS) that lack combined multi-source coverage with fine-grained ATT&CK labeling. Three small language models (Qwen2.5-1.5B, Llama-3.2-3B, Phi-4-Mini) were fine-tuned with LoRA on the dataset, achieving chunk classification accuracy of 90–97% versus ~8% for base variants, though ATT&CK technique identification remained harder at 42% exact-match accuracy.
Meta releases Llama 3.2-3B open-weights text generation model
Meta released Llama 3.2-3B, a 3-billion parameter open-weights language model, on Hugging Face under the meta-llama organization. The model supports multiple languages including English, German, French, and Italian, and uses the standard transformers/safetensors format. With over 900K downloads and 800+ likes, it has seen substantial community adoption.
Mistral AI Releases Ministral 3B and 8B Edge Models
Mistral AI has introduced two new small language models, Ministral 3B and Ministral 8B, targeting on-device and edge computing use cases. Both models support up to 128k context length and claim state-of-the-art performance in the sub-10B parameter category, outperforming comparable models from Google and Meta on internal benchmarks. Ministral 8B features an interleaved sliding-window attention mechanism for memory-efficient inference and is priced at $0.1/M tokens via API, while Ministral 3B is priced at $0.04/M tokens. Weights for Ministral 8B Instruct are available for research use, with commercial licensing available on request.
Llama 3.2 Multimodal and Edge Models Launch on Hugging Face
Meta released Llama 3.2, introducing vision-capable multimodal models alongside lightweight models optimized for on-device inference. Hugging Face published a blog post covering integration support, model availability, and deployment options across the ecosystem. The release marks Meta's first open-weights multimodal Llama models, adding image understanding to the Llama family. Smaller 1B and 3B parameter variants target edge and mobile deployment scenarios.
Llama 3.2 in Keras
Hugging Face published a blog post detailing the integration of Meta's Llama 3.2 models into the Keras framework. The post covers how developers can use Keras to load, fine-tune, and run inference with Llama 3.2, expanding the ecosystem of tools available for working with the model. This represents a tooling/framework integration update rather than a new capability announcement.