NeSyCat Torch: Differentiable tensor framework unifying neurosymbolic semantics via monadic abstraction
NeSyCat Torch extends the NeSyCat/ULLER neurosymbolic framework with neural network support for predicates and functions, implemented via probabilistic programming and tensor backends (HaskTorch, JAX, PyTorch). The key technical contribution is a lazy log-tensor monad over the log-semiring enabling numerically stable, differentiable training, alongside a batch monad for efficient batched inference. On MNIST addition benchmarks, the implementations outperform LTN and DeepProbLog in speed and accuracy while remaining within a uniform categorical framework that generalizes across first-order neurosymbolic approaches. The work positions itself as a unifying foundation for classical, fuzzy, probabilistic, and neural truth semantics.
Related guides (1)
Related events (8)
NF-CoT: Latent reasoning with normalizing flows preserves autoregressive LLM advantages
Researchers propose NF-CoT, a latent reasoning framework that replaces discrete chain-of-thought token streams with continuous intermediate states modeled by normalizing flows embedded inside an LLM backbone. The approach uses a TARFlow-style normalizing flow head alongside the standard language model head, enabling exact likelihoods, KV-cache-compatible left-to-right decoding, and policy-gradient optimization in latent space. On code-generation benchmarks, NF-CoT improves pass rates over both explicit CoT and prior latent-reasoning baselines while reducing intermediate reasoning cost. The work addresses a key limitation of existing latent reasoning methods, which typically sacrifice probabilistic tractability or autoregressive compatibility.
torchtune: PyTorch Native Post-Training Library for LLMs
Meta's PyTorch team introduces torchtune, a PyTorch-native library for post-training LLMs that emphasizes modularity, hackability, and direct access to underlying PyTorch components. The library supports fine-tuning, experimentation, and deployment-oriented workflows across distributed training settings. Benchmarked against popular frameworks Axolotl and Unsloth, torchtune demonstrates competitive performance and memory efficiency while maintaining flexibility for research iteration. The paper presents design principles, model builders, training recipes, and distributed training stack details.
Program synthesis used to reverse-engineer transformer attention heads with executable Python surrogates
Researchers propose a pipeline that approximates transformer attention heads with executable Python programs generated by a language model, then re-ranked by held-out predictive accuracy. Applied to GPT-2, TinyLlama-1.1B, and Llama-3B, fewer than 1,000 programs reproduce attention patterns with >75% average IoU similarity on TinyStories. Replacing 25% of attention heads with programmatic surrogates incurs only a 16% average perplexity increase while preserving downstream QA performance, demonstrating a path toward symbolic transparency in neural models.
Neurosymbolic Learning for Inference-Time Argumentation in Claim Verification
This paper introduces Inference-Time Argumentation (ITA), a trainable neurosymbolic framework for ternary claim verification (true/false/uncertain) that integrates formal argumentation semantics with LLM training. The framework uses argumentation semantics both to guide LLM training for argument generation and scoring, and to compute final predictions deterministically from explicit argumentative structures. Unlike conventional reasoning models that rely on potentially unfaithful post-hoc explanations, ITA produces verdicts that are faithful by construction to the underlying arguments. Experiments on two ternary claim verification datasets show ITA outperforms argumentative baselines and competes with non-argumentative direct-prediction approaches.
SymbolicLight V1: Spike-Gated Dual-Path Language Model with High Activation Sparsity
SymbolicLight V1 is a 194M-parameter spiking language model that combines binary Leaky Integrate-and-Fire spike dynamics with a continuous residual stream, replacing dense self-attention with a dual-path module using exponential-decay aggregation and spike-gated local attention. Trained from scratch on a 3B-token Chinese-English corpus, it achieves validation perplexity of 8.88–8.93 at over 89% per-element activation sparsity, trailing GPT-2 201M by 7.7% in PPL. Ablations indicate that temporal integration via LIF dynamics contributes more to performance than sparsity alone, and a 0.8B-parameter scale-up on 48.8B tokens demonstrates optimization stability. Current dense-hardware inference is slower than GPT-2; neuromorphic deployment is framed as a future opportunity.
Conservation laws from data symmetry in neural network gradient-flow training
A new arXiv preprint investigates whether intrinsic symmetries in training data produce conserved quantities during gradient-flow training of neural networks. The authors prove that for analytic, non-polynomial loss functions, data symmetries generically do not induce additional integrals of motion, but for MSE loss, data augmentation can yield extra conserved quantities. They introduce a framework of 'tensorizable networks'—architectures including linear, polynomial, and Lightning Attention networks—where parameter and input dependence can be separated via an intermediate representation.
Neuronal Stochastic Attention Circuit (NSAC) for Probabilistic Representation Learning
Researchers introduce NSAC, a biologically-inspired continuous-time attention architecture that models attention logits as solutions to an Ornstein-Uhlenbeck stochastic differential equation, drawing on C. elegans Neuronal Circuit Policy wiring to induce Gaussian distributions over attention weights. The architecture enables joint quantification of aleatoric and epistemic uncertainty via a two-term objective combining Gaussian negative log-likelihood with an epistemic-separation regularizer. Empirical evaluation spans irregular time-series function approximation, multivariate regression, long-range forecasting, Industry 4.0 tasks, and autonomous vehicle lane-keeping, showing competitive accuracy with well-calibrated uncertainty estimates.
ATLAS: Unified Agentic and Latent Visual Reasoning via Functional Tokens
ATLAS proposes a framework where a single discrete 'functional token' serves dual roles as both an agentic operation trigger and a latent visual reasoning unit in multimodal models. This design avoids the computational cost of generating intermediate images while sidestepping the context-switching latency of external tool calls and the generalization limitations of pure latent methods. The framework is compatible with standard SFT and RL training pipelines without architectural changes, and introduces Latent-Anchored GRPO (LA-GRPO) to stabilize reinforcement learning when functional tokens are sparse. Experiments show strong performance on visual reasoning benchmarks with maintained interpretability.