Entity · paper

A Diffusion Approximation for Temporal-Difference Learning with Linear Features under Markovian Noise

paperactivea-diffusion-approximation-for-temporal-difference-learning-with-linear-features-under-markovian-noise-01e6b946·1 events·first seen Jun 17, 2026

Aliases: A Diffusion Approximation for Temporal-Difference Learning with Linear Features under Markovian Noise

Co-occurring entities

TD(0)

More like this (12)

Temporal Difference Learning Denoising Diffusion Probabilistic Models What Does a Discrete Diffusion Model Learn?Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models Adaptive Multi-Step Lookahead Decoding for Diffusion Language Models Induction in Both Directions: A Mechanistic Analysis of In-Context Learning in Masked Diffusion Language Models T^2MLR: Transformer with Temporal Middle-Layer Recurrence Self-Augmenting Retrieval for Diffusion Language Models Knowledge Editing in Masked Diffusion Language Models Selective Timestep Weighting and Advantage-Based Replay for Sample-Efficient Diffusion RLHF LESS: Mutual-Stability Sampling for Diffusion Language Models Audio-Native Speech Recognition with a Frozen Discrete-Diffusion Language Model

Recent events (1)

4arXiv · cs.LG·Jun 17, 2026·source ↗

SDE approximation for TD learning with linear features under Markovian noise

A new arXiv preprint replaces the classical ODE description of linear TD(0) learning with a stochastic differential equation (SDE) approximation that accounts for Markovian sampling noise. The model separates contraction dynamics governed by the projected Bellman operator from the influence of Markovian long-run covariance, providing a theoretical explanation for the constant-stepsize error floor. The work is a theoretical contribution to the foundations of reinforcement learning policy evaluation.

Alignment and RLHF TD(0)A Diffusion Approximation for Temporal-Difference Learning with Linear Features under Markovian Noise