paper

A Diffusion Approximation for Temporal-Difference Learning with Linear Features under Markovian Noise

paperactiveprovisionala-diffusion-approximation-for-temporal-difference-learning-with-linear-features-under-markovian-noise-01e6b946·1 events·first seen 7h ago

Aliases: A Diffusion Approximation for Temporal-Difference Learning with Linear Features under Markovian Noise

Co-occurring entities

TD(0)

More like this (12)

Temporal Difference Learning Denoising Diffusion Probabilistic Models Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models Self-Augmenting Retrieval for Diffusion Language Models Knowledge Editing in Masked Diffusion Language Models LESS: Mutual-Stability Sampling for Diffusion Language Models Masked Diffusion Models likelihood approximation FlashbackCL: Mitigating Temporal Forgetting in Federated Learning multivariate time series representation learning diffusion posterior sampling discrete diffusion models

Recent events (1)

4arXiv · cs.LG·7h ago·source ↗

SDE approximation for TD learning with linear features under Markovian noise

A new arXiv preprint replaces the classical ODE description of linear TD(0) learning with a stochastic differential equation (SDE) approximation that accounts for Markovian sampling noise. The model separates contraction dynamics governed by the projected Bellman operator from the influence of Markovian long-run covariance, providing a theoretical explanation for the constant-stepsize error floor. The work is a theoretical contribution to the foundations of reinforcement learning policy evaluation.

Alignment and RLHF TD(0)A Diffusion Approximation for Temporal-Difference Learning with Linear Features under Markovian Noise