SOURCE-LINKED INTELLIGENCE

The Bias of Nonlinear Two-Time-scale Stochastic Approximation under Constant Step-Sizes

arXiv · AI, language, vision and robotics · article · Sep 17, 2026 · UTC

Two-timescale stochastic approximation (TTSA) is a fundamental tool for analyzing coupled iterative algorithms in reinforcement learning, optimization, and stochastic control. However, finite-time guarantees for nonlinear two-timescale schemes remain difficult to obtain, especially under constant step-sizes. In this paper, we study nonlinear TTSA with step-sizes $α\ggβ$. Under standard stability, regularity, and Markovian noise assumptions, we upper bound the mean-squared error and the bias of both iterates around their limiting equilibria. Our bounds scale as $O(α+β^2/α^2)$, which we prove to

Read original source ↗ Open in workspace

recordType

paper

region

Global