SOURCE-LINKED INTELLIGENCE

AdamX: Cosine similarity meets gradient descent

arXiv · AI, language, vision and robotics · article · Sep 10, 2026 · UTC

We introduce AdamX, a first-order optimizer that incorporates cosine similarity as an adaptive mechanism for controlling update magnitudes. The proposed method is scalable, model-agnostic, and straightforward to integrate into existing training pipelines. We further introduce a variance rectification scheme that promotes smoother optimization during the early stages of training. Overall, we provide empirical evidence that AdamX achieves competitive convergence rates across a range of benchmark datasets and architectures. Performance is evaluated in terms of the number of epochs required to rea

Read original source ↗ Open in workspace

recordType

paper

region

Global