SOURCE-LINKED INTELLIGENCE

RiLM: Parameter-Efficient Language Modeling via Geodesic Decoding

arXiv · AI, language, vision and robotics · article · Sep 9, 2026 · UTC

Language models under one million parameters matter for edge deployment, domain adaptation, and reproducible research, yet a two-layer LSTM or Transformer at embedding width d = 128 still spends roughly one third of its capacity on the output matrix W_out in R^(d x |V|). We propose Riemannian Language Models (RiLM), which remove that layer entirely: context unfolds as a trajectory on a Riemannian manifold, and next-token probabilities arise from squared geodesic distance between the current state and vocabulary embeddings. The same embedding map serves input and output -- decoding is geometry.

Read original source ↗ Open in workspace

recordType

paper

region

Global