SOURCE-LINKED INTELLIGENCE

Puro-2B: Poor Lab's Qwen2-1.5B Trained on RTX 5090 within $5090

arXiv · AI, language, vision and robotics · article · Aug 27, 2026 · UTC

Language model pretraining has become almost synonymous with prohibitive cost, placing it out of reach for much of the academic and open-source communities. Although strong open-source efforts already exist, including open-weight models and open-source training recipes, a cost-efficient, hardware-accessible, and open-source pretraining recipe has long been missing. Even at a small scale, training Llama-3.2-3B costs over \$1.5M, and reproducing SmolLM3-3B needs over \$700K. In this report, we present an open pretraining recipe designed to lower this barrier. Using this recipe, we train a collec

Read original source ↗ Open in workspace

recordType

paper

region

Global