SOURCE-LINKED INTELLIGENCE

Size Matters: Foundation Model for Czech HTML documents

arXiv · AI, language, vision and robotics · article · Sep 16, 2026 · UTC

Creating universal, high-quality representations of web documents in high-traffic industrial environments requires models that are both performant and economic. Existing approaches, however, often depend on large models, overlook the structural information inherent in HTML, or are constrained by short context windows, limiting their ability to process real-world web pages. We present HTML-LM, a compact foundation model with 154 million parameters that addresses these limitations through HTML-aware training and a ModernBERT-based architecture. It was trained on 100 million web documents using m

Read original source ↗ Open in workspace

recordType

paper

region

Global