SOURCE-LINKED INTELLIGENCE

TabuLM: Morphology-Aware Tabular Pre-training for Low-Resource Languages

arXiv · AI, language, vision and robotics · article · Aug 27, 2026 · UTC

We present TabuLM, the first language model pre-trained on Kinyarwanda tabular data. Kinyarwanda is a morphologically rich Bantu language spoken by over 12 million people in Rwanda, yet lacks any dedicated tabular representation learning resource. TabuLM extends KinyaBERT-large, a two-tier morphological transformer, with additive row, column, and cell-type embeddings and a learned table-structure attention bias that sharpens same-row and same-column attention. Pre-training uses two new objectives: Masked Cell Recovery (MCR), which masks entire cells and forces reconstruction from row and colum

Read original source ↗ Open in workspace

recordType

paper

region

Global