SOURCE-LINKED INTELLIGENCE

Beyond BLEU: A Case for Redefining Sign Language Translation Benchmarks

arXiv · AI, language, vision and robotics · article · Sep 3, 2026 · UTC

BLEU-4 is the standard metric for evaluating sign language translation (SLT), but spoken-language metrics may not adequately reflect sign language proficiency. The multimodal, low-resource context of SLT allows models to exploit spurious correlations and spoken-language priors, rather than learning stronger sign representations. In this paper, we evaluate the relationship between spatio-temporal understanding and BLEU-4 across six SLT models on Phoenix-2014T and CSL-Daily, showing that gains in BLEU-4 are not on their own evidence of better sign language understanding. This work introduces an

Read original source ↗ Open in workspace

recordType

paper

region

Global