SOURCE-LINKED INTELLIGENCE

Synthetic Semantic Supervision for Contrastive Code Representation Learning in Small Transformers: An Empirical Study

arXiv · AI, language, vision and robotics · article · Sep 3, 2026 · UTC

General-purpose code embeddings power tools for code search, classification, and retrieval. Compact transformer encoders for code typically rely on either human-written docstrings (labor-intensive and inconsistent) or mined structural signals such as execution traces (setting-specific and costly to collect). We empirically study an alternative: contrastive pretraining of small encoders with synthetically generated natural-language descriptions emphasizing code functionality and intent, paired with code in a dual-encoder framework at training and discarded at inference. We benchmark this approa

Read original source ↗ Open in workspace

recordType

paper

region

Global