SOURCE-LINKED INTELLIGENCE

Can LLMs Use Relational Transformer Embeddings?

arXiv · AI, language, vision and robotics · article · Aug 31, 2026 · UTC

Injecting frozen relational-encoder embeddings as soft tokens into a large language model (LLM) is a conceptually appealing fusion strategy: the encoder handles multi-table structure, the LLM handles language and reasoning, and no lossy text serialization is required. We test this hypothesis concretely by injecting embeddings from a frozen Relational Transformer (RT) into Qwen3.5-4B via a learned MLP projection and LoRA adaptation, trained first with supervised fine-tuning (SFT) on chain-of-thought reasoning traces and then with group-based reinforcement learning (GSPO). We evaluate across 10

Read original source ↗ Open in workspace

recordType

paper

region

Global