SOURCE-LINKED INTELLIGENCE

FastE: Readout-Triggered Token Compression for LLM Embedding Inference

arXiv · AI, language, vision and robotics · article · Sep 8, 2026 · UTC

In this study, we identify depth-dependent prefix redundancy in final-readout LLM embedding models, notably across representative backbones including Qwen3-Embedding and Qwen3-VL-Embedding. We find that removing prefix states is substantially more damaging in shallow layers than at greater depth, showing that prefix states become increasingly compressible as the prefix and readout states propagate through the network. To this end, we introduce FastE, a training-free, plug-and-play method. FastE uses a shared fixed threshold on batch-mean readout-prefix alignment as a lightweight online heurist

Read original source ↗ Open in workspace

recordType

paper

region

Global