SOURCE-LINKED INTELLIGENCE

XMerge: Cross-Axis Selection and Reconstructive Layer Merging for LLM Depth Compression

arXiv · AI, language, vision and robotics · article · Sep 2, 2026 · UTC

Removing complete transformer layers preserves a standard serving architecture, but existing depth-compression methods can lose substantial quality, and the loss varies unpredictably across models. We introduce XMerge, a post-training method with two components. Cross-axis selection identifies a block with low relative-magnitude and angular hidden-state change, and local boundary reconstruction re-fits the adjacent surviving block to match the original two-block output. XMerge uses no task labels or end-to-end fine-tuning, and it introduces neither architectural changes nor additional inferenc

Read original source ↗ Open in workspace

recordType

paper

region

Global