SOURCE-LINKED INTELLIGENCE

Forward-Free LLM Depth Pruning via Weight Redundancy

arXiv · AI, language, vision and robotics · article · Sep 9, 2026 · UTC

Depth pruning reduces large language model (LLM) inference cost by removing complete Transformer blocks. Activation-based methods collect hidden states through forward passes on calibration data, while existing forward-free methods score each Transformer block separately without measuring similarity between blocks. We propose Weight-Redundancy Pruning (WRP), a forward-free depth-pruning method that estimates inter-layer redundancy from checkpoint weights to select blocks without calibration data or model forward passes. WRP compares attention output and MLP down-projection weights across layer

Read original source ↗ Open in workspace

recordType

paper

region

Global