SOURCE-LINKED INTELLIGENCE

LaMoC: Loss-Aware Modular Compression for LLMs

arXiv · AI, language, vision and robotics · article · Aug 31, 2026 · UTC

Modular compression has enabled considerable parameter reduction in LLMs while preserving strong language understanding and downstream task accuracy. However, existing joint modular compression methods primarily rely on activation statistics, leaving loss-sensitivity information and its module-level characterization underexplored. We investigate addressing this gap with LaMoC, a loss-aware modular compression methodology that blends activation and Empirical Fisher statistics through gradient-error alignment. LaMoC improves joint compression by selecting compression statistics that better align

Read original source ↗ Open in workspace

recordType

paper

region

Global