SOURCE-LINKED INTELLIGENCE

Speculative Probing: LLM Monitoring at Speculative-Decoding Cost

arXiv · AI, language, vision and robotics · article · Aug 28, 2026 · UTC

Real-time classification during language model inference is valuable for safety filtering, behavioral analysis, and model monitoring, but current approaches force a trade-off between accuracy and efficiency. Hidden-state probes are fast but limited: they are either not context-aware: operating on a single vector and cannot model interactions across positions; or they are very costly: having dedicated classifier models (Llama Guard, Qwen Guard, LLM-as-judge) or performing computation on hidden states for all tokens and then pooling the results (MultiMax). This shows an intrinsic trade-off betwe

Read original source ↗ Open in workspace

recordType

paper

region

Global