SOURCE-LINKED INTELLIGENCE

JustFit: 200K-Token LLM Serving on a 24 GiB Laptop with Just-in-Time State Management

arXiv · AI, language, vision and robotics · article · Sep 15, 2026 · UTC

Capable open-weight models make local coding and reasoning attractive, but their context and execution state strain laptop memory. We present JustFit, an MLX-based inference runtime that combines KVExec for compressed KV execution, PhaseSwap for component residency, and StateTrans for state-preserving serving transitions. These mechanisms fuse reconstruction and coordinate just-in-time materialization and release, independently of model-weight quantization. In full-execution capacity tests on a 24 GiB M4 Pro MacBook running Qwen3.8-27B MXFP4, three independent runs complete 196,608 input and 1

Read original source ↗ Open in workspace

recordType

paper

region

Global