SOURCE-LINKED INTELLIGENCE

ProgramDistill: From Interactive Web Apps to Verifiable Reference-Guided SWE Tasks

arXiv · AI, language, vision and robotics · article · Sep 16, 2026 · UTC

Coding agents are typically evaluated with desired behavior specified through issues or instructions. In practical web development, however, agents may need to infer behavior from working software and implement it in an incomplete application. We introduce ProgramDistill, a benchmark evaluating coding agents on features discovered through interaction with fully functional reference applications. We build ProgramDistill by factorizing applications into features of different granularities, each associated with replayable behaviors executable via its gold patch. Our pipeline, mine-craft-patch, di

Read original source ↗ Open in workspace

recordType

paper

region

Global