SOURCE-LINKED INTELLIGENCE

Form Over Content In Gradient-Based Data Attribution Methods

arXiv · Artificial Intelligence · article · Sep 17, 2026 · UTC

Data attribution methods using gradient similarity are widely used to analyze and select training data for large language models, but what gradient similarity actually measures is debated. Some interpret it as identifying task-relevant skills, while other work reports that surface form is the main factor. We resolve this debate for supervised fine-tuning examples by varying task and answer format independently. Specifically, we render benchmarks in different answer formats, such that datasets can share a task without a format or a format without a task. We find that gradient alignment follows

Read original source ↗ Open in workspace

recordType

paper

region

Global