SOURCE-LINKED INTELLIGENCE

Curvature Cryptanalysis of Smooth Transformer Feed-Forward Networks

arXiv · AI, language, vision and robotics · article · Aug 28, 2026 · UTC

We show that smooth two-layer feed-forward networks (FFNs) expose an additional structural model extraction channel under a chosen-input raw-output oracle at the FFN branch; consider transformer FFN branches with GELU or SiLU activations under chosen-input raw-output access, without access to parameters, gradients, or internal activations; exploit a second-order leakage channel in which projected input Hessians form different mixtures of the same hidden symmetric rank-one factors induced by the FFN input weights. We formalize resulting Hessian collection as a partially symmetric decomposition

Read original source ↗ Open in workspace

recordType

paper

region

Global