SOURCE-LINKED INTELLIGENCE

Scalar-Order Theories of Deep Kernel Adaptation

CORDIS · observation · Publication date unknown

Scalar-Order Theories of Deep Kernel Adaptation The AI revolution is under way, yet we still lack a thermodynamic understanding of deep learning, that is explaining what a network can learn in the limit of large dataset and large network width. The central proposal of SOTA is that, the output of a deep network on real data can be predicted by an effective kernel - a similarity measure between data points - that shows simple low-dimensional adaptation to the dataset. This reconnects two regimes often seen as completely different: “lazy” learning (where a fixed kernel predictor explains outputs) and “rich” learning (where network features are plastic and little understood). Indeed, SOTA argues that rich learning in fully-connected networks effectively reduces predictor variance, while almost no adaptation of the mean predictor is predicted and observed. Convolutional networks instead show local adaptations of the kernel that also change the mean predictor. This is where the number of data samples is proportional to

Read original source ↗ Open in workspace

recordType

award

status

SIGNED

region

EU

value

193643.28

unit

EUR