SOURCE-LINKED INTELLIGENCE

Adaptive Tokenization and Memory in Foundation Models for Efficient and Long-Horizon AI

CORDIS · observation · Publication date unknown

Adaptive Tokenization and Memory in Foundation Models for Efficient and Long-Horizon AI "The recent revolution in generative AI is powered by the ever-growing scale of Foundation Models (FMs). This, however, causes a series of harmful ramifications, such as their unsustainable energy demand and environmental pollution, which accelerate climate change. Moreover, the scale of FMs jeopardises data privacy, as it compels users to deploy them on third-party servers rather than edge devices. AToM-FM sets out to reverse this trend by remedying a fundamental source of inefficiency in FMs: the granularity of the ""atomic"" units for representing information in current FMs is fixed, as it entirely depends on how they update their memory and segment input data (a process known as tokenization). Instead, AToM-FMs will couple their granularity with the complexity of each task, allocating only as much computing effort as needed. As its key technical breakthrough, AToM-FM will make memory and tokenization adaptive

Read original source ↗ Open in workspace

recordType

award

status

SIGNED

region

EU

value

1499453

unit

EUR