SOURCE-LINKED INTELLIGENCE

Breakthrough 3D-Stacked AI Inference Chip Enabling the Deployment of Multi-Billion-Parameter LLMs on Edge Devices

CORDIS · observation · Publication date unknown

Breakthrough 3D-Stacked AI Inference Chip Enabling the Deployment of Multi-Billion-Parameter LLMs on Edge Devices Large Language Models (LLMs) are at the center of the roadmaps of almost all large tech companies for new features in electronic devices. However, they usually consists of billions of parameter and running them outside the cloud in edge devices limits the chip size often to less than a square centimetre of silicon and less than 1 W continuous power consumption amongst other criteria. Current technologies not just fail to provide these technical features but also lack of the potential to meet these requirements in the future. This includes digital hardware (energy in-efficiency) as well as new compute paradigms (e.g. in-memory computing) that can only run models of few million parameters on such a small footprint. Instead of the expensive and limited technology node scaling, SEMRON is working on a 3D scaled AI inference chip to incorporate LLMs on a tiny footprint without overheating. Thi

Read original source ↗ Open in workspace

recordType

award

status

SIGNED

region

EU

value

2499999

unit

EUR