Towards Foundation Models for Intelligent Real-Time Edge Applications

---

Authors:

Prof. Tarek Abdelzaher

Abstract:

"Advances in self-supervised AI revolutionized modern machine intelligence, but important challenges remain when applying these solutions in IoT contexts - specifically, on lower-end distributed embedded devices with multimodal specialized sensors, where ample training data are not readily available. The talk discusses the challenges in advancing self-supervised machine intelligence services and foundation models for intelligent real-time edge applications. We argue that, at training time, the key bottlenecks are data-related. Embedded computing relies on scarce sensor data modalities, unlike those commonly addressed in mainstream AI, necessitating solutions for efficient learning from scarce sensor data. At inference time, the bottlenecks are computational, calling for improved resource economy and novel scheduling policies. Further ahead, the convergence of AI around large language models (LLMs) introduces additional model-related challenges in embedded contexts. The talk discusses novel research directions in addressing these bottlenecks, covering data-, resource-, and model-related challenges in the IoT domain."