The deployment of foundation models in resource-constrained mobile and wearable devices presents unique challenges and opportunities, particularly in health and wearable sensing applications. This advanced seminar explores the cutting-edge intersection of large AI models, embedded systems, and digital health, with a focus on real-world applications in wearable and mobile biosignal sensing platforms.

The course examines two primary themes: (1) the application of foundation models in multimodal biosignal sensing contexts, including real-time physiological monitoring, neural signal processing, behavioral understanding, and clinical decision support through wearable devices such as EEG headsets, smartwatches, smart rings, eyewear, and earables; and (2) the technical challenges and solutions for deploying these powerful models on resource-constrained devices, including model compression, efficient inference, on-device learning, and real-time processing requirements. Example topics we will cover include:

• Application Domains in Health and Wearables: Foundation models across key health monitoring domains including sleep science through multimodal biosignal integration, movement analysis using IMU and pressure sensors for gait assessment, cognitive state monitoring via EEG and eye-tracking in smart glasses, and cardiovascular health through continuous PPG/ECG monitoring in everyday wearables. Also explore how foundation models enable synthetic data generation to address data scarcity challenges in health sensing, particularly for rare conditions and diverse populations.
• Systems Optimizations and Architecture: Core technical themes include efficient execution through model compression and power management, specialized cooperative foundation models sharing compressed representations, modular architectures allowing selective deployment based on available resources, firmware-level integration for always-on monitoring, cross-modal knowledge transfer between different biosignals, and adaptive model selection based on device context and constraints.
• IoT Applications and Environments: Beyond health sensing, we also briefly look at foundation models in broader IoT contexts such as ecological monitoring, smart homes, environmental monitoring, smart agriculture, etc with a focus on systems that need to operate locally on embedded resource-constrained platforms.

The course will primarily consist of paper readings, presentations, and discussions. Students will critically examine recent advances in the field through both academic papers and industry developments. For 3-credit option (PhD-level recommended), a semester-long research project is required, involving either the development of a novel application using foundation models on wearable platforms or the implementation of optimization techniques for embedded deployment.

Hours:
Wed. at 4-5:15pm in LGRC A104A

Instructor
Deepak Ganesan
341 LGRC, Computer Science Department

VP Nguyen
A235 LGRC, Computer Science Department

Course Mailing List: AT cs.umass.edu