692EF: Embedded Foundation Models in Wearable Computing, IoT and Mobile Health Sensing
Overview Schedule Reading List
Overview

The deployment of foundation models in resource-constrained mobile and wearable devices presents unique challenges and opportunities, particularly in health and wearable sensing applications. This advanced seminar explores the cutting-edge intersection of large AI models, embedded systems, and digital health, with a focus on real-world applications in wearable and mobile biosignal sensing platforms.

The course examines two primary themes: (1) the application of foundation models in multimodal biosignal sensing contexts, including real-time physiological monitoring, neural signal processing, behavioral understanding, and clinical decision support through wearable devices such as EEG headsets, smartwatches, smart rings, eyewear, and earables; and (2) the technical challenges and solutions for deploying these powerful models on resource-constrained devices, including model compression, efficient inference, on-device learning, and real-time processing requirements. Example topics we will cover include:

  • Application Domains in Health and Wearables: Foundation models across key health monitoring domains including sleep science through multimodal biosignal integration, movement analysis using IMU and pressure sensors for gait assessment, cognitive state monitoring via EEG and eye-tracking in smart glasses, and cardiovascular health through continuous PPG/ECG monitoring in everyday wearables. Also explore how foundation models enable synthetic data generation to address data scarcity challenges in health sensing, particularly for rare conditions and diverse populations.
  • Systems Optimizations and Architecture: Core technical themes include efficient execution through model compression and power management, specialized cooperative foundation models sharing compressed representations, modular architectures allowing selective deployment based on available resources, firmware-level integration for always-on monitoring, cross-modal knowledge transfer between different biosignals, and adaptive model selection based on device context and constraints.
  • IoT Applications and Environments: Beyond health sensing, we also briefly look at foundation models in broader IoT contexts such as ecological monitoring, smart homes, environmental monitoring, smart agriculture, etc with a focus on systems that need to operate locally on embedded resource-constrained platforms.

The course will primarily consist of paper readings, presentations, and discussions. Students will critically examine recent advances in the field through both academic papers and industry developments. For 3-credit option, a semester-long research project is required, involving either the development of a novel application using foundation models on wearable platforms or the implementation of optimization techniques for embedded deployment.

Seminar Structure

The course consists of regular meetings with student presentations. Students are expected to participate in the following activities:

Presentations

Each student will present individually or in a group during one meeting session. Presentations will cover a paper selected from the reading list available on Moodle. Each presentation consists of:

  1. Context and Problem Statement (5 minutes)

    Introduce the paper's core problem and its significance in the field. Briefly outline relevant background work and highlight the specific challenges this paper addresses.

  2. Technical Contribution (10 minutes)

    Detail the key technical aspects and methodological approaches presented in the paper.

  3. Future Directions (5 minutes)

    Outline potential research directions building upon this work. Students may present a specific project proposal including concrete objectives and milestones.

  4. Interactive Discussion (15 minutes)

    Open discussion on the paper's contributions, methodology, and proposed extensions.

Paper Discussion Preparation

To demonstrate paper comprehension and prepare for meaningful discussion, each student must prepare one substantive question or critical observation about the paper to share during the discussion period. These can focus on connecting concepts across papers, critiquing methodology choices, proposing alternative approaches or discussing real-world applications or implications.

3-Credit Option

Students have the option to take this course for 3 credits by completing an independent project. This project can expand upon topics covered in the presentations or explore related research directions. Students choosing the 3-credit option must:

  • Discuss and finalize their project proposal with the instructor(s) within the first two weeks of the semester
  • Define concrete objectives and milestones for the semester
  • Meet regularly with the instructor(s) to discuss progress

Attendance

Students are expected to attend all class sessions. If you need to miss a class, please notify the instructor in advance for approval.

Hours:
Wed. at 4-5:15pm in LGRC A104A

Instructor
Deepak Ganesan
341 LGRC, Computer Science Department

VP Nguyen
A235 LGRC, Computer Science Department

Course Mailing List: AT cs.umass.edu