This course provides a comprehensive exploration of Physical AI, a critical and rapidly evolving field at the intersection of intelligence and the material world. We will begin by situating this term within the context of related concepts such as Embedded AI, Cyber-Physical Systems (CPS), the Internet of Things (IoT), and Ubiquitous Computing. A central objective of this course is to move beyond buzzwords and critically analyze how Physical AI differentiates itself and what unique principles it encompasses.

Throughout the semester, students will investigate the fundamental components of intelligent systems that perceive, reason about, and act in our physical world. While this scope naturally includes robotics, we will also explore broader concepts of embodiment, real-time sensing, and dynamic decision-making in tangible environments.

The course is structured in two distinct phases. The first half, leading up to the midterm, will focus on foundational theories and methods in Physical AI and robotics. In the second half, students will form teams to apply this knowledge in a significant, hands-on project. This capstone project will involve developing and deploying solutions on real robots or other embedded computing systems, providing practical experience in building physically embodied intelligence.

Required prerequisite subject: Electronic Mathematics A (Or other course dealing with linear algebra, probability, and random process)

Week 1: Introduction to Physical AI class: Course overview, goals, and logistics. Defining "Physical AI" vs. IoT, CPS, Robotics.

Week 2: Learning from Demonstration: Sensing and Action: The core loop of embodied agents. Imitation Learning: Behavior Cloning (BC). Inverse Reinforcement Learning (IRL) and its challenges.

Week 3: Foundations of Embodied Perception. How AI "see" and "hear": Sensor fundamentals. Intro to Computer Vision (CV) for robotics: Object detection, segmentation. Representations: From raw sensor data to meaningful state.

Week 4: The Transformer Revolution & Self-Attention: Breaking down the Transformer architecture. Applications beyond NLP: Vision Transformers (ViT). Why attention mechanisms matter for robotics.

Week 5: Vision-Language Models (VLMs) Connecting vision and text (e.g., CLIP). Zero-shot generalization and emergent capabilities. Paper Reading Session 1: Key VLM papers.

Week 6: Vision-Language-Action (VLA) Models From understanding to acting: Models that output actions (e.g., RT-1).Tokenizing everything: sensors, language, and motor commands. Paper Reading Session 2: Seminal VLA papers.

Week 7: Current Topics & Midterm Review Hot topics: Sim-to-Real, foundation models for robotics, safety. Review of all concepts (Weeks 1-6). Q&A session for midterm exam.

Week 8: Midterm Exam Written exam covering all theoretical concepts.

Week 9: Team Project Kick-off & Hardware Introduction, Introduction to the final team project competition. Overview of available hardware platforms. Team formation and initial project brainstorming.

Week 10: Lab 1: Sensing the World: Reading data from physical sensors.

Week 11: Lab 2: Taking Action: Controlling actuators, Basics of real-time control.

Week 12: Project Workshop I: System Integration: Dedicated in-class time for team projects. How to structure your project code.

Week 13: Project Workshop II: Milestone 1 Check-in, Team project milestone presentations , Peer feedback, and troubleshooting session.

Week 14: Advanced Lab Topic / Project Workshop III (Flexible), Advanced lab topic (e.g., On-device ML, simple CV) OR Dedicated project work time.

Week 15: Project Workshop IV: Final Preparations, Final in-class work session. Debugging, tuning, and preparing for the final demonstration.

Week 16: Final Team Project Competition & Demos, Teams demonstrate their Physical AI systems. Presentations, awards, and course wrap-up.

Learning theory, and team project.

Please bring your own laptop after the midterm exam.
Undergraduates students can also take this course.