Lesson 3 · System Design

1.3 · The Complete Engineering Stack

A production Physical AI system is a stack, not a single model. Reliable behavior depends on the interfaces between sensing, state estimation, learned inference, runtime authorization, control, monitoring, and recovery.

Learning outcomes

  • Map the layers of a production Physical AI system.
  • Locate safety and authorization boundaries.
  • Recognize physical hallucinations as system-level failures.

From sensors to state

Raw sensor streams require calibration, timestamp alignment, filtering, and representation. A learned model cannot recover information destroyed by an inconsistent frame, stale timestamp, or missing sensor.

Proposal, authorization, execution

A learned model should usually propose an action or trajectory. A runtime layer then checks constraints, confidence, timing, collision risk, and operating mode before a lower-level controller executes the authorized command.

Physical hallucinations

A physically invalid output may be semantically plausible yet geometrically impossible, dynamically infeasible, temporally stale, or unsafe under contact. The monitor and fallback path must be designed alongside the learned component.

Key takeaway

Reliability lives at the interfaces between the model and the machine.