2.1 · State, Observation, Belief, and Control
Many apparent architecture failures are interface failures. A tensor named state may omit velocity, contact, or controller memory; an action may be a waypoint rather than a motor command.
- Distinguish state, observation, history, and belief.
- Separate policy action from low-level control.
- Write a complete action interface specification.
State versus observation
A Markov state contains the information needed by the transition model. An observation is what sensors record. Images, joint positions, or force readings can be incomplete even when the physical process itself is Markov.
Belief as an information state
Under partial observability, the system should represent uncertainty over possible hidden states. A belief or learned history representation can support decisions when one observation is insufficient.
Action is not control
A policy may output joint targets, Cartesian deltas, velocities, torques, trajectories, or action chunks. The lower-level controller converts that representation into fast actuator commands. Units, frame, rate, horizon, and application semantics are part of the interface.
Tensor shape is not an interface specification; physical meaning must be explicit.
