What is physical AI? Follow a robot from sensing to action

Physical AI connects machine intelligence to sensing and action in the physical world. A mobile robot makes that connection concrete: it observes an aisle, chooses a motion, and checks what happened. Following that loop shows where learning helps and where geometry, estimation, and control still do essential work.

What physical AI means here

This lesson uses physical AI as a working label for AI systems that interpret physical surroundings and help choose or coordinate actions there. The term covers several approaches. It does not name one algorithm or a required kind of robot.

NVIDIA’s glossary, checked on October 11, 2026, uses a broad definition. Its examples include robots, autonomous vehicles, and cameras that support actions in physical spaces. That usage extends beyond machines with moving limbs.

For this lesson, the example is a mobile cart with sensors and motors. The useful questions are concrete: what does it observe, what does it infer, and how does that change its motion?

How embodied AI overlaps

Embodied AI emphasizes an agent interacting with an environment through its available observations and actions. The body and sensors affect what the agent can learn or do. An observation changes when the agent moves or acts.

Researchers can study embodiment in simulation. The 2019 Habitat paper describes embodied agents as virtual robots with configurable sensors. Its platform supports defining tasks such as navigation and instruction following inside simulated three-dimensional scenes.

These uses overlap; they do not create a fixed boundary between two fields. “Embodied” does not guarantee physical hardware, and NVIDIA’s “physical AI” usage includes fixed cameras. When reading a claim, check the actual agent, environment, task, and evidence behind the label.

Follow one mobile robot

Imagine a small cart moving toward a marked delivery point. A box partly blocks the aisle. A camera sees the scene, and wheel encoders report wheel rotation.

The cart needs to locate itself, identify occupied space, and choose a route with enough clearance. It then needs commands that produce the intended motion. New observations can reveal that the box moved or the cart slipped.

Illustrated robot loop

One action changes the next observation

Follow a mobile cart toward a waypoint in its map.

This drawing separates useful functions. A working system can combine several functions in one model, or run them at different rates. A learned policy might choose actions directly from observations, without exposing every intermediate box.

Separate the jobs inside the loop

The same sensor data can support different questions. Finding a box in an image, locating the cart on a map, and selecting wheel commands are different jobs.

Role	Question for this cart	Example output
Observation	What did the sensors measure?	An image and wheel-rotation counts
Perception	What useful scene features are present?	A detected marker and occupied space
State estimation	Where is the cart, and how is it moving?	Estimated position, heading, and uncertainty
Planning or policy	What should the cart do next?	A waypoint, route, or action
Control	Which command should reduce the motion error?	Requested wheel speed or motor torque
Actuation and world	What motion actually resulted?	Wheel forces, cart movement, and a changed view

Perception interprets measurements. State estimation combines available evidence with a model to infer quantities that the robot needs. SLAM joins estimation of the robot’s location with estimation of a map.

A planner can select a route using geometry. A controller then uses feedback to track the chosen motion. Modern Robotics’ control overview traces this relationship between sensors, controllers, actuators, and robot motion.

If you want to work on one of these functions, the robotics career roadmap compares roles and the skills their job postings ask for. Use the loop to choose a subsystem you want to understand, then choose a project that demonstrates that work.

Locate the learned parts

Learning can enter several parts of the loop. A trained image model might detect the box. A policy trained from demonstrations might select steering actions, as in the behavior-cloning lesson.

Classical methods can perform other jobs. Geometry can check whether the cart fits through a gap. A proportional controller can correct a measured motion error, and an estimator can combine encoder readings with known landmarks.

Robotics does not require every component to use machine learning. This example also needs no language model: the destination can arrive as a numerical waypoint. Choosing a learned component changes how one function is implemented; it does not remove physical constraints or measurement errors.

Moravec’s paradox asks why apparently ordinary perception and movement hide difficult engineering work. This lesson follows how those functions connect. The connections matter even when individual components perform well on their own tests.

Work through a biased position estimate

Now isolate the cart’s final approach on a clear, straight segment. This invented calculation examines one conventional controller inside the wider robot. It contains no learned model and is not a hardware experiment.

The stored destination is exactly r = 2.0 m in a fixed world frame. The true initial position is x[0] = 1.0 m. The cart’s position estimate is always 0.2 m ahead of its true position:

estimated position:  xhat[k] = x[k] + 0.2 m
position error:     e[k] = r − xhat[k]
commanded move:     u[k] = 0.5 e[k]
next true position: x[k+1] = x[k] + u[k]

Here k counts completed moves. The command u is a displacement in meters, and the gain 0.5 is dimensionless. Each move executes exactly before the next estimate arrives; the model omits acceleration, slip, delay, random noise, and command limits.

The destination is unbiased; only the current-position estimate has the stated offset. On the first move, the cart reports 1.2 m, so its error is 2.0 − 1.2 = 0.8 m. It commands 0.4 m and reaches a true position of 1.4 m.

Move k	True position before move (m)	Estimated position (m)	Commanded move (m)	True position after move (m)
0	1.000	1.200	0.400	1.400
1	1.400	1.600	0.200	1.600
2	1.600	1.800	0.100	1.700
3	1.700	1.900	0.050	1.750

Explain why the cart stops short

Each reported error is half the previous one. The controller keeps reducing the error it receives. Yet the true position approaches 1.8 m, leaving the cart 0.2 m short of its destination.

Substituting the estimate into the update gives:

x[k+1] = x[k] + 0.5(2.0 − x[k] − 0.2)
       = 0.5 x[k] + 0.9

x[k] = 1.8 − 0.8 × (0.5)^k

As moves continue, the estimate approaches 2.0 m while the true position approaches 1.8 m. The ideal model reaches neither limit in a finite number of moves. “Stops short” describes its limiting position; a real stopping tolerance would end the sequence earlier.

Feedback corrects the error it receives. It cannot identify this fixed bias from the same biased reading alone. Calibration or an independent measurement could provide the missing evidence.

The feedback-control lesson separates true error from measured error in another small model. For a physical cart, wheel odometry adds another source of disagreement between an estimated route and actual motion.

Ask what the demonstration establishes

An end-to-end task needs end-to-end evidence. A detector’s image accuracy does not establish that the cart reaches its destination. A controller that tracks its reported position does not establish that the position is correct.

For a robot demonstration, record the task and success tolerance. Identify the learned components, human inputs, and quantities the experiment assumed exact. Describe the starting positions, obstacles, sensor conditions, and failures the evaluation included.

Simulation provides useful controlled experiments. It also depends on assumptions about sensing, contact, and dynamics. Sim-to-real transfer examines how a policy’s behavior changes when those assumptions change, and distinguishes simulated evidence from hardware results.

The calculation here establishes one narrow result: this controller converges to the wrong true position under a fixed measurement bias. It makes no claim about a complete navigation system’s performance.

Try it yourself

Exercise 1: Reverse the bias. Keep the same destination, initial position, and gain. Change the estimate to xhat[k] = x[k] − 0.1 m. What are the first commanded move, the next true position, and the limiting true position?

Check the calculation

The first estimate is 0.9 m, so the error is 1.1 m. The command is 0.55 m, taking the cart to 1.55 m. At the limit, the estimate equals the 2.0 m destination, so the true position approaches 2.1 m: the cart ends up beyond the goal.

Exercise 2: Locate the missing evidence. A report says the camera detected the box correctly on 99 of 100 labeled images. Does that establish that the cart reached its destination without hitting the box? Name two additional measurements needed for that claim.

Check the reasoning

The image result evaluates detection on those images. It does not measure the route that the cart actually traveled. Record ground-truth final position against a stated arrival tolerance, and record collisions or minimum clearance along each attempted route.

Those measurements need a defined set of trials and a denominator that includes failures. Depending on the task, timing and human intervention may also matter. A successful detector is one component of the evidence.

The vision-language-action lesson follows a learned policy’s inputs and action interface. For communication between software components, compare ROS 2 topics, services, and actions: a stream of observations, a request with a reply, and a goal with feedback each need a different exchange.

The autonomy and teleoperation lesson follows the human and robot decisions within one task. It distinguishes continuous operator commands, shared assistance, and a bounded task the robot executes after a goal request.

Sources and further study

• NVIDIA: What Is Physical AI? gives one current industry definition, including robots and cameras that support actions in physical spaces. Checked October 11, 2026; this lesson attributes that terminology to NVIDIA.
• Savva and colleagues: Habitat, ICCV 2019 studies embodied AI using virtual robots, configurable sensors, and tasks in simulation.
• Lynch and Park: Control System Overview explains sensor feedback, actuator commands, and simplified robot-control models. The cart’s discrete calculation is an original teaching example.

To examine one learned component, continue to behavior cloning. Train a small policy, then compare its prediction errors with the motion it actually produces.