Robot learning generalization: what is actually unseen?

“The robot generalizes” needs a second sentence: what did training leave out? A new starting position, a new object instance, and a new task each ask something different of a policy. One successful trial cannot stand in for all three.

This lesson holds training fixed and changes two observed coordinates. You will see a missing combination of familiar values, new values inside familiar ranges, and values beyond those ranges. Two fitted policies make the limits of each claim visible.

Name what changes before claiming generalization

A useful claim names the training conditions, evaluation conditions, available inputs, and task. It also says whether the policy stayed frozen or received new demonstrations before evaluation.

For example: “After fitting on three start-target pairs, the frozen policy reached the target from the held-out pair (1 m, 1 m) under exact displacement mechanics.” That statement identifies a checkable result. It says nothing yet about perception, unfamiliar objects, or another robot.

The LIBERO authors' benchmark description separates task suites that emphasize spatial relationships, objects, and goals. That separation illustrates why “new condition” needs a named axis. This lesson's numeric reaching grid does not reproduce LIBERO or measure its benchmark tasks.

Separate the axes of novelty

Record the dimensions that could affect behavior before building a split:

BridgeData V2 studies robot learning across varied tasks and environments. Its collection protocol varies objects, camera poses, and workspace positions, and supplies task information through language or goal conditioning. Those are named experimental choices, not interchangeable measures of general ability.

Our experiment implements only start-position and target-position variation. It keeps the task rule, observation quality, and mechanics unchanged. The other rows above describe questions for other experiments.

Define a reaching task with two observed inputs

A point robot starts at x meters on a line. It observes both x and the desired target g exactly. Its action u commands a displacement in meters, and the toy environment applies:

The expert's action is u⋆ = g − x. It reaches g in one step. This environment imposes no speed, torque, collision, or displacement limits, and it includes no sensor noise.

The goal is a legitimate policy input. At the same start, different targets require different actions, so withholding g would change the learning problem. Both compared policies receive identical information.

The experiment evaluates an algebraic reaching task. It contains no camera model, physical robot, learned low-level actuator, or continuous motion trajectory.

Keep three training conditions fixed

Each training record comes from a separate one-action expert demonstration:

Training contains two distinct values on each axis: −1 and +1. It contains only three of their four possible combinations. The pair (+1, +1) remains held out even though each coordinate value appears elsewhere in training.

No evaluation control changes these records. Selecting another start or target does not add a label, refit with that condition, or tune a model setting.

The robot-data leakage lesson addresses a related boundary: keeping evaluation episodes out of training. Here, every demonstration is one action, and the extra question is which condition values those separate episodes cover.

Fit two policies to the same records

The first model uses one nearest neighbor. It stores the three training records, finds the closest condition in the two observed coordinates, and copies that record's action:

Both coordinates use meters and equal distance weights. Squared distances within 10⁻¹² m² count as a numerical tie; the alphabetically earlier record ID wins. There is no input scaling learned from evaluation data.

The second model fits an affine policy by ordinary least squares:

The three noncollinear training conditions identify all three coefficients. Solving their equations gives:

• A: b₀ − bₓ − b𝓰 = 0.
• B: b₀ − bₓ + b𝓰 = 2.
• C: b₀ + bₓ − b𝓰 = −2.

Subtracting A from B gives b𝓰 = 1. Subtracting A from C gives bₓ = −1. Substituting back gives b₀ = 0, so the fitted rule is u = −x + g.

Both models fit every training action exactly. Their training errors therefore cannot select between their behavior at unseen conditions. The linear-regression lesson explains the squared-error objective behind the affine fit.

Map axis coverage and joint coverage

The lab checks 25 conditions formed from x, g ∈ {−2, −1, 0, 1, 2}. The symbols describe training membership:

• T: one of the three training pairs.
• R: a new pair of familiar axis values.
• X: an unseen start value with a familiar target value.
• G: an unseen target value with a familiar start value.
• B: both values unseen in training.

An unseen value can still lie inside an axis's training range. Zero lies between −1 and +1; ±2 lie beyond it. Neither observation establishes membership in an unspecified future probability distribution, so the widget reports explicit coverage labels. Any “out-of-distribution” claim also needs a stated reference population.

Joint coverage is stricter. The convex hull is the set of mixtures of the three training points, which forms a triangle. Here it satisfies x ≥ −1, g ≥ −1, and x + g ≤ 0.

The held-out pair (+1, +1) lies outside that triangle, despite using familiar values on both axes. Predicting there requires joint extrapolation relative to these samples. At (0, 0), both values are unseen but the pair lies on the triangle's edge, so it falls within the joint interpolation region.

The map stays fixed while you select a condition. Both endpoint rows use identical starts, targets, and mechanics. Their position axis also stays fixed; the connecting lines show displacement, not motion over time.

Test the missing combination

At x = 1 m and g = 1 m, no motion is necessary. The nearest-neighbor model compares:

The tie rule selects B and copies u = +2 m. The robot ends at 3 m with signed error +2 m. The affine fit gives u = −1 + 1 = 0, so it stays at the target.

Now try x = −2 m and g = 2 m. Both values extend beyond their training ranges. The nearest neighbor again copies B's +2 m command and ends at 0 m, giving error −2 m. The affine fit predicts +4 m and reaches 2 m under the same exact mechanics.

That comparison reports two specific evaluation conditions. The next section explains the unusually strong assumption that makes the affine result possible.

Explain why the affine policy succeeds

The expert rule belongs to the affine model family. Exact observations, noiseless labels, and three independent training equations let the fit recover that rule. The unchanged environment then applies its predicted displacement exactly.

These conditions explain the result. They do not show that three demonstrations generally suffice for robot learning, or that an affine policy can understand unseen tasks.

Many other functions agree with the same three training labels. For example:

The added term vanishes at A, B, and C. At (1 m, 1 m), however, it adds 4 m. Training labels alone cannot rule out this alternative; choosing an affine model supplies a restriction on how the learner extends those labels.

The toy's declared mechanics and expert objective establish which extension is correct here. A real evaluation must test the assumptions its fitted model relies on. Changes to observations, task goals, contact, or mechanics can invalidate an extension that worked on another axis.

Write a claim that matches the evaluation

Keep a record of the conditions you train on and the conditions you evaluate. Then state what stays fixed:

1. Observed inputs: what information can the policy actually use at deployment?
2. Training membership: which instances, coordinate values, combinations, or groups contribute demonstrations?
3. Held-out novelty: which axis changes, and which combinations remain unseen?
4. Adaptation: does the policy stay frozen, or does it receive labels, parameter updates, or calibration?
5. Outcome: what behavior counts, over what time or action budget, and under which termination rules?

For this lab, a precise conclusion is that the fitted affine policy reproduces the declared reaching rule at the checked start-target conditions. It does not establish new-object recognition, task transfer, or physical reliability.

The three training conditions also define a model-fitting set, not a probability distribution for future starts. The grid has no assigned deployment frequencies. Averaging every square equally would introduce a weighting choice that should appear in the report.

After defining the condition split, use robot-policy evaluation to specify outcomes and uncertainty. To vary mechanics after fitting, continue with sim-to-real transfer. To study states created by a policy's own actions, use behavior-cloning covariate shift.

The domain-randomization lesson changes training coverage while holding the learner and label budget fixed. It then compares both fits on identical familiar and shifted test conditions.

Reproduce the fit and condition checks in Python

This standard-library example solves the three training equations with exact fractions. Because the square design matrix has full rank, their unique solution is also the least-squares fit. Nearest-neighbor prediction uses the same training records and stated tie rule.

from fractions import Fraction as F

train = [("A", -1, -1, 0), ("B", -1, 1, 2), ("C", 1, -1, -2)]
# Rows [1, x, g | u]. Evaluation conditions never enter this matrix.
matrix = [[F(v) for v in (1, x, g, u)] for _, x, g, u in train]
for column in range(3):
    pivot = max(range(column, 3), key=lambda i: abs(matrix[i][column]))
    matrix[column], matrix[pivot] = matrix[pivot], matrix[column]
    divisor = matrix[column][column]
    matrix[column] = [value / divisor for value in matrix[column]]
    for row in range(3):
        if row != column:
            factor = matrix[row][column]
            matrix[row] = [a - factor * b
                           for a, b in zip(matrix[row], matrix[column])]
b0, bx, bg = [matrix[i][3] for i in range(3)]
print(f"affine coefficients: {b0}, {bx}, {bg}")

for name, x, g in [
    ("training", -1, 1),
    ("recombination", 1, 1),
    ("new start", 0, -1),
    ("new target", -1, 0),
    ("both inside", 0, 0),
    ("both beyond", -2, 2),
]:
    nearest = min(train, key=lambda r: ((x-r[1])**2 + (g-r[2])**2, r[0]))
    nn_error = x + nearest[3] - g
    affine_error = x + b0 + bx*x + bg*g - g
    print(f"{name}: nearest={nearest[0]}, "
          f"NN error={nn_error:.6f} m, affine error={float(affine_error):.6f} m")

Expected output:

affine coefficients: 0, -1, 1
training: nearest=B, NN error=0.000000 m, affine error=0.000000 m
recombination: nearest=B, NN error=2.000000 m, affine error=0.000000 m
new start: nearest=A, NN error=1.000000 m, affine error=0.000000 m
new target: nearest=A, NN error=-1.000000 m, affine error=0.000000 m
both inside: nearest=A, NN error=0.000000 m, affine error=0.000000 m
both beyond: nearest=B, NN error=-2.000000 m, affine error=0.000000 m

Check two generalization claims

Exercise 1. Select x = 0 m and g = 1 m. Which axis value is unseen? Is the pair inside the joint training triangle? Compute both policies' actions and signed endpoint errors.

Exercise 2. A report says the affine model's result at (−2 m, +2 m) proves generalization to new objects and new tasks. Rewrite the claim using what the lab actually changes. Does zero error at (0, 0) establish that the nearest-neighbor model generalizes to every unseen condition?

Sources and next steps

• Liu and colleagues, LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning, NeurIPS 2023, with the authors' implementation and task-suite definitions.
• Walke and colleagues, BridgeData V2: A Dataset for Robot Learning at Scale, CoRL 2023, including its varied collection conditions, task conditioning, and stated evaluation setup.

Choose one claim you want a robot policy to support. Write its condition axes and held-out groups before working through robot-policy evaluation.