Linear quadratic regulator: choosing feedback by cost

A cart can return to its target with many different feedback gains. LQR chooses gains by charging for state error and control effort. The cost you choose determines what the controller tries to improve.

Choose feedback through a cost

A linear quadratic regulator, or LQR, designs feedback for a linear state model with a quadratic objective. This lesson uses continuous time, an infinite horizon, and a fixed target at the origin. The controller knows the full state and faces no input constraint.

Russ Tedrake's MIT LQR notes derive the feedback law and its quadratic value function. The original cart example below makes both quantities checkable: you can calculate the commanded force and the cost still ahead.

Start with a cart and a regulation target

Use the same horizontal cart as the pole-placement lesson. Mass is m = 2 kg; viscous damping is b = 1 N·s/m. Position p starts at 1 m, velocity v starts at zero, and applied force u acts along the position axis.

The second equation uses the stated SI units: mass times acceleration equals applied force minus damping force. The target is p = 0 and v = 0. A position-only command cannot tell whether the cart already moves toward the target too quickly.

No disturbance, sensor error, or actuator lag enters this model. The state-space model determines what an optimal solution can mean.

Give the LQR objective clear units

Meters squared and newtons squared need explicit scales before we combine them. Define normalized state z, input a, and time τ:

All three weights are positive and dimensionless. The normalized time makes J dimensionless too. The unit scales make these numerical A and B entries match the familiar cart coefficients.

Choosing centimeters as the position unit without changing the scaling would change the objective. Always record the state order, normalization, and weight values with an LQR result.

Interpret Q and R as design choices

Each weight assigns a price to a specific squared quantity:

• qₚ charges for displacement from the origin.
• qᵥ charges for velocity, including motion toward the target.
• R charges for commanded force in either direction.

The controller balances their accumulated costs across the whole future. A larger position weight changes both feedback gains in this cart. A larger force weight reduces its initial force magnitude when the state weights stay fixed.

Normalizing by acceptable state and input scales can provide a starting point. MathWorks documents this approach as Bryson's rule, then checks the resulting response against design goals. Those scales do not impose hard limits on the motion or force.

Costs from different weight choices measure different objectives. Multiplying Q and R by four multiplies every candidate's cost by four. The minimizing feedback and its motion stay unchanged.

Solve the continuous-time Riccati equation

LQR uses a symmetric matrix P to describe the optimal future cost from state z: V(z) = zᵀPz. For this model, P satisfies the continuous-time algebraic Riccati equation, often shortened to CARE:

The stabilizing solution makes the eigenvalues of A − BK have negative real parts. General LQR software solves the matrix equation numerically. This two-state cart also permits a direct solution.

Let K = [kₚ, kᵥ] in normalized coordinates. Expanding the three distinct entries of the symmetric CARE and selecting the stabilizing branch gives:

The numbers 2 and 1 come from this normalized cart's mass and damping coefficients. With the chosen unit scales, the physical gain numbers match K: position gain has units N/m, and velocity gain has units N·s/m. These formulas are specific to this cart and diagonal Q.

Calculate one controller by hand

Choose qₚ = 4, qᵥ = 7, and R = 1. The formulas give kₚ = 2 and kᵥ = 3, with:

The physical closed-loop poles are both −1 s⁻¹. At the initial state, u = −2 N and V([1, 0]ᵀ) = 8. This cost includes the entire return to the origin.

You can check the CARE without a solver:

Subtract the second matrix from the first and add Q. Every entry becomes zero. The resulting position and velocity are z₁(τ) = (1 + τ)e^−τ and z₂(τ) = −τe^−τ.

Compare weights, motion, and force

The lab starts with equal weights: qₚ = qᵥ = R = 1. Move the time probe to inspect the exact response, or choose a preset. Each experiment starts from the same 1 m offset at rest.

Choose a cost, calculate feedback, inspect the result

What does this LQR objective ask the cart to do?

A 2 kg cart starts at 1 m with zero velocity. Damping is 1 N·s/m. Feedback uses exact position and velocity; the actuator applies unlimited force.

LQR presetCustom weightsEqual weightsPosition priorityForce priorityRepeated polesScale every weight by four

Position weight qp0.251416

Velocity weight qv0.2514716

Force weight R0.251416

Inspect time (s)

2.000000

Reset LQR lab

Normalize position by 1 m, velocity by 1 m/s, force by 1 N, and time by 1 s. All weights and costs are dimensionless. The objective integrates qp × position² + qv × velocity² + R × force² in these normalized coordinates.

Position gain kp (N/m)

1.000000

Velocity gain kv (N·s/m)

1.449490

Pole 1 (1/s)

-0.612372 + 0.353553j

Pole 2 (1/s)

-0.612372 - 0.353553j

Pole pattern

Complex conjugate poles

Current position (m)

0.554006

Current velocity (m/s)

-0.269952

Current force (N)

-0.162714

Initial force (N)

-1.000000

Accumulated cost

2.084644

Remaining cost

0.364845

Infinite-horizon cost

2.449490

Normalized value matrix P = [[2.449490, 2.000000], [2.000000, 2.898979]]. Remaining cost is zᵀPz at the inspected time. Accumulated cost plus remaining cost equals the infinite-horizon cost, subject to readout rounding.

Each weight choice defines its own objective. A smaller cost after changing weights does not establish a better controller. Scaling every weight by four preserves the motion and multiplies every cost by four.

Return the cart to the origin

Force that produces this response

All choices have stable poles for this exact continuous model. The eight-second plot does not certify settling, force feasibility, or robustness to model error. Readouts round to six decimal places.

Try these comparisons:

• Position priority: raise qₚ to 16. Initial force changes from −1 N to −4 N.
• Force priority: raise R to 4 with qₚ = qᵥ = 1. Initial force becomes −0.5 N; position at 2 s changes from 0.554006 m to 0.723173 m.
• Repeated poles: use the hand calculation above. At 2 s, force is zero even though the cart still moves.
• Scale every weight by four: the curves remain identical to equal weights, while the cost becomes four times larger.

For a distinct-real-pole example, set qₚ = 1, qᵥ = 16, and R = 1. The poles are approximately −0.244256 and −2.047032 s⁻¹. All permitted weights produce stable poles for this model.

The curves sample analytic solutions at 321 times. The force axis rescales using an analytic bound, while the position axis stays fixed. The eight-second display does not decide whether a particular settling-time requirement passes.

Account for the cost after the plot ends

The cart has future cost even after the visible window ends. Along the LQR trajectory, the Riccati equation gives:

For equal weights at t = 2 s, accumulated cost is 2.084644 and remaining cost is 0.364845. Their unrounded sum equals the infinite-horizon cost √6 ≈ 2.449490. Six-decimal display rounding can leave a one-unit difference in the final decimal place.

This split evaluates a fixed infinite-horizon controller. A separate finite-horizon LQR problem includes an ending time and usually a terminal cost; it generally produces a time-varying gain. The MIT finite-horizon derivation explains that change in the design problem.

Cost also makes a fair controller comparison possible when the objective stays fixed. The gains [2, 3] have equal-weight cost 2.75 from this starting state. Equal-weight LQR achieves √6, which is smaller; [2, 3] remains optimal for its own weights [4, 7, 1].

State what optimality guarantees

For the standard infinite-horizon result, require a stabilizable pair (A, B), positive-definite R, positive-semidefinite Q, and a detectable pair (Q¹ᐟ², A). Detectability means any mode invisible to the state penalty must already decay. Drake's LQR documentation states the corresponding stabilizability and detectability conditions.

Our cart is controllable, and every diagonal entry of Q is positive. Thus the penalty sees both states, and the stabilizing P is positive definite. The controllability and observability lesson explains the mode tests behind these conditions.

Within the declared linear model, LQR minimizes this quadratic objective over admissible unconstrained inputs. That statement gives no general guarantee about an actual robot's limits:

• A force penalty allows large commands when they reduce total cost. Saturating those commands changes the dynamics used by the proof.
• Model error, delay, and sampling require separate analysis. A stable nominal model alone proves no robustness margin for those changes.
• Unmeasured states require estimation. A Luenberger observer introduces estimation-error dynamics that this lab omits.
• A persistent load or a changed target needs an appropriate equilibrium, feedforward term, or integral-action design.

For nonlinear robots, a linearization can support local control near an operating point. The region where it works needs its own validation. This example teaches regulation of a known cart; it contains no learned dynamics or hardware result.

The Lyapunov stability lesson checks a quadratic function directly against a closed-loop state matrix. It supplies a stability-certificate route alongside the optimal-cost argument here.

For a direct comparison of separate feedback loops and input cancellation, see multivariable control. Its nearly singular example shows why a small tracking error can require large opposing commands, motivating an explicit effort cost.

Reproduce the gains and cost split

This standard-library Python example solves the same cart. All variables use the normalized coordinates above, so its time number also equals seconds. It handles complex, repeated, and distinct real poles.

from math import cos, cosh, exp, sin, sinh, sqrt


def design(qp, qv, r):
    kp = sqrt(qp / r)
    kv = sqrt(1 + 4 * kp + qv / r) - 1
    return kp, kv, (r * kp * (1 + kv), 2 * r * kp, 2 * r * kv)


def response(qp, qv, r, t):
    kp, kv, (p11, p12, p22) = design(qp, qv, r)
    d = (1 + kv) / 4
    beta = kp / 2
    delta = (1 + qv / r - 4 * kp) / 16
    if delta == 0:
        c, s = 1.0, t
    elif delta > 0:
        h = sqrt(delta)
        c, s = cosh(h * t), sinh(h * t) / h
    else:
        h = sqrt(-delta)
        c, s = cos(h * t), sin(h * t) / h
    p = exp(-d * t) * (c + d * s)
    v = -exp(-d * t) * beta * s
    u = -kp * p - kv * v
    tail = p11 * p * p + 2 * p12 * p * v + p22 * v * v
    return p, v, u, p11 - tail, tail, p11


def show(label, numbers):
    print(label, " ".join(f"{n:.6f}" for n in numbers))


show("default gains", design(1, 1, 1)[:2])
show("default state", response(1, 1, 1, 2)[:3])
show("default costs", response(1, 1, 1, 2)[3:])
show("repeated P", design(4, 7, 1)[2])
show("repeated state", response(4, 7, 1, 2)[:3])
show("repeated costs", response(4, 7, 1, 2)[3:])
show("scaled costs", response(4, 4, 4, 2)[3:])

Expected output:

default gains 1.000000 1.449490
default state 0.554006 -0.269952 -0.162714
default costs 2.084644 0.364845 2.449490
repeated P 8.000000 4.000000 6.000000
repeated state 0.406006 -0.270671 0.000000
repeated costs 7.120849 0.879151 8.000000
scaled costs 8.338578 1.459381 9.797959

The browser uses a short series near repeated poles to avoid division by a tiny frequency. It uses the same exact solution and checks the running-cost identity against independent numerical integration of the physical force balance.

Try two LQR exercises

Exercise 1. Equal weights give K = [1, √6 − 1] and total cost √6 from the stated initial state. Multiply qₚ, qᵥ, and R by four. What happens to K, initial force, and total cost? Does the larger cost show worse motion?

Check the common-scaling answer

K stays [1, √6 − 1], and initial force stays −1 N. The total cost becomes 4√6 ≈ 9.797959. Both trajectories match because every candidate input sequence's objective increased by the same positive factor.

The larger number reflects a rescaled objective. It does not show worse motion.

Exercise 2. For qₚ = 4, qᵥ = 7, and R = 1, use P = [[8, 4], [4, 6]]. At τ = 2, the state is [3e⁻², −2e⁻²]ᵀ. Calculate the force and remaining cost. Has the cart stopped?

Check the zero-force and future-cost answer

Force is −2(3e⁻²) − 3(−2e⁻²) = 0 N. Remaining cost is (72 − 48 + 24)e⁻⁴ = 48e⁻⁴ ≈ 0.879151. The accumulated cost is 8 − 48e⁻⁴ ≈ 7.120849.

Velocity is −2e⁻² ≈ −0.270671 m/s, so the cart still moves. At this instant, damping supplies its positive acceleration while commanded force passes through zero.

Sources and further study

• MIT Underactuated Robotics: Linear Quadratic Regulators, for the feedback law, value function, and distinction between infinite- and finite-horizon designs.
• Drake: feedback control design, for the LQR problem and its stabilizability and detectability conditions.
• MathWorks: LQR design, for weight definitions, state-feedback output, and normalization through Bryson's rule.

Next, use LQG control to drive a cart from noisy position readings. That lesson designs a sampled, finite-horizon controller and compares Kalman estimates with raw feedback under matched disturbances. The Luenberger observer remains a separate route into choosing continuous-time estimation-error poles.