Mathematical Optimization

PEAR: Which Prediction Errors Actually Change the Decision?

A constrained optimizer cannot react to every prediction error. PEAR keeps the directions that can move the decision, making it most natural when system dynamics and constraints stay fixed while objective coefficients change across instances.

This paper argues that a model need not predict every objective coefficient equally well. Some prediction errors may be large without changing the optimizer at all. A much smaller error in another direction can change the selected solution and sharply increase the realized cost. PEAR uses this difference: it trains the predictor on the part of its error that can move the downstream decision under the current constraints.

The approach is especially relevant when the system dynamics and constraints are reused while cost or reward coefficients change from one instance to the next. Examples include process operation with a fixed plant model but changing electricity prices, dispatch over the same network with changing marginal costs, and portfolio optimization with a fixed risk model but changing expected returns.

For a regular strictly convex problem, the paper shows that this intuition is exactly the local gradient of decision regret. The scope is narrower for the LP method and the portfolio experiment. The former adds smoothing and a heuristic normal component; the latter stops a covariance-gradient path that is present in the experimental optimization problem. The distinction between the theorem and these extensions is the main point to keep in view.

1. Repeated system, changing objective coefficients

Let x denote the information available before a decision and c∈ℝn an objective coefficient that is not yet known. A neural network predicts

c^ =fθ(x).

The prediction is not the final decision. It enters an optimization problem:

z*(c^) = arg minz [φ(z)+c^⊤z] subject to Az=b, Gz≤h.

Here φ and the constraint system describe the part of the optimization model that is already known. PEAR’s exact theorem treats the prediction as entering only through the linear coefficient c^. In a control problem, for example, the plant dynamics, input bounds, and quadratic control penalty may be fixed while a model predicts a linear cost term associated with the current operating condition. In a mean-variance portfolio, the same structure appears if covariance and portfolio constraints are fixed while expected returns are predicted.

Ordinary supervised learning minimizes the coefficient error,

LMSE =12 ‖c^−c‖2.

Decision-focused learning instead evaluates the decision made with the prediction under the true coefficient:

R(c^;c) = φ(z*(c^)) +c⊤z*(c^) − [φ(z*(c)) +c⊤z*(c)].

The first term is the realized cost of the decision induced by the prediction. The second is the perfect-information optimum. Training on this regret requires the sensitivity ∂z*/∂c^, which is usually the expensive part of differentiating through the optimizer.

2. Discard errors that cannot move the decision

Consider the constraint z1+z2=1. Feasible decisions lie on a line. Compare the prediction errors (1,−1) and (1,1).

The first changes the relative costs of z1 and z2, so it can move the solution along the feasible line. The second raises both costs equally. For every feasible decision,

(c1+1)z1 +(c2+1)z2 =c1z1 +c2z2+1.

It adds the same constant to every feasible objective value, so the optimizer does not change. MSE tries to correct both errors. PEAR removes the second type before sending the learning signal back to the predictor.

This is a statement about directions, not individual coefficients. PEAR does not label one coefficient important and another irrelevant. A combination of coefficient errors can be invisible to the optimizer because it lies normal to the feasible set. Another combination of the same size can be tangent to that set and change the decision.

3. Active constraints define the relevant directions

At the current solution, collect the equality constraints and active inequalities in

J= [AGA].

A small feasible displacement must satisfy Jdz=0. The local tangent and normal spaces are therefore

T=ker(J), N=range(J⊤).

The simple two-variable example used ordinary Euclidean geometry. A general strictly convex objective has its own local curvature. With

H=∇2φ(z*),

PEAR first scales the error by H−1 and then projects it onto T in the metric defined by H. Calling the entire operation a Euclidean projection would be inaccurate unless H=αI.

4. The projected error is the exact local regret gradient

Assume the active set does not change under a small perturbation. The local KKT conditions are

∇φ(z*) +c^ +J⊤y*=0, Jz*=b~.

Differentiating them gives

[ HJ⊤ J0 ] [ dz dy ] = [ −dc^ 0 ].

Eliminating the dual displacement yields

dz=−PHdc^, PH =H−1 − H−1J⊤ (JH−1J⊤)−1 JH−1.

Let e=c^−c. Stationarity gives ∇φ(z*)+c=−e−J⊤y*. Since PHJ⊤=0, the dual term disappears from the chain rule, leaving

∇c^R =PH (c^−c).

This is the paper’s main result. Under its assumptions, PEAR is not merely a plausible surrogate direction. It equals the local gradient of regret with respect to the predicted linear coefficient.

5. The implementation changes the backward signal

The released QP code returns an ordinary squared error as its forward value. Its backward method overrides the MSE derivative and returns PHe. PEAR is therefore better understood as a custom gradient rule than as a new scalar loss.

The full n×n matrix PH need not be formed. If k constraints are active, the method solves

(JH−1J⊤)v =JH−1e

and returns

g=H−1e −H−1J⊤v.

The central solve is k×k. This is attractive when relatively few constraints are active and solves with H are cheap.

6. Where the exact theorem stops

The derivation requires H≻0, linearly independent active constraints, and strict complementarity. These assumptions keep the active set locally fixed and the solution map differentiable. The result is local. At an active-set boundary, J changes and the gradient can be nonsmooth.

A pure LP does not satisfy the curvature assumption because H=0. Its optimizer is piecewise constant in the cost vector: a small cost change usually leaves the optimal vertex unchanged, while crossing a normal-cone boundary can make the solution jump.

The paper handles this by adding quadratic smoothing,

φ(z) =λ2‖z‖2, H=λI.

The resulting gradient belongs to the smoothed problem, not the original LP. The implementation then adds a normalized normal component,

n=λ−1J⊤v, ginj =g+β ‖g‖‖n‖n.

This can help escape a flat LP region, but it is a heuristic. It also reintroduces the normal direction that the original geometric argument removed as locally decision-irrelevant. The strictly convex QP result, the smoothed LP gradient, and the LP gradient with normal injection are three different claims.

7. The portfolio experiment omits the covariance path

The portfolio problem is

minw λ2w⊤Σw −μ⊤w subject to 1⊤w=1, w≥0.

The theorem matches this problem when Σ is fixed and only μ is predicted. In the experiment, the network predicts a 21-day return path. Both the sample mean and a covariance matrix constructed from historical and predicted returns enter the optimizer. The decision is therefore w*(μ^,Σ^), and the total derivative with respect to predicted returns has two paths:

dRdr^ = ∂R∂μ^ ∂μ^∂r^ + ∂R∂Σ^ ∂Σ^∂r^.

The released PEAR backward pass returns the projected gradient for the predicted mean and None for the predicted covariance. It computes a partial derivative with respect to the mean while treating the current covariance as fixed. Differentiable QP baselines can receive gradients through both inputs. Without an ablation, it is unclear how much of PEAR’s portfolio result comes from the projection and how much comes from stopping the covariance gradient.

8. The computational advantage is conditional

PEAR reads the active set from a finite-tolerance primal-dual solution. A nearly binding constraint can enter or leave the set when the numerical threshold changes. Near degeneracy, J may be unstable and JH−1J⊤ may be ill-conditioned. Numerical stability relative to a differentiable solver is therefore an empirical question.

Runtime also depends on problem structure. PEAR is most favorable when k≪n and the Hessian is sparse, diagonal, or already factorized. If many nonnegativity constraints are active, as can happen in a sparse portfolio, k can approach n. A dense covariance matrix still requires expensive solves with H.

9. Strong results, but not a universal win

The LP benchmarks use a 5-by-5 shortest-path problem and a 100-item knapsack, with increasingly nonlinear feature-to-cost mappings. PEAR and LAVA train on an LP relaxation for knapsack and are evaluated on the original integer problem.

At polynomial degree 8, normalized regret on knapsack is 2.285% for MSE, 0.763% for SPO+, and 0.437% for PEAR. PEAR also records the best degree-8 shortest-path result at 4.246%. It does not win every setting: at degree 4, SPO+ obtains 0.761% and PEAR 0.774%. The phrase “best decision quality among all baselines” is slightly broader than the table supports.

PEAR is faster than the differentiable QP layers in the reported portfolio comparison: 122.3 seconds versus 147.6 for QPTH and 321.9 for CVXPYLayers. MSE takes only 33.0 seconds. The defensible claim is that PEAR is one of the faster decision-focused methods, not the fastest baseline without qualification.

The portfolio results are promising but variable. PEAR reports normalized regret of 85.38%, a Sharpe ratio of 1.44, and the lowest maximum drawdown. Across five seeds, cumulative return is 184.19±86.24%, compared with 139.77±115.74% for QPTH. That uncertainty is too large to establish clear economic superiority.

The constraint-shift experiment exposes a useful limitation. When the shortest-path source and target change, MSE is best at every tested degree. At degree 8, MSE scores 14.00, PEAR 21.42, and SPO+ 43.40. DFL deliberately concentrates accuracy on the training-time decision geometry. When that geometry changes, the same specialization can hurt transfer.

10. Assessment

The paper’s main theorem is both simple and useful. When a regular strictly convex optimizer repeatedly solves the same system with changing linear objective coefficients, the regret gradient is ordinary prediction error after curvature scaling and tangent-space projection. This gives a clear answer to which prediction errors deserve learning capacity.

The qualifications are equally concrete. The LP version is a smoothed and then heuristically modified method. The portfolio implementation omits a covariance-gradient path. The speed advantage depends on a small, stable active set and cheap Hessian solves. None of these points invalidates the theorem; they mark the boundary between the theorem and the broader experimental method.

Reference

Junhyeong Lee, Sangjin Jin, and Yongjae Lee. Decision-Focused Learning via Tangent-Space Projection of Prediction Error. ICML, 2026. A source URL, DOI, and arXiv identifier were not provided with the reviewed material.