Model inversion and the orthogonal space#

A fitted PLS model runs forwards: measurements go in, a predicted quality comes out. Running it backwards asks the question a product developer actually has: which inputs would give me the quality I want? Fixing the target and solving for the inputs is called model inversion (Jaeckle and MacGregor, 2000).

The answer is rarely a single recipe. A PLS model compresses several correlated inputs into a few latent directions, and one target value can pin down only one of them. What is left over is free, so inversion returns a flat set of designs, every one of which the model says will hit the target. That set is the null space, and its dimension is the number of components minus the rank of the response.

Note

This page is the API-level summary. The worked example it is drawn from, with the geometry, the uncertainty in the null-space direction, and the specification-region case, is in Process Improvement using Data.

Inverting a PLS model#

invert() takes the desired response on its original scale and returns a Bunch:

import pandas as pd
from process_improve.multivariate import PLS

cheese = pd.read_csv("https://openmv.net/file/cheddar-cheese.csv")
X = cheese[["Acetic", "H2S", "Lactic"]].iloc[4:]
y = cheese[["Taste"]].iloc[4:]

pls = PLS(n_components=2).fit(X, y)
result = pls.invert(y_desired=20.9)

result.x_new                  # the proposed inputs, in original units
result.null_space_dimension   # 1: a line of equally valid designs
result.null_space_basis       # the direction along that line
result.hotellings_t2          # how far the design sits from the data

The point returned is the direct-inversion solution, the one of smallest score norm. Because a step along the null space is at right angles to it, Pythagoras gives

\[\left\| \boldsymbol{\tau}_\text{DI} + s\,\mathbf{g} \right\|^2 = \left\| \boldsymbol{\tau}_\text{DI} \right\|^2 + s^2\]

so the norm grows by \(s^2\) and is least when no step is taken.

Warning

Smallest score norm is not smallest Hotelling’s \(T^2\). \(T^2\) divides each score by that score’s standard deviation before squaring, so it is a weighted sum of squares and its least value sits slightly off the direct-inversion point. For the cheese model above the two differ by a step of \(-0.103\). The gap widens as the score spreads become more unequal.

Walking the null space#

Pass null_space_coordinates to move along the free directions. The predicted response does not change; the recipe does.

for step in (-1.0, 0.0, 1.0):
    moved = pls.invert(y_desired=20.9, null_space_coordinates=[step])
    print(moved.x_new.round(2).to_list(), moved.hotellings_t2.round(2))
# [4.95, 6.1, 1.33]  1.63
# [5.52, 5.56, 1.4]  0.06
# [6.09, 5.02, 1.46] 2.44

That freedom is what you spend on a secondary goal: cost, supplier availability, or staying inside a regulatory window. It is free in terms of the predicted response, but not in terms of how much the data support the design, which is what the rising \(T^2\) records.

Choosing the number of components#

Inversion places a different demand on the model than prediction does. To rebuild a full set of inputs from one target number, the model has to describe the X-space well enough to map a score back into it, not merely describe the response. A one-component model spans only a line in the input space, so it returns exactly one recipe and the question of equivalent designs never arises.

Cross-validation chosen for predictive accuracy may therefore keep fewer components than inversion needs. Choosing more is a deliberate decision, made on different grounds, and worth stating as such.

The orthogonal space, and why it is the same set#

OPLS separates X into a single predictive component and one or more Y-orthogonal components:

\[\mathbf{X} = \mathbf{t}_\text{p} \mathbf{p}_\text{p}^T + \mathbf{T}_\text{o} \mathbf{P}_\text{o}^T + \mathbf{E}, \qquad \mathbf{y} = q_\text{p} \mathbf{t}_\text{p} + \mathbf{f}\]

The orthogonal scores do not appear in the equation for y, so moving along them changes X and leaves the prediction alone. That is the same property that defines the null space of an inverted PLS model, and for a single response the two subspaces are provably identical (García-Carrión et al., 2025).

The practical consequence is that inverting an O-PLS model needs no linear algebra. One equation in one unknown gives the predictive score by division:

from process_improve.multivariate import OPLS

opls = OPLS(n_orthogonal_components=1).fit(X, y)
opls_result = opls.invert(y_desired=20.9)

opls_result.predictive_score        # y_desired / q_p
opls_result.orthogonal_space_basis  # the freedom, handed back as an axis

Both routes describe the same set of designs. They report different representative points on it: PLS the point of smallest score norm, O-PLS the point whose orthogonal score is zero.

Note

The equivalence is proved for a single response. OPLS therefore accepts one response only. Multi-response inversion still works through PLS.invert, which returns a null space of dimension \(A - \text{rank}(\mathbf{Y})\); extending the equivalence proof to several responses is open work.

How well is the direction determined?#

The null space is exactly what the algebra says it is for a given model. That is not the same as saying the data pin it down. Its direction depends on the ratio of the y-loadings, and the later loadings are often the ones cross-validation did not keep.

For the cheese model above, a bootstrap over the calibration set puts the direct-inversion point on firm ground while leaving the direction poorly determined: the resampled null space sits a median of 21 degrees away from the reported one, and beyond 45 degrees in 15% of resamples. The asymmetry is worth checking before acting on a design reached by a long walk along the null space:

import numpy as np

rng = np.random.default_rng(0)
for _ in range(2000):
    sample = X.join(y).sample(len(X), replace=True, random_state=rng.integers(1 << 32))
    refit = PLS(n_components=2).fit(sample[X.columns], sample[y.columns])
    ...  # compare refit.invert(20.9) against the model fitted on all the data

A design proposed at the direct-inversion solution rests on firmer ground than one reached by stepping a long way along the null space.

Specification regions#

A product is usually accepted over a range rather than at a point. Sweeping the target across that range sweeps its null space across the score plot, and the swept lines fill out a multivariate specification region. Bounding it by the \(T^2\) limit keeps it inside the space the data support (Paris et al., 2021).

Warning

Reporting such a region as one range per input describes a box, and the box is not the region. The region is flat, since every point on it is rebuilt from the scores, while the box is a solid. For the cheese model, every one of the eight corners of that box satisfies all three ranges and none is an acceptable lot: six predict a response outside the window, and the other two predict an acceptable response from a recipe well beyond the \(T^2\) limit. Treat per-input limits as a summary of the region, not as the specification.

See also#