Panel diagnostics: can this attribute be modelled?#

Descriptive panel data: validate, check the panel, relate to the product scores each panelist. This page is about the prior question, asked of each attribute: does it behave like an intensity that assessors read off a linear scale at all? Two things break that assumption, and both break it quietly, so process_improve.sensory.diagnostics tests them before a model is fitted rather than after.

from process_improve.sensory import (
    assessor_variance_equality,
    boundary_occupancy,
    detection_rate,
)

occupancy = boundary_occupancy(validated.normalized_df)
equality = assessor_variance_equality(validated.normalized_df)

An attribute pinned against a bound#

The Mixed Assessor Model’s premise is that assessors compress or expand a linear scale. In a region where everyone records the same value, no scaling difference is expressible: there is nothing for the model to estimate, and what is left over is reported as disagreement.

boundary_occupancy() measures how much of each attribute lives against the floor or the ceiling:

>>> boundary_occupancy(panel).query("frac_floor > 0.5")[["attribute", "frac_floor", "frac_exact_zero"]]
     attribute  frac_floor  frac_exact_zero
2       burnt        0.78             0.71
5   medicinal        0.64             0.00

Floor, ceiling and exact-zero occupancy are reported separately because they are separate questions. Look at the second row above: medicinal is floor-pinned but has no exact zeros at all. That is the signature of a panel whose convention is to record “not perceived” as a small positive number rather than a zero. It looks pinned and is not, and only the exact_zero column tells the two apart.

The response that suits such an attribute#

For an attribute most assessors do not perceive, “how intense is it” has no answer, but “how often is it perceived at all” does. detection_rate() gives a product-by-attribute table of detection probabilities:

>>> detection_rate(panel)["burnt"].sort_values(ascending=False).head(3)
product
P07    0.83
P02    0.42
P11    0.08

Warning

A detection rate is not comparable with an intensity score. It is a probability on [0, 1], it does not carry the attribute’s units, and it must not be dropped into the same table, correlation matrix or PLS block as intensity-scored attributes without saying what it is. Two attributes with the same mean intensity can have very different detection rates, and the reverse.

A product-attribute pair nobody assessed comes back NaN rather than 0: never detected and never asked are different answers.

Unequal assessor variance, mistaken for scale use#

This is the highest-value check on the page. Grossmann et al. (2023) show that the Mixed Assessor Model reads unequal assessor variance as a scaling effect: an assessor who is simply noisier than the rest loads onto the same term that a scale-compressor does, which shifts the MAM F-test so that real disagreement is understated.

assessor_variance_equality() tests that precondition directly. Residuals are taken against the product mean first, which removes the genuine product effects that would otherwise dominate the spread; Levene’s test (median-centred, the Brown-Forsythe variant) then compares assessors:

>>> assessor_variance_equality(panel).query("p_equal_variance < 0.05")
    attribute  levene_stat  p_equal_variance  spread_ratio_max_min  n_assessors
1     bitter          6.42            0.0004                  5.31           11

A small p_equal_variance means the assessors genuinely differ in spread, and the MAM’s scaling coefficients for that attribute are measuring partly that. spread_ratio_max_min is the effect size to read alongside it: a p-value below 0.05 with a ratio of 1.4 across eleven assessors is a different situation from the same p-value with a ratio of 5.

An empty panel is an error#

mixed_assessor_model() used to return frames with no columns when handed a panel with no rows, so an over-filtered panel surfaced as KeyError: 'f_product_mam' somewhere downstream. It now raises a ValueError naming the condition and pointing at the filter that is the usual cause. The three functions on this page do the same.

References#

Grossmann, Ellis, Hopfer and others, “The effect of unequal assessor variance on the Mixed Assessor Model”, Food Quality and Preference, 105, 104792, 2023, doi:10.1016/j.foodqual.2022.104792.

API#

Kevin Dunn, 2010-2026. MIT License.

Preconditions for panel analysis: can this attribute be modelled at all?

Every model in this subpackage assumes the attribute behaves like an intensity that assessors read off a linear scale, and that what separates assessors is how much of that scale they use. Two things break the assumption, and both of them break it quietly:

The attribute is pinned against a bound. In a region where everyone records the same value, no scaling difference is expressible, so the Mixed Assessor Model has nothing to estimate and reports the residue as disagreement. boundary_occupancy() measures how much of an attribute lives against the floor or the ceiling; detection_rate() gives the response that is appropriate instead, a probability of detection rather than an intensity.
Assessors differ in how noisy they are, not in how they scale. assessor_variance_equality() tests that directly. Grossmann et al. (2023) show that the Mixed Assessor Model reads unequal assessor variance as a scaling effect, which shifts its F-test so that real disagreement is understated. A small p-value here means the model’s scaling coefficients are measuring partly that, and the F-test should be read with the finding in mind.

All three take long-format panel data with panelist_id, product, attribute and score columns; the canonical descriptive_long schema from validate_descriptive() satisfies that.

References

Grossmann, Ellis, Hopfer and others, “The effect of unequal assessor variance on the Mixed Assessor Model”, Food Quality and Preference, 105, 104792, 2023, doi:10.1016/j.foodqual.2022.104792.

process_improve.sensory.diagnostics.boundary_occupancy(panel, lo=0.0, hi=10.0, band=0.1)[source]#

Measure how much of each attribute sits against the ends of the scale.

An attribute pinned against a scale bound violates the Mixed Assessor Model’s premise that assessors compress or expand a linear scale: no scaling difference is expressible in a region where everyone records the same value. Use this before modelling to decide whether an attribute can be treated as an intensity at all, and detection_rate() when it cannot.

Floor, ceiling and exact-zero occupancy are reported separately because they are different questions. In particular, a panel whose convention is to record “not perceived” as a small positive number rather than an exact zero will look floor-pinned when it is not, and the exact_zero column is what distinguishes the two.

Parameters:

panel (pandas.DataFrame) – Long-format panel data with panelist_id, product, attribute and score columns.
lo (float, default 0.0) – Lower bound of the rating scale.
hi (float, default 10.0) – Upper bound of the rating scale.
band (float, default 0.10) – Width of the floor and ceiling bands, as a fraction of the scale range. The default counts a score as “at the floor” when it is within the bottom 10% of the scale. Must lie in [0, 0.5).

Returns:

One row per attribute, sorted by attribute, with columns:

attribute: The attribute name.
n: Number of non-missing scores.
at_floor, at_ceiling: Counts of scores inside the floor and ceiling bands.
exact_zero: Count of scores exactly equal to lo.
frac_floor, frac_ceiling, frac_exact_zero: The same three as fractions of n, which is what a keep/drop decision is actually made on.

Return type:

pandas.DataFrame

Raises:

ValueError – If a required column is missing, the panel has no rows, or the scale bounds and band are not a usable combination.

Examples

>>> occupancy = boundary_occupancy(validated.normalized_df)
>>> occupancy.query("frac_floor > 0.5")  # candidates for detection_rate instead

process_improve.sensory.diagnostics.detection_rate(panel, lo=0.0, band=0.1, hi=10.0)[source]#

Report, per product and attribute, the fraction of assessments that detected it.

This is the appropriate response for an attribute that boundary_occupancy() shows is pinned against the floor: the question “how intense is it” has no answer when most assessors record nothing, but “how often is it perceived at all” does.

Warning

A detection rate is not comparable with an intensity score. It is a probability on [0, 1], it does not share the attribute’s units, and it must not be dropped into the same table, correlation matrix or PLS block as intensity-scored attributes without saying what it is. Two attributes with the same mean intensity can have very different detection rates, and vice versa.

Parameters:

panel (pandas.DataFrame) – Long-format panel data with panelist_id, product, attribute and score columns.
lo (float, default 0.0) – Lower bound of the rating scale.
band (float, default 0.10) – Width of the floor band, as a fraction of the scale range. A score strictly above lo + band * (hi - lo) counts as detected.
hi (float, default 10.0) – Upper bound of the rating scale.

Returns:

Products (rows) by attributes (columns) of detection probabilities. A product-attribute pair that nobody assessed is NaN rather than 0: “never detected” and “never asked” are different answers.

Return type:

pandas.DataFrame

Raises:

ValueError – If a required column is missing, the panel has no rows, or the scale bounds and band are not a usable combination.

Examples

>>> rates = detection_rate(validated.normalized_df)
>>> rates["burnt"].sort_values(ascending=False)

process_improve.sensory.diagnostics.assessor_variance_equality(panel)[source]#

Test whether assessors are equally variable, per attribute.

The Mixed Assessor Model splits the assessor-by-product interaction into a scaling part and a disagreement part, and reads the scaling part as “this assessor uses a wider or narrower range of the scale”. Grossmann et al. (2023) show that an assessor who is simply noisier than the others loads onto that same scaling term, which shifts the MAM F-test so that real disagreement is understated. This function tests the precondition directly, so a caller can tell which of the two they are looking at.

Method: take residuals as score minus the product mean, within each attribute, which removes the genuine product effects that would otherwise dominate the spread. Then apply Levene’s test (median-centred, i.e. the Brown-Forsythe variant, for robustness against non-normal residuals) across assessors.

A small p_equal_variance means the assessors genuinely differ in spread, and the MAM scaling coefficients for that attribute are measuring partly that rather than scale use alone.

Parameters:

panel (pandas.DataFrame) – Long-format panel data with panelist_id, product, attribute and score columns. Replicates are used as-is; more replicates give the test more to work with, but it runs on unreplicated data too by drawing the spread from across products.

Returns:

One row per attribute, sorted by attribute, with columns:

attribute: The attribute name.
levene_stat: Levene’s test statistic. NaN when fewer than two assessors have enough observations to have a spread.
p_equal_variance: The p-value for the null “all assessors have the same residual spread”. Small means they do not.
spread_ratio_max_min: The largest assessor’s residual standard deviation divided by the smallest, an effect size to read alongside the p-value. inf where some assessor has no residual spread at all.
n_assessors: Number of assessors contributing to the attribute.

Return type:

pandas.DataFrame

Raises:

ValueError – If a required column is missing or the panel has no rows.

Examples

>>> equality = assessor_variance_equality(validated.normalized_df)
>>> equality.query("p_equal_variance < 0.05")["attribute"].tolist()