{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Course wrap-up: vocabulary, API map, and what comes next\n",
    "\n",
    "**Source worksheet:** [yint.org/w12](https://yint.org/w12) - the final week's concept review.\n",
    "\n",
    "Modules 1 to 7 spent ~50 pages of narrative and code on the\n",
    "fundamentals of designed experiments.  This wrap-up module gathers\n",
    "**every term and habit** the course covers, in one table, so you can\n",
    "test your own understanding by either restating each term in plain\n",
    "language or jumping to the module where it first appeared.\n",
    "\n",
    "It also maps each habit to the part of `process_improve` that\n",
    "implements it, so when a colleague asks \"how do I do X again?\", the\n",
    "answer is at most two clicks away."
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    ".. tip::\n",
    "\n",
    "   Two ways to use this page:\n",
    "\n",
    "   - **As a checklist.**  Scan the 35 concepts below.  For each one,\n",
    "     can you explain it to a colleague in two sentences?  If not,\n",
    "     jump back to the module that owns it.\n",
    "   - **As an API map.**  When you start a new study, the right\n",
    "     entry point into :mod:`process_improve` is usually one of a\n",
    "     handful of functions; the table at the end of this page lists\n",
    "     them."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The 35 concepts, organized by module\n",
    "\n",
    "Every concept from the week-12 worksheet, in order of where it first\n",
    "shows up in this series.\n",
    "\n",
    "### Phase A - Foundations (Modules 1 and 2)\n",
    "\n",
    "| Concept | One-line definition | First in |\n",
    "|---|---|---|\n",
    "| factor | Something we deliberately change. | [Module 1](01_two_factor_mindset.ipynb) |\n",
    "| categorical factor | A factor whose levels are discrete (e.g., R / W). | [Module 1](01_two_factor_mindset.ipynb) |\n",
    "| numeric factor | A factor on a continuous scale. | [Module 1](01_two_factor_mindset.ipynb) |\n",
    "| outcome / response | A measured quantity that depends on the factors. | [Module 1](01_two_factor_mindset.ipynb) |\n",
    "| objective | What we are trying to do (maximize, minimize, target). | [Module 1](01_two_factor_mindset.ipynb) |\n",
    "| levels | The specific values a factor takes in the design. | [Module 1](01_two_factor_mindset.ipynb) |\n",
    "| main effect | Average response change moving a factor from low to high. | [Module 1](01_two_factor_mindset.ipynb) |\n",
    "| coded value | Factor on the ``[-1, +1]`` scale. | [Module 2](02_coding_linear_model.ipynb) |\n",
    "| real-world value | Factor in its physical units. | [Module 2](02_coding_linear_model.ipynb) |\n",
    "| average effect | Mean response across one factor level. | [Module 1](01_two_factor_mindset.ipynb) |\n",
    "| model prediction | What the fitted equation says at a given point. | [Module 2](02_coding_linear_model.ipynb) |\n",
    "| extrapolate | Predict outside the design region (use with care). | [Module 2](02_coding_linear_model.ipynb) |\n",
    "| interactions | Effect of one factor depending on the level of another. | [Module 2](02_coding_linear_model.ipynb) |\n",
    "| one-factor-at-a-time | Sequential single-factor tweaks. Misses interactions. | [Module 2](02_coding_linear_model.ipynb) |\n",
    "\n",
    "### Phase B - Full factorial designs (Modules 3 and 4)\n",
    "\n",
    "| Concept | One-line definition | First in |\n",
    "|---|---|---|\n",
    "| full factorial | All combinations of all factor levels. | [Module 3](03_full_factorials.ipynb) |\n",
    "| cube plot | 3-D visualization of a 2^3 design. | [Module 3](03_full_factorials.ipynb) |\n",
    "| contour plot | 2-D map of the response surface. | [Module 3](03_full_factorials.ipynb) |\n",
    "| center point | A run at the middle of every factor range. | [Module 3](03_full_factorials.ipynb) |\n",
    "| replicates | Repeated runs at the same condition. | [Module 3](03_full_factorials.ipynb) |\n",
    "| standard error | Estimated noise level of a coefficient. | [Module 3](03_full_factorials.ipynb) |\n",
    "| interaction plots | Lines per level of one factor against another. | [Module 3](03_full_factorials.ipynb) |\n",
    "| Pareto plot | Bar chart of effect magnitudes, biggest first. | [Module 3](03_full_factorials.ipynb) |\n",
    "| little / no effect | A factor whose coefficient is dwarfed by noise. | [Module 3](03_full_factorials.ipynb) |\n",
    "| noise level | Run-to-run variation under \"identical\" conditions. | [Module 3](03_full_factorials.ipynb) |\n",
    "\n",
    "### Phase C - Doing less, learning more (Modules 5 and 6)\n",
    "\n",
    "| Concept | One-line definition | First in |\n",
    "|---|---|---|\n",
    "| half-fraction | Run half the corners of a full factorial. | [Module 5](05_fractional_factorials.ipynb) |\n",
    "| generators | The equation(s) that build extra factors from existing columns. | [Module 5](05_fractional_factorials.ipynb) |\n",
    "| aliases / confounding | Two effects whose contributions cannot be separated. | [Module 5](05_fractional_factorials.ipynb) |\n",
    "| defining relation | The product of all generators with ``I``. | [Module 5](05_fractional_factorials.ipynb) |\n",
    "| words | Each term in the defining relation (e.g., ``ABCD``). | [Module 5](05_fractional_factorials.ipynb) |\n",
    "| resolution | Length of the shortest word in the defining relation. | [Module 5](05_fractional_factorials.ipynb) |\n",
    "| screening experiments | Sift many factors with a small design. | [Module 5](05_fractional_factorials.ipynb) |\n",
    "| trade-off table | Standard chart for picking ``k`` and ``p``. | [Module 6](06_power_and_evaluation.ipynb) |\n",
    "| covariate | A variable you can measure but not control. | [Module 5](05_fractional_factorials.ipynb) |\n",
    "| disturbance | A variable you can neither measure nor control. | [Module 5](05_fractional_factorials.ipynb) |\n",
    "| nuisance variable | A controllable factor you do not scientifically care about. | [Module 5](05_fractional_factorials.ipynb) |\n",
    "| controlled variable | A factor you can set (factor / nuisance). | [Module 5](05_fractional_factorials.ipynb) |\n",
    "| blocking | Account for nuisance factors by structuring the design. | [Module 6](06_power_and_evaluation.ipynb) |\n",
    "| baseline | A reference run at a known operating point. | [Module 6](06_power_and_evaluation.ipynb) |\n",
    "\n",
    "### Phase D - Optimization (Module 7)\n",
    "\n",
    "| Concept | One-line definition | First in |\n",
    "|---|---|---|\n",
    "| sequential experiments | A *string* of small designs, each one informed by the last. | [Module 6](06_power_and_evaluation.ipynb) |\n",
    "| optimization | Find the factor settings that best meet the objective. | [Module 6](06_power_and_evaluation.ipynb) |\n",
    "| response surface | The mapping from factors to response. | [Module 7](07_response_surfaces.ipynb) |\n",
    "| steepest ascent | Move along the response gradient. | [Module 7](07_response_surfaces.ipynb) |\n",
    "| augmented model | Adding new runs (e.g., axial points) to an existing design. | [Module 7](07_response_surfaces.ipynb) |\n",
    "| nonlinearity | Curvature in the response surface that linear models miss. | [Module 7](07_response_surfaces.ipynb) |\n",
    "\n",
    "That is the 35 concepts of week 12.  If you can rattle off all of\n",
    "them, you are done."
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "The ``process_improve`` API, mapped to each step\n",
    "------------------------------------------------\n",
    "\n",
    ".. list-table::\n",
    "   :header-rows: 1\n",
    "   :widths: 28 50 22\n",
    "\n",
    "   * - Step\n",
    "     - Function / class\n",
    "     - Module\n",
    "   * - Define factors with real-world ranges\n",
    "     - :func:`~process_improve.experiments.structures.c`,\n",
    "       :class:`~process_improve.experiments.structures.Column`\n",
    "     - ``process_improve.experiments``\n",
    "   * - Collect columns into an experiment\n",
    "     - :func:`~process_improve.experiments.structures.gather`\n",
    "     - ``process_improve.experiments``\n",
    "   * - Build a full factorial\n",
    "     - :func:`~process_improve.experiments.designs_factorial.full_factorial`\n",
    "     - ``process_improve.experiments``\n",
    "   * - Build a fractional / response-surface design\n",
    "     - :func:`~process_improve.experiments.designs.generate_design`\n",
    "     - ``process_improve.experiments``\n",
    "   * - Fit a linear model\n",
    "     - :func:`~process_improve.experiments.models.lm`\n",
    "     - ``process_improve.experiments.models``\n",
    "   * - Run a full analysis pipeline\n",
    "     - :func:`~process_improve.experiments.analysis.analyze_experiment`\n",
    "     - ``process_improve.experiments.analysis``\n",
    "   * - Predict at new points\n",
    "     - :func:`~process_improve.experiments.models.predict`\n",
    "     - ``process_improve.experiments.models``\n",
    "   * - Evaluate a design before running it\n",
    "     - :func:`~process_improve.experiments.evaluate.evaluate_design`\n",
    "     - ``process_improve.experiments.evaluate``\n",
    "   * - Augment a design (axial points, replicates)\n",
    "     - :func:`~process_improve.experiments.augment.augment_design`\n",
    "     - ``process_improve.experiments.augment``\n",
    "   * - Multi-response optimization (desirability)\n",
    "     - :func:`~process_improve.experiments.optimization.optimize_responses`\n",
    "     - ``process_improve.experiments.optimization``\n",
    "\n",
    "Plot helpers all live under\n",
    "``process_improve.experiments.visualization`` and are reached either\n",
    "through the dispatch function ``visualize_doe(plot_type=...)`` or\n",
    "the explicit constructors:\n",
    "\n",
    "- ``visualize_doe(plot_type=\"square_plot\" | \"cube_plot\" | \"contour\" | \"surface_3d\")``\n",
    "  for design and surface plots.\n",
    "- ``visualize_doe(plot_type=\"pareto\" | \"half_normal\" | \"daniel\")``\n",
    "  for effect-magnitude plots.\n",
    "- ``visualize_doe(plot_type=\"residuals_vs_fitted\" | \"normal_probability\" | \"residuals_vs_order\" | \"box_cox\")``\n",
    "  for residual diagnostics.\n",
    "- ``visualize_doe(plot_type=\"interaction\" | \"steepest_ascent_path\")``\n",
    "  for interaction and optimization-trace plots.\n",
    "\n",
    "See the :doc:`../api/experiments` reference page for the full module\n",
    "contents."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A quick multi-response example\n",
    "\n",
    "The library can pick a compromise operating point that balances\n",
    "several competing responses using **desirability**.  Below, the same\n",
    "quadratic surface fitted in Module 7 is paired with a second response\n",
    "(call it \"cost\") whose objective is to be **minimized**.  The\n",
    "desirability search returns the factor settings that maximize the\n",
    "geometric mean of the two desirabilities."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "from process_improve.experiments import optimize_responses\n",
    "\n",
    "# Re-fit the Module 7 surface and pair it with a synthetic \"cost\".\n",
    "# We give the optimizer the coefficient lists directly.\n",
    "\n",
    "yield_coefs = [\n",
    "    {\"term\": \"Intercept\",    \"coefficient\": 69.77},\n",
    "    {\"term\": \"x1\",           \"coefficient\":  8.07},\n",
    "    {\"term\": \"x2\",           \"coefficient\":  3.75},\n",
    "    {\"term\": \"I(x1 ** 2)\",   \"coefficient\": -3.03},\n",
    "    {\"term\": \"I(x2 ** 2)\",   \"coefficient\": -1.80},\n",
    "    {\"term\": \"x1:x2\",        \"coefficient\": -2.11},\n",
    "]\n",
    "cost_coefs = [\n",
    "    {\"term\": \"Intercept\", \"coefficient\": 50.0},\n",
    "    {\"term\": \"x1\",        \"coefficient\":  5.0},   # cost climbs with x1\n",
    "    {\"term\": \"x2\",        \"coefficient\":  3.0},   # and with x2\n",
    "]\n",
    "\n",
    "result = optimize_responses(\n",
    "    fitted_models=[\n",
    "        {\"response_name\": \"yield_pct\", \"coefficients\": yield_coefs,\n",
    "         \"factor_names\": [\"x1\", \"x2\"]},\n",
    "        {\"response_name\": \"cost_eur\",  \"coefficients\": cost_coefs,\n",
    "         \"factor_names\": [\"x1\", \"x2\"]},\n",
    "    ],\n",
    "    goals=[\n",
    "        {\"response\": \"yield_pct\", \"goal\": \"maximize\", \"low\": 50, \"high\": 80},\n",
    "        {\"response\": \"cost_eur\",  \"goal\": \"minimize\", \"low\": 40, \"high\": 70},\n",
    "    ],\n",
    "    method=\"desirability\",\n",
    ")\n",
    "desir = result[\"desirability\"]\n",
    "print(\"Best compromise factor settings (coded units):\")\n",
    "for k, v in desir[\"optimal_coded\"].items():\n",
    "    print(f\"  {k} = {v:+.3f}\")\n",
    "print()\n",
    "print(\"Predicted responses at the compromise point:\")\n",
    "for k, v in desir[\"predicted_responses\"].items():\n",
    "    print(f\"  {k} = {v:.2f}\")\n",
    "print()\n",
    "print(\"Individual desirabilities:\")\n",
    "for k, v in desir[\"individual_desirability\"].items():\n",
    "    print(f\"  {k} = {v:.3f}\")\n",
    "print(f\"Composite desirability: {desir['composite_desirability']:.3f}\")"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    ".. admonition:: Guidance\n",
    "\n",
    "   Desirability is a *compromise*, not a Pareto optimum.  If two\n",
    "   responses point in different directions (here, increasing yield\n",
    "   also increases cost) the optimizer settles somewhere in the\n",
    "   middle, weighted by the importance of each goal.  Always *plot*\n",
    "   the desirability overlay (``visualize_doe(plot_type=\"overlay\")``)\n",
    "   to see what you are giving up."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Confirmation and sensitivity\n",
    "\n",
    "The single most under-rated step at the end of an optimization:\n",
    "\n",
    "1. **Confirm.**  Run one experiment at the predicted optimum.  If\n",
    "   the measured response matches the prediction within noise, the\n",
    "   model is trustworthy *at that point*.\n",
    "2. **Sensitivity check.**  Run two or three experiments slightly\n",
    "   away from the optimum (in each factor direction).  None of them\n",
    "   should beat the optimum; if one does, the search continues.\n",
    "3. **Document.**  Save the entire script and the design history;\n",
    "   reproducibility is the difference between \"we found the\n",
    "   optimum\" and \"we have a story we tell ourselves about the optimum\".\n",
    "\n",
    "These three together are how you decide *the work is done*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What to read next\n",
    "\n",
    "- The **companion textbook**: [Process Improvement using Data](https://learnche.org/pid), Chapter 5, for the formal\n",
    "  derivations and a more rigorous treatment of resolution and\n",
    "  confounding.\n",
    "- The `process_improve.experiments` **API reference** for the\n",
    "  full set of helpers (optimization strategies, design augmentation,\n",
    "  knowledge-base advice).\n",
    "- The **case studies** under ``docs/user_guide/case_studies`` for\n",
    "  larger real-world worked analyses, including the oil-company\n",
    "  factorial.\n",
    "\n",
    "That is the end of Applied DoE.  Good luck with your next study."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}