Repository navigation
Conversation
…ooks
New subject folder, three notebooks on one synthetic five-factor reactor
(data/raw/doe_reactor_runs.csv, generated by orteach.doe from a printed seed
and a stated true model). No student or lab data; the instance is invented.
Source: written new for this library, prompted by recurring dissertation-
committee questions from students in other departments (how many runs, how
many replicates, how to cut a design down, what to do with runs already
taken). No term folder was migrated, so there is no prior defect to record.
What each notebook does:
- planning_an_experiment (seed_planning.py): six steps in the order a study
is planned, each with one piece of theory.
1. Noise: replicates vs repeated measures; chi-square interval for sigma
and a one-sided bound as the planning value.
2. Run count: SE = 2 sigma / sqrt(N), N ~ 31 (sigma/Delta)^2, hidden
replication, replicates by number of factors; exact noncentral-t power
(0.810 for the 36-run design) checked by simulation (0.809).
3. Reduction ladder 3^5x3 -> 2^5x3+4 -> 2^5+4 -> half -> quarter, with the
power each rung keeps; why center points add df but not precision.
4. Randomize and block: drift as a fake effect (bias = drift x the effect's
correlation with run order); blocking on ABCDE leaves main effects and
2FIs unchanged.
5. Runs already taken: OFAT rank 6, no error df, variance 2 sigma^2 vs
4 sigma^2 / N.
6. Runs to add: greedy D-optimal augmentation with a block column,
D-efficiency, fold-over (stirring's +6.06 alias falls to +0.13), axial
points named.
Closes with an answer key (true sigma 1.5: planned power 0.81, actual 0.95).
- factorial_designs (seed_factorial.py): effects by hand, half-normal plot and
Lenth margins, half and quarter fractions with alias algebra drawn on the
cube, center-point curvature test. Now signposts which planning step or
ladder rung each section explains.
- space_filling_designs (seed_space_filling.py): Latin hypercubes in 2-D and
3-D, maximin, why run count does not grow with factors. Marked as for
computer experiments only.
Package: orteach.doe (model_matrix, effect_se, sigma_interval, sigma_bound,
power, runs_needed, augment, d_efficiency, log_det, plus the factorial and
LHS functions). AUGMENT_RIDGE is a regularizer, not a comparison tolerance.
Tests: tests/test_doe.py, 36 tests, including a coverage simulation for the
sigma interval, t-power vs simulation, t-power -> normal as df grows, OFAT
rank and variance, augmentation rank and no repeats, D-efficiency of full and
half fractions, and a concavity bound on the ridge.
Guards checked by breaking them: an effect_se using len(x) fails two tests
and the planning agreement cell; a >= tie-break in augment fails only the
planning agreement cell (the augmentation order), not pytest.
Checks: pytest 251 passed; check_notebooks 25 notebooks, 0 failing;
check_seeds 19 seeds, 0 failures.
Not done: Colab clones REF v1.0.1, which does not contain orteach.doe, so
these notebooks run locally but not in Colab until a v1.1.0 release.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Source: Jones's review of PR #12. The notebook used regression and showed confounding (drift, aliasing) but never said why a regression line through runs nobody designed is not an experiment. Adds a second theory section to step 5, "a regression line is not an experiment": - Omitted-variable bias, b_A = beta_A + beta_B * delta, as "correlation is not causation" written as a formula, and the three ways to get delta = 0: measure and include (costs 1/sqrt(1 - r^2) in SE), set the factors orthogonally, randomize. - 40 logged runs simulated from the reactor's true model (seed 1119), with catalyst tied to temperature (r = 0.85). Temperature-only slope 9.34, 95% CI 8.39 to 10.29, misses the true 6; the formula reproduces it exactly; adding catalyst gives 6.29 with SE inflated 1.92x; the designed 2^5 gives 5.847 with or without catalyst. - Figure: logged runs coloured by catalyst with fitted and true slopes, and the four coefficient intervals against the truth. - What to do with undesigned runs, tied to step 6 (det(M'M) ~ 1 - r^2 for two columns, so D-optimal augmentation decorrelates them). Also: intro table and "In short" step 5, an RHO exercise, the agreement cell now compares the four coefficients against doe.coefficients (30 numbers, agree to 2.2e-16), folder README theory list. Checks: pytest 251 passed; check_notebooks 25, 0 failing; check_seeds 19, 0 failures; ruff clean. Earlier outputs unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Source: Jones's review of PR #12. Studies often measure a vector of responses per run; the notebook handled one response only. Adds "More than one response" at the end of step 2: - Theory: Y = MB + E, B-hat = (M'M)^-1 M'Y. M depends only on the settings, so the same runs serve every response, and the joint fit equals separate fits column by column (checked: gap 1.4e-14). - Purity, a second response simulated on the same 36 runs from a model stated in the cell (seed 2612, sigma 0.8, Delta 1). Sizing per response: purity's sigma/Delta 1.364 against yield's 0.945, 58.4 runs against 28.1; the 36-run design gives purity 0.504 power. So the second response sets the size. - False alarms: 32 tests, 1.6 expected by chance. Purity flags BE and CE at 5%, not in its true model; Bonferroni keeps exactly A, D, AD, and also drops yield's real curvature, so the prose recommends a primary response. - Figure: t statistics for both responses from one model matrix, with the 5% and Bonferroni lines. - Pointer: readings of one unit over time are repeated measures; two-stage curve parameters become the columns of Y. Degradation notebook not written. Seed choice: 2611 gave a purity pilot s of 0.205 against a true 0.8, which made purity look easy and hid the point; 2612 gives 0.789. The exercise asks the reader to vary SEED_P and see how often four center runs mislead. Also: intro table and "In short" step 2, a DELTA_P/SEED_P exercise, folder README theory list; agreement cell now 48 numbers (purity sigma plan, runs needed, power, 15 coefficients vs doe.coefficients), agree to 7.1e-15. Checks: pytest 251 passed; check_notebooks 25, 0 failing; check_seeds 19, 0 failures; ruff clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The abstract still counted the notebooks before 02 and 15 were added. README and CLAUDE.md already say twenty-five. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Source: Jones's review of PR #12. The plain analysis script written for a student's study was much easier to follow than this notebook's code, which used terse names (M36, tc36, B_joint, Xo, sig_lo), packed several ideas into each line, and had few comments. Every code cell in seed_planning.py now uses plain names (design36, t_crit36, coef_both, X_ofat, sigma_low), one idea per block, and a comment on each step. Specific changes: - The reduction ladder is a list of dicts with named fields (runs, sum_x2, terms, se, df, power) instead of positional tuples read as r[3], r[4]. - Term names and their column indices are built once (terms, term_index) and reused, instead of re-deriving "ABCDE".index(ch) in five cells. - The fold-over mirror rows come from one readable comprehension instead of a 32x8 boolean matrix and argmax. - The augmentation loop names its pieces (current, M_trial, best_logdet) and says why ties go to the first candidate. - Plot calls are one per line instead of chained with semicolons. What the notebook computes is unchanged, and this is checked: all 27 text outputs (every printed number, table and Figure line) are byte-identical to the previous execution, so no prose number moved. The exercise that named words2[:5] now names terms[:5]. Checks: pytest 251 passed; check_notebooks 25, 0 failing; check_seeds 19, 0 failures; ruff clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Source: Jones's review of PR #12, following the same rewrite of the planning notebook (ec09fbf). The other two DOE notebooks used the same terse style. factorial_designs (seed_factorial.py): - Plain names: Xh/yh -> X_half/y_half, Xq/yq -> X_quarter/y_quarter, n_f/n_c -> n_corner/n_center, q -> hn_quantile, hi/lo -> mean_high/mean_low, corner -> is_corner, C -> corr. - The interaction cell names its four run groups (A_high, B_low, ...) instead of repeating the boolean conditions inline. - Lenth's PSE, ME and SME each carry a comment saying what they are. - Comments on every step; one plot call per line instead of semicolon chains. space_filling_designs (seed_space_filling.py): - Comments on the construction, the distance computation, the maximin search and the sampling study; the repeated pair index is named once (pair). - One plot call per line, including the 3-D axes, instead of semicolon chains. - Every random-number call is left exactly where it was, so the drawn designs cannot change. What the notebooks compute is unchanged, and this is checked against the previous execution: factorial's 23 text outputs are byte-identical; space-filling's printed text is identical once Jupyter's stream chunking is ignored (one cell's print now arrives as one chunk instead of two). Checks: pytest 251 passed; check_notebooks 25, 0 failing; check_seeds 19, 0 failures; ruff clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A new subject folder,
notebooks/15_design_of_experiments/, with three notebooks on one synthetic five-factor reactor, plus theorteach.doepackage module and its tests.planning_an_experimentseed_planning.pyfactorial_designsseed_factorial.pyspace_filling_designsseed_space_filling.pyThe folder README gives the reading order and says where to stop.
Key numbers (from outputs)
Data
The instance is invented and generated from a printed seed. No student, lab or sponsor data is included, and no private material is committed.
Checks
pytest: 251 passed, 36 of them intest_doe.pytools/check_notebooks.py: 25 notebooks, 0 failingtools/check_seeds.py: 19 seeds, 0 failureseffect_sefails two tests and the planning agreement cell.>=tie-break inaugmentfails only the agreement cell. pytest does not catch it.Not done
REF = v1.0.1, which predatesorteach.doe, so they run locally but not in Colab until a v1.1.0 release:paths.TAG, the pyproject version andREFin the seeds and notebooks;🤖 Generated with Claude Code