perf(csr): avoid CSR copies and full-grid scaling when freezing constraints

[FabianHofmann]

Closes #977.

Note

The following content was generated by AI.

Changes proposed in this Pull Request

Adding a frozen constraint from a sparse expression copied the lhs CSR matrix two to three times. It also broadcast the scaling and the mask over the full coordinate grid before it picked the active rows. This PR removes these copies.

• Share the lhs matrix. CSRConstraint.from_csr and _kept skip the row slice when every row stays active. The constraint then shares the matrix with the expression. This is safe, because all code that changes a constraint's matrix replaces it and does not write to it in place.
• Copy only before eliminate_zeros. assign_labels copies the matrix only when it contains explicit zeros and is still shared, so the user's expression is never changed.
• Pick mask and scaling at the active rows. A new helper, CSRConstraint._active_values, reads a broadcast view per dimension instead of expanding it to the full grid. A scalar scaling (the default 1) needs no grid array at all.
• Keep full scaling validation. Scaling is still validated on every value the user passes, including masked cells. This matches the unfrozen path. validate_scaling now also accepts a plain np.ndarray.

Peak memory of add_constraints(..., freeze=True), measured with the reproducer from the issue:

nnz	grid	active rows	mask	lhs CSR	before	after
1,000,000	10,000	10,000	–	12.0 MB	24.7 MB	0.3 MB
1,000,000	99,997	99,997	–	12.4 MB	31.2 MB	3.2 MB
1,000,000	99,997	89,843	90 %	12.4 MB	28.3 MB	15.8 MB
2,000,000	1,256,000	12,523	1 %	29.0 MB	69.2 MB	32.7 MB

With a 90 % mask, one copy of the kept rows is still needed. With a 1 % mask, most of the remaining peak comes from broadcasting the rhs over the grid, which this PR does not change.

Tests in test/test_csr.py:

• The row scaling test now covers scalar, per-dimension and full-grid scaling, each with and without a mask.
• New tests check that invalid scaling on masked rows still raises, and that the peak memory of a frozen build stays well below the size of the lhs matrix.
• test_zero_coefficient_rows_stay_active now also checks that the lhs expression is not changed.

Checklist

• AI-generated content is marked (see AGENTS.md).
• Code changes are sufficiently documented; i.e. new functions contain docstrings and further explanations may be given in doc.
• Unit tests for new features were added (if applicable).
• A note for the release notes doc/release_notes.rst of the upcoming release is included.
• I consent to the release of this PR's code under the MIT license.

[codspeed]

Merging this PR will not alter performance

✅ 181 untouched benchmarks
⏩ 181 skipped benchmarks¹

---

_{Comparing perf/977-csr-frozen-copies (22ed32b) with master (f74c309)}

Footnotes

1. 181 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[github-actions]

Build cost — v1 vs legacy

v1 build peak & time relative to legacy, on this commit — not a comparison against master (that is CodSpeed).

peak — v1 / legacy	time — v1 / legacy

Full table (time + peak, mean)

benchmarks/drivers/test_build.py::test_build[basic-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.07617 (1.11)   │   15.03 (1.00) 
 (v1)        0.06873 (1.0)   │    15.00 (1.0) 

benchmarks/drivers/test_build.py::test_build[basic-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.08721 (1.11)   │   12.04 (1.00) 
 (v1)        0.07878 (1.0)   │    12.04 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=0]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03249 (1.08)   │   15.20 (1.0) 
 (v1)        0.03001 (1.0)   │   15.20 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=100]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.05498 (1.12)   │   44.93 (1.0) 
 (v1)        0.04903 (1.0)   │   44.93 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=50]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03737 (1.08)   │   11.51 (1.0) 
 (v1)        0.03461 (1.0)   │   11.51 (1.0) 

benchmarks/drivers/test_build.py::test_build[expression_arithmetic-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.08475 (1.07)   │   24.34 (1.06) 
 (v1)        0.07925 (1.0)   │    23.04 (1.0) 

benchmarks/drivers/test_build.py::test_build[expression_arithmetic-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.09876 (1.08)   │   16.12 (1.00) 
 (v1)        0.09144 (1.0)   │    16.12 (1.0) 

benchmarks/drivers/test_build.py::test_build[knapsack-n=10000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.02157 (1.14)   │   752.18 (1.10) 
 (v1)        0.01895 (1.0)   │    685.15 (1.0) 

benchmarks/drivers/test_build.py::test_build[knapsack-n=100]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.01991 (1.06)   │   3.12 (1.33) 
 (v1)        0.01876 (1.0)   │    2.34 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=0]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.07238 (1.29)   │   126.16 (1.44) 
 (v1)        0.05612 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=100]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.07022 (1.29)   │   126.16 (1.44) 
 (v1)        0.05458 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=50]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.07109 (1.28)   │   126.16 (1.44) 
 (v1)        0.05562 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[masked-n=100]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.04647 (1.02)   │    715.12 (1.0) 
 (v1)        0.04573 (1.0)   │   787.73 (1.10) 

benchmarks/drivers/test_build.py::test_build[masked-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.04486 (1.09)   │   4.54 (1.27) 
 (v1)         0.0412 (1.0)   │    3.57 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=0]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.3182 (1.05)   │   704.12 (1.09) 
 (v1)        0.3034 (1.0)   │    643.85 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=100]
                 time (s)        peak (MiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.3387 (1.05)   │   18.34 (1.0) 
 (v1)        0.3215 (1.0)   │   18.34 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=50]
                 time (s)       peak (MiB) 
 name                mean   │         mean 
───────────────────────────────────────────
 (legacy)   0.3344 (1.06)   │   9.54 (1.0) 
 (v1)        0.3161 (1.0)   │   9.54 (1.0) 

benchmarks/drivers/test_build.py::test_build[milp-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.06425 (1.09)   │   3.77 (1.12) 
 (v1)        0.05889 (1.0)   │    3.37 (1.0) 

benchmarks/drivers/test_build.py::test_build[milp-n=50]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.0642 (1.09)   │   216.59 (1.10) 
 (v1)       0.05914 (1.0)   │    196.23 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=0]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.03275 (1.09)   │   938.49 (1.0) 
 (v1)        0.02992 (1.0)   │   938.49 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=100]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.03506 (1.08)   │   9.66 (1.0) 
 (v1)        0.03251 (1.0)   │   9.66 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=50]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.03394 (1.10)   │   5.32 (1.0) 
 (v1)        0.03096 (1.0)   │   5.32 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance_sparse-severity=0]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.01711 (1.03)   │   1.28 (1.0) 
 (v1)        0.01665 (1.0)   │   1.28 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance_sparse-severity=100]
                 time (s)       peak (MiB) 
 name                mean   │         mean 
───────────────────────────────────────────
 (legacy)   0.0171 (1.03)   │   1.28 (1.0) 
 (v1)       0.01661 (1.0)   │   1.28 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance_sparse-severity=50]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.01703 (1.02)   │   1.28 (1.0) 
 (v1)        0.01663 (1.0)   │   1.28 (1.0) 

benchmarks/drivers/test_build.py::test_build[piecewise-n=1000]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.1601 (1.05)   │   946.85 (1.06) 
 (v1)        0.1523 (1.0)   │    891.54 (1.0) 

benchmarks/drivers/test_build.py::test_build[piecewise-n=10]
                 time (s)         peak (KiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.1571 (1.05)   │   12.01 (1.00) 
 (v1)          0.15 (1.0)   │    11.99 (1.0) 

benchmarks/drivers/test_build.py::test_build[qp-n=1000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.04092 (1.07)   │   147.70 (1.06) 
 (v1)        0.03809 (1.0)   │    139.87 (1.0) 

benchmarks/drivers/test_build.py::test_build[qp-n=10]
                 time (s)        peak (KiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.0404 (1.07)   │   2.60 (1.09) 
 (v1)        0.0379 (1.0)   │    2.38 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=0]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.03378 (1.10)   │   696.75 (1.03) 
 (v1)        0.03082 (1.0)   │    673.70 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=100]
                 time (s)         peak (MiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.1046 (1.06)   │   137.97 (1.0) 
 (v1)       0.09866 (1.0)   │   137.97 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=50]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.06477 (1.12)   │   69.22 (1.0) 
 (v1)        0.05792 (1.0)   │   69.22 (1.0) 

benchmarks/drivers/test_build.py::test_build[sos-n=1000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.03884 (1.11)   │   402.33 (1.00) 
 (v1)        0.03491 (1.0)   │    402.30 (1.0) 

benchmarks/drivers/test_build.py::test_build[sos-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03824 (1.11)   │   3.19 (1.19) 
 (v1)        0.03435 (1.0)   │    2.69 (1.0) 

benchmarks/drivers/test_build.py::test_build[sparse_network-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.04244 (1.09)   │   29.00 (1.54) 
 (v1)        0.03911 (1.0)   │    18.84 (1.0) 

benchmarks/drivers/test_build.py::test_build[sparse_network-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.05763 (1.15)   │   37.95 (1.43) 
 (v1)        0.05007 (1.0)   │    26.51 (1.0) 

benchmarks/drivers/test_build.py::test_build[storage-n=10]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)    0.07993 (1.0)   │    410.93 (1.0) 
 (v1)       0.08115 (1.02)   │   427.84 (1.04) 

benchmarks/drivers/test_build.py::test_build[storage-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)    0.08945 (1.0)   │     9.94 (1.0) 
 (v1)       0.09301 (1.04)   │   10.22 (1.03) 

📊 Interactive plots + CSV: download the semantics-report-v1-vs-legacy artifact from this run.

_{Report-only · not a gate · refreshed on every push · obsolete once legacy is dropped.}