perf(solvers): stop setting names in direct solver APIs by default by FabianHofmann · Pull Request #988 · PyPSA/linopy

FabianHofmann · 2026-09-25T11:41:29Z

Closes #978.

Note

The following content was generated by AI.

Changes proposed in this Pull Request

Setting the default names x{label} / c{label} was the largest single cost of the direct solver APIs. On a model with 200k variables and 200k constraints, to_highspy took 136 ms with names and 69 ms without. The names carry no information beyond the index, and the solution is already mapped back by position.

Why not the fix suggested in the issue: building a highspy.HighsLp in one pass is slower than the current getLp/passModel round trip. highspy copies numpy arrays into std::vector element by element (about 55 ms for 600k nonzeros, against about 25 ms for the round trip). There is no bulk name setter (a passColName loop takes 168 ms), and polars is already the fastest name builder tested (28 ms, against 42–55 ms for f-strings or numpy). So the only big lever is not setting names, as JuMP does with set_string_names_on_creation.

Changes

Model.set_names_in_solver_io now defaults to False (behaviour change). Solver._build resolves set_names=None from the model, and to_highspy, to_gurobipy, to_mosek and to_xpress now take set_names: bool | None = None and follow the model setting.
Model.solve keeps names on when warmstart_fn or basis_fn is given, because basis files refer to names. Without this, warm-starting a direct Gurobi solve from a basis written by an LP-file solve raised GurobiError: variable name x0 doesn't exist.
Variable.get_solver_attribute (Gurobi) now maps values by position for the direct API. It used to parse labels from VarName, which returned the values of the wrong variables when names were off and a variable was masked. It also works on sliced variables.
Model._compute_infeasibilities_gurobi now reads the IISConstr attribute instead of parsing constraint names from a written .ilp file. With Gurobi's default names the old parser returned nothing, and the temporary file was never deleted.

Tests

to_highspy follows the model default and an explicit override.
get_solver_attribute with a masked and a sliced variable, for lp, direct with names, and direct without names.
Masked-constraint infeasibility for lp and direct without names.
Direct warm start from a basis written by an LP-file solve.

Benchmark (HiGHS 1.15.1)

vars=200,000 cons=200,100 nnz=598,000
before: to_highspy() 136.0 ms   to_highspy(set_names=False) 68.8 ms
after:  to_highspy()  61.7 ms   to_highspy(set_names=True)  as before

Measured breakdown of the name cost: name strings 29 ms, assigning col_names_/row_names_ 12 ms, getLp/passModel round trip 25 ms.

Checklist

AI-generated content is marked (see AGENTS.md).
Code changes are sufficiently documented; i.e. new functions contain docstrings and further explanations may be given in doc.
Unit tests for new features were added (if applicable).
A note for the release notes doc/release_notes.rst of the upcoming release is included.
I consent to the release of this PR's code under the MIT license.

) Model.set_names_in_solver_io now defaults to False; to_* and Solver builds follow it. Names are kept when warmstart_fn/basis_fn is given. Gurobi get_solver_attribute and IIS map by position for the direct API instead of parsing names.

codspeed · 2026-09-25T11:44:13Z

Merging this PR will improve performance by 50.68%

⚠️

Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 19 improved benchmarks
✅ 162 untouched benchmarks
⏩ 181 skipped benchmarks¹

Performance Changes

	Benchmark	`BASE`	`HEAD`	Efficiency
⚡	`test_to_solver[highs-rolling-severity=0]`	3.2 MB	1.6 MB	+96.19%
⚡	`test_to_solver[highs-merge_balance-severity=0]`	3.3 MB	1.8 MB	+89.39%
⚡	`test_to_solver[highs-nodal_balance-severity=0]`	3.8 MB	2.1 MB	+84.72%
⚡	`test_to_solver[highs-storage-n=250]`	37.7 MB	20.5 MB	+83.75%
⚡	`test_to_solver[highs-basic-n=250]`	54.7 MB	30.2 MB	+80.86%
⚡	`test_to_solver[highs-storage-n=10]`	1,606.4 KB	902 KB	+78.1%
⚡	`test_to_solver[highs-knapsack-n=10000]`	2.5 MB	1.4 MB	+77.83%
⚡	`test_to_solver[highs-piecewise-n=1000]`	4.1 MB	2.4 MB	+68.62%
⚡	`test_to_solver[highs-milp-n=50]`	805.4 KB	511.8 KB	+57.36%
⚡	`test_to_solver[highs-masked-n=100]`	660.6 KB	423 KB	+56.15%
⚡	`test_to_solver[highs-expression_arithmetic-n=250]`	33.9 MB	22.3 MB	+51.94%
⚡	`test_to_solver[highs-basic-n=10]`	152.6 KB	111.9 KB	+36.33%
⚡	`test_to_solver[highs-sparse_network-n=10]`	129 KB	97.5 KB	+32.41%
⚡	`test_to_solver[highs-cumsum-severity=0]`	116.7 KB	90.4 KB	+29.04%
⚡	`test_to_solver[highs-masked-n=10]`	99.1 KB	79.5 KB	+24.65%
⚡	`test_to_solver[highs-expression_arithmetic-n=10]`	118.2 KB	95.9 KB	+23.31%
⚡	`test_to_solver[highs-qp-n=1000]`	687.3 KB	588.6 KB	+16.78%
⚡	`test_to_solver[gurobi-basic-n=250]`	68.2 MB	60 MB	+13.51%
⚡	`test_to_solver[highs-qp-n=10]`	81.5 KB	72.8 KB	+11.88%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.

_{Comparing perf/978-highs-names (ab32177) with master (26e000d)}

181 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

github-actions · 2026-09-25T11:46:09Z

Build cost — v1 vs legacy

v1 build peak & time relative to legacy, on this commit — not a comparison against master (that is CodSpeed).

peak — v1 / legacy	time — v1 / legacy

Full table (time + peak, mean)

benchmarks/drivers/test_build.py::test_build[basic-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.08795 (1.09)   │   15.03 (1.00) 
 (v1)        0.08055 (1.0)   │    15.00 (1.0) 

benchmarks/drivers/test_build.py::test_build[basic-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.09701 (1.13)   │   12.04 (1.00) 
 (v1)        0.08576 (1.0)   │    12.04 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=0]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03929 (1.12)   │   15.20 (1.0) 
 (v1)        0.03516 (1.0)   │   15.20 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=100]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.05342 (1.11)   │   44.93 (1.0) 
 (v1)        0.04816 (1.0)   │   44.93 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=50]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.04116 (1.08)   │   11.51 (1.0) 
 (v1)        0.03815 (1.0)   │   11.51 (1.0) 

benchmarks/drivers/test_build.py::test_build[expression_arithmetic-n=10]
                time (s)         peak (KiB) 
 name               mean   │           mean 
────────────────────────────────────────────
 (legacy)     0.1 (1.07)   │   24.34 (1.06) 
 (v1)       0.0933 (1.0)   │    23.04 (1.0) 

benchmarks/drivers/test_build.py::test_build[expression_arithmetic-n=250]
                 time (s)         peak (MiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.1095 (1.08)   │   16.12 (1.00) 
 (v1)         0.101 (1.0)   │    16.12 (1.0) 

benchmarks/drivers/test_build.py::test_build[knapsack-n=10000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.02401 (1.09)   │   752.18 (1.10) 
 (v1)        0.02207 (1.0)   │    685.15 (1.0) 

benchmarks/drivers/test_build.py::test_build[knapsack-n=100]
                 time (s)        peak (KiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.0233 (1.07)   │   3.12 (1.33) 
 (v1)       0.02172 (1.0)   │    2.34 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=0]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.06263 (1.20)   │   126.16 (1.44) 
 (v1)        0.05207 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=100]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.06199 (1.22)   │   126.16 (1.44) 
 (v1)        0.05081 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=50]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.06216 (1.22)   │   126.16 (1.44) 
 (v1)        0.05095 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[masked-n=100]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.05466 (1.02)   │    715.12 (1.0) 
 (v1)        0.05357 (1.0)   │   787.73 (1.10) 

benchmarks/drivers/test_build.py::test_build[masked-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.05418 (1.11)   │   4.54 (1.27) 
 (v1)        0.04861 (1.0)   │    3.57 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=0]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.3678 (1.03)   │   704.12 (1.09) 
 (v1)        0.3576 (1.0)   │    643.85 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=100]
                 time (s)        peak (MiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.3864 (1.03)   │   18.34 (1.0) 
 (v1)        0.3765 (1.0)   │   18.34 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=50]
                 time (s)       peak (MiB) 
 name                mean   │         mean 
───────────────────────────────────────────
 (legacy)   0.3827 (1.03)   │   9.54 (1.0) 
 (v1)        0.3719 (1.0)   │   9.54 (1.0) 

benchmarks/drivers/test_build.py::test_build[milp-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.07416 (1.08)   │   3.77 (1.12) 
 (v1)        0.06885 (1.0)   │    3.37 (1.0) 

benchmarks/drivers/test_build.py::test_build[milp-n=50]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.07627 (1.10)   │   216.59 (1.10) 
 (v1)        0.06922 (1.0)   │    196.23 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=0]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.03778 (1.09)   │   938.49 (1.0) 
 (v1)        0.03482 (1.0)   │   938.49 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=100]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.03906 (1.09)   │   9.66 (1.0) 
 (v1)        0.03593 (1.0)   │   9.66 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=50]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.03827 (1.07)   │   5.32 (1.0) 
 (v1)        0.03564 (1.0)   │   5.32 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance_sparse-severity=0]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.01888 (1.01)   │   1.28 (1.0) 
 (v1)        0.01878 (1.0)   │   1.28 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance_sparse-severity=100]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)    0.01881 (1.0)   │   1.28 (1.0) 
 (v1)       0.01884 (1.00)   │   1.28 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance_sparse-severity=50]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.01893 (1.01)   │   1.28 (1.0) 
 (v1)        0.01871 (1.0)   │   1.28 (1.0) 

benchmarks/drivers/test_build.py::test_build[piecewise-n=1000]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.1856 (1.04)   │   946.85 (1.06) 
 (v1)        0.1793 (1.0)   │    891.54 (1.0) 

benchmarks/drivers/test_build.py::test_build[piecewise-n=10]
                time (s)         peak (KiB) 
 name               mean   │           mean 
────────────────────────────────────────────
 (legacy)   0.182 (1.04)   │   12.01 (1.00) 
 (v1)       0.1743 (1.0)   │    11.99 (1.0) 

benchmarks/drivers/test_build.py::test_build[qp-n=1000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.04708 (1.06)   │   147.70 (1.06) 
 (v1)        0.04446 (1.0)   │    139.87 (1.0) 

benchmarks/drivers/test_build.py::test_build[qp-n=10]
                 time (s)        peak (KiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.0467 (1.06)   │   2.60 (1.09) 
 (v1)       0.04391 (1.0)   │    2.38 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=0]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.03852 (1.07)   │   696.75 (1.03) 
 (v1)        0.03611 (1.0)   │    673.70 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=100]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.08367 (1.01)   │   137.97 (1.0) 
 (v1)        0.08297 (1.0)   │   137.97 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=50]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.05975 (1.09)   │   69.22 (1.0) 
 (v1)        0.05483 (1.0)   │   69.22 (1.0) 

benchmarks/drivers/test_build.py::test_build[sos-n=1000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.04693 (1.14)   │   402.33 (1.00) 
 (v1)        0.04113 (1.0)   │    402.30 (1.0) 

benchmarks/drivers/test_build.py::test_build[sos-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.04638 (1.14)   │   3.19 (1.19) 
 (v1)        0.04054 (1.0)   │    2.69 (1.0) 

benchmarks/drivers/test_build.py::test_build[sparse_network-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.05013 (1.10)   │   29.00 (1.54) 
 (v1)         0.0454 (1.0)   │    18.84 (1.0) 

benchmarks/drivers/test_build.py::test_build[sparse_network-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.06115 (1.17)   │   37.95 (1.43) 
 (v1)        0.05221 (1.0)   │    26.51 (1.0) 

benchmarks/drivers/test_build.py::test_build[storage-n=10]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.09562 (1.01)   │    410.93 (1.0) 
 (v1)        0.09454 (1.0)   │   427.84 (1.04) 

benchmarks/drivers/test_build.py::test_build[storage-n=250]
                 time (s)         peak (MiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.1019 (1.01)   │     9.94 (1.0) 
 (v1)        0.1006 (1.0)   │   10.22 (1.03)

📊 Interactive plots + CSV: download the semantics-report-v1-vs-legacy artifact from this run.

_{Report-only · not a gate · refreshed on every push · obsolete once legacy is dropped.}

FabianHofmann added 2 commits September 25, 2026 15:06

refactor(solvers): resolve set_names default in one place

e3432b7

test(io): check constraint names in test_to_solver_set_names

ab32177

FabianHofmann merged commit dbb556c into master Sep 26, 2026
24 checks passed

FabianHofmann deleted the perf/978-highs-names branch September 26, 2026 06:17

FabianHofmann mentioned this pull request Sep 26, 2026

perf(sparse): trim the stages around the CSR solver hand-off #992

Open

4 of 5 tasks

Navigation Menu

perf(solvers): stop setting names in direct solver APIs by default - #988

perf(solvers): stop setting names in direct solver APIs by default#988
FabianHofmann merged 3 commits into
masterfrom
perf/978-highs-names

FabianHofmann commented Sep 25, 2026

Uh oh!

codspeed Bot commented Sep 25, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 25, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

FabianHofmann commented Sep 25, 2026

Changes proposed in this Pull Request

Checklist

Uh oh!

codspeed Bot commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will improve performance by 50.68%

Performance Changes

Footnotes

Uh oh!

github-actions Bot commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build cost — v1 vs legacy

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

codspeed Bot commented Sep 25, 2026 •

edited

Loading

github-actions Bot commented Sep 25, 2026 •

edited

Loading