Constrain Python 3.12 dependency updates

[joelaforet]

This is a small follow-up to PR #58 based on dependency solver debugging for Python 3.12.

Findings:

• The source-level Python 3.12 issue is the private importlib.resources._common usage in pkginspect.py; the existing PR fix using ModuleSpec.submodule_search_locations looks correct.
• Current conda-forge can solve python=3.12 with mbuild + openff-toolkit, and with mbuild-base + openff-toolkit-base. That pair does not appear to be the current hard blocker.
• The broader test/optional stack has an opposing NumPy constraint: current mbuild pulls gmso>=0.16.1, which requires numpy>=2.0, while espaloma_charge pulls dgl -> tensorflow-base=2.17.0, whose Python 3.12 build requires numpy>=1.26.4,<2.0a0.
• The saved sequential-install logs in polymerist_py312_context were pinned to python 3.13.*, not 3.12. For this PR, opening requires-python to unbounded >=3.11 makes the solver test future Python versions that are outside the intended compatibility target.

Changes here:

• Cap Polymerist support to >=3.11,<3.13 for this pass.
• Remove the pyproject numpy>=2 runtime requirement so Polymerist does not force one side of the mbuild/GMSO vs Espaloma/TensorFlow conflict.
• Restore setuptools<82 and the unrelated packmol/lammps pins instead of changing them in the Python 3.12 compatibility PR.

Validation run locally:

• mamba dry-run solve for the updated Python 3.12 runtime dependency set succeeded.
• mamba dry-run python=3.12 mbuild openff-toolkit succeeded.
• Focused test: mamba run -n polymerist-env python -m pytest -q polymerist/tests/genutils/importutils/test_pkginspect.py -> 44 passed, 9 xfailed.

[joelaforet]

What are your thoughts on deprecating support for Espaloma as a charge model? It appears many of the Python 3.12 env problems are coming from incompatible NumPy/TensorFlow constraints associated with this charger.

Specifically, current mbuild pulls gmso>=0.16, which requires numpy>=2, while espaloma_charge pulls dgl -> tensorflow-base=2.17, which requires numpy<2 on Python 3.12. mbuild + openff-toolkit itself seems to solve fine without that broader Espaloma stack.

From my understanding, we want people using OpenFF NAGL/AshGC now anyway. Maybe the cleanest path is to remove espaloma_charge from the default/test envs, keep EspalomaCharger temporarily as deprecated/backward-compatible, and switch examples/docs toward NAGL.

[joelaforet]

Note, this can be closed should a PR be opened to address #60 . These are the env files I used to get 3.12 working though for reference.

[timbernat]

Cap Polymerist support to >=3.11,<3.13 for this pass.

Lower cap makes sense, but why the upper cap at 3.13? Which tests/deps fail without it?

Restore setuptools<82 and the unrelated packmol/lammps pins instead of changing them in the Python 3.12 compatibility PR.

Regrettable these are still needed (was hoping to ditch all upper version pins while rooting around during these updates), but I can confirm polymerist/tests/mdtools/lammpstools/test_lammpseval.py does fail with the same missing Shared Object file Exception as before that pin was added; will add this back on #58 as well.

From my understanding, we want people using OpenFF NAGL/AshGC now anyway. Maybe the cleanest path is to remove espaloma_charge from the default/test envs, keep EspalomaCharger temporarily as deprecated/backward-compatible, and switch examples/docs toward NAGL.

That sounds like a reasonable path forward, as you pointed out in more detail in #60

These are the env files I used to get 3.12 working though for reference.

I don't see any attachments, did you forget to include them? At any rate, apart from verifying that the shipped conda envs YAMLs install, I also want to try to follow a mock pip-install as detailed in this comment, as that will be most useful as a pre-check that a PyPI build will be installable.

[timbernat]

As a semi-related small test fix, I noticed the mdtools/openmmtools/test_schedule.py tests failed due to numpy apparently deprecating ndarray.ptp in 2.0 (now must call as np.ptp(arr) instead. I patched that which should increase test coverage in a backwards-compatible way.

[joelaforet]

Thanks, that makes sense. I reran the validation with the newer mBuild context from Chris, and I think we now have a clearer path.

On >=3.11,<3.13: Python 3.13 currently has a few blockers:

• python=3.13 openbabel does not solve on conda-forge. The available openbabel builds top out at Python 3.12.
• pip install -e .[interactive] fails on Python 3.13 because nglview==3.0.6 builds through jupyter_packaging, which imports the removed stdlib module pipes.
• A reduced Python 3.13 env without OpenBabel can install core Polymerist and openff-toolkit/mbuild, but some tests expose a source-level registry issue: subclass_registry resolves as a property/method instead of a mapping. That affects MolCharger.subclass_registry.items() and LAMMPSUnitStyle.subclass_registry[...], likely due to the @classmethod @property pattern in register_subclasses.
  So I’d keep requires-python = ">=3.11,<3.13" for Python 3.12 compatibility #58/Constrain Python 3.12 dependency updates #59 and treat 3.13 as a follow-up.
  For mBuild, Chris confirmed the mbuild.plugins failure is from mbuild=0.16.4 under Python 3.12+. That conda package should have had a Python cap but did not. I validated that using mbuild>=0.17.1 avoids the import failure; the solver currently selects mbuild=1.3.1.

Validation I ran on py312-updates:

• Fresh env: python=3.12
• pip install -e .[interactive] passed
• mamba install openff-toolkit "mbuild>=0.17.1" openbabel pytest passed
• mBuild smoke test passed:
  • import mbuild
  • from mbuild.lib.recipes.polymer import Polymer
• Full suite: 722 passed, 31 skipped, 54 xfailed
  Focused tests:
• polymerist/tests/genutils/importutils/test_pkginspect.py: 52 passed, 9 xfailed
• polymerist/tests/polymers/building/test_linear.py and test_sequencing.py: 133 passed, 10 xfailed
• polymerist/tests/mdtools/openmmtools/test_schedule.py: 3 passed
• polymerist/tests/mdtools/lammpstools/test_lammpseval.py: 3 passed
• polymerist/tests/mdtools/openfftools/partialcharge/test_molchargers.py: 8 passed

Recommended changes for #58:

• Add mbuild>=0.17.1 anywhere mbuild appears in shipped envs.
• Keep requires-python = ">=3.11,<3.13" for this pass.
• Keep EspalomaCharger guarded in source, but remove espaloma_charge from default/test/docs envs and install docs.
• Restore/retain the known pins:
  • setuptools<82
  • packmol<=20.15.1
  • lammps<=2024.08.29

I’m fine closing #59 if you’d rather roll the useful pieces directly into #58.

[timbernat]

Chris confirmed the mbuild.plugins failure is from mbuild=0.16.4 under Python 3.12+... I validated that using mbuild>=0.17.1 avoids the import failure; the solver currently selects mbuild=1.3.1.

Was able to confirm as well, looks like pinning mbuild>=1.0 along with the setuptools<82 re-pin seems to be enough to fix the mbuild import issues

I’m fine closing #59 if you’d rather roll the useful pieces directly into #58.

That's the plan, though let's wait to close #59 until #58 is merged. I want to make sure I've resolved any pending concerns before closing anything.

python=3.13 openbabel does not solve on conda-forge. The available openbabel builds top out at Python 3.12.

The conda solver should be smart enough to figure 3.12 is the only workable version without the upper pin, as I've confirmed with the local pip-like install detailed #60 (which included openbabel) and in the updated env YAMLs, which now pass CI for 3.12 in #58 without the explicit pin.

The main reason I'm concerned about it is to avoid a similar process to this update in a year and a half when Python 3.13 hits end-of-life. Ideally the deps mentioned have updated their compat before then, and the lack of upper pins means the install here *should* adapt accordingly if they have, though of course the devil is in the details. Returning to the present, if you can confirm that the updated envs solve and pass tests as the most recent CI indicates, then all that remains is for me to update docs and examples per #60 and the 3.12-compat release in #58 should be ready to roll out!

[joelaforet]

I re-tested latest py312-updates at e90c526 in a fresh Python 3.12 env using your updated envs.

Results:

• pip install -e .[interactive] passed.
• mamba install openff-toolkit mbuild openbabel pytest passed.
• Conda selected python 3.12.13, setuptools 81.0.0, numpy 2.2.6, mbuild 1.3.1.
• from mbuild.lib.recipes.polymer import Polymer works.
• Shipped env dry-runs solve for test-env.yml, docs-env.yml, and release-build.yml.
• Focused tests all passed.
• Full suite passed: 722 passed, 31 skipped, 54 xfailed in 24.67s.

So #58 looks good for Python 3.12 now. The mbuild>=1.0 change avoids the broken mbuild=0.16.4 path, setuptools<82 is respected, and removing espaloma_charge from the default env/install path does not break the suite.

Thanks for your help with this!

[timbernat]

So #58 looks good for Python 3.12 now.

Great! I'll update docs and examples then and do a dummy release to see if pip install works from the TestPyPI server. If it does, I'll put out an official release which you ought to be able to install into any extant envs the clear up the <3.11 blockage; I'll poke later today when that's ready to install.

Thanks for your help with this!

Of course, thanks for pointing out the spots to look, helped with debug!

[timbernat]

Alright @joelaforet , I've removed the references to Espaloma-charge in the tutorials in polymerist_examples #2, updated the install instructions + links in the docs in #61, and drafted release notes for 1.1.0; everything should be ready for deployment. Poking to get another set of eyes and see if you spot anything last-minute which looks off-color, but if not I'll push the updated version to PyPI and close all the requisite tabs (viz. Issues #57 and #60, and PRs #58, #59, and #61)

[timbernat]

Closed with 1.1.0 release