refactor: make openadapt-ml a leaf; break ml<->evals import cycle#65
Merged
Conversation
Phase 1 of the evals->ml refactor. Fork A: import the Benchmark* vocabulary (Task/Observation/Action/Agent) from openadapt-types instead of openadapt-evals. Removes the one hard, unconditional module-level ml->evals import in benchmarks/agent.py. Fork B: add a thin RolloutEnv Protocol (reset/step/observe/collect_rollout /screen_size) in training/grpo/rollout_env.py. The GRPO collector/trainer type against this interface; the concrete WAA env/adapters stay in openadapt-evals and implement it structurally. Also convert the remaining guarded module-level openadapt-evals imports (rollout_collector, verl_backend, trainer TaskConfig) to lazy in-function imports so openadapt-ml has zero module-level openadapt-evals imports. Adds openadapt-types as a dependency (>=0.2.0) with a local editable uv source until the openadapt-types release is published. Tests: tests/ (excl. integration) 451 passed, 5 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CKrVJJy5jWVCkXAqgUqtqZ
…l editable source
abrichr
marked this pull request as ready for review
July 13, 2026 16:44
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 1 of the openadapt evals→ml refactor. Makes
openadapt-mla clean model-library leaf by removing every module-level import ofopenadapt-evals.Depends on openadapt-types PR #5 (adds
openadapt_types.benchmark).Fork A — Benchmark* vocabulary from openadapt-types
The one hard, unconditional module-level
ml → evalsimport wasopenadapt_ml/benchmarks/agent.pyimportingBenchmarkAction/BenchmarkAgent/BenchmarkObservation/BenchmarkTaskfromopenadapt_evals. These types now live inopenadapt-types;agent.py(and the lazyBenchmarkActionimports intrainer.py/runner.py) import them from there.Fork B — thin RolloutEnv Protocol in openadapt-ml
New
openadapt_ml/training/grpo/rollout_env.pydefines a minimalRolloutEnvProtocol(reset/step/observe/collect_rollout/screen_size) — the surface the GRPO trainer/collector actually drive. The concrete WAA envs/adapters (RLEnvironment,WAALiveAdapter,WAADesktopEnv) stay in openadapt-evals and implement it structurally. ml depends on the interface; evals provides the implementation.Cleanup — remaining module-level evals imports made lazy
Converted the guarded module-level
openadapt-evalsimports to lazy in-function imports:rollout_collector.py— env/adapter imported inside__init__(TYPE_CHECKING for annotations)verl_backend.py—WAADesktopEnvbehind_load_waa_desktop_env()trainer.py—TaskConfiginside_load_task_configsVerification
All remaining hits are lazy (in-function), TYPE_CHECKING-only, or docstrings — no module-level
ml → evalsimport remains.import openadapt_ml.benchmarks.agentnow succeeds with openadapt-evals not installed.Tests:
pytest tests/ --ignore=tests/integration→ 451 passed, 5 skipped. (tests/integrationrequires a live openadapt-evals install and is excluded in CI, as before.)openadapt-typesrelease (addsopenadapt_types.benchmark).pyproject.tomlhere: currentlyopenadapt-types>=0.2.0with a local editable[tool.uv.sources]entry (../openadapt-types) so imports resolve pre-release. Replace the editable source with the published version once released.Pre-existing junk (left untouched)
This working tree has pre-existing untracked junk NOT part of this change and deliberately not committed:
.git-backup-*.bundle,test_artifacts/, demo.txtfiles underexperiments/waa_demo/annotated_demos/, and a self-referentialopenadapt-mlsymlink at repo root (the symlink causes recursive test collection if you run barepytestfrom the root — runpytest tests/as CI does).🤖 Generated with Claude Code
https://claude.ai/code/session_01CKrVJJy5jWVCkXAqgUqtqZ