Skip to content

fix: avoid reusing request seeds across GRPO repeats - #628

Merged
pan-x-c merged 2 commits into
agentscope-ai:mainfrom
nashjojo:codex/fix-tinker-grpo-repeat-seeds
Sep 29, 2026
Merged

pan-x-c merged 2 commits into
agentscope-ai:mainfrom
nashjojo:codex/fix-tinker-grpo-repeat-seeds

Conversation

@nashjojo

Copy link
Copy Markdown
Collaborator

Description

Tinker inference currently falls back to the same configured seed for every sampling request. Repeated rollouts that start with the same prompt can therefore replay the same trajectory, removing the within-group reward variation that GRPO needs.

Leave the Tinker request seed unset unless the caller explicitly supplies one. Expose the task's repeat index to non-repeatable workflows before execution, including the offset when a group is split across runners, and use that index to give step-wise ALFWorld repeats distinct reproducible request seeds. Explicit seeds, including zero, remain supported.

This change is independent of training-loss or TuFT server changes.

Validation

  • Three new CPU regression tests passed, including six subtests: explicit/default Tinker seeds, ALFWorld repeat seeds, and repeat indices across sequential, asynchronous, and multithreaded runners.
  • Six existing runner tests passed for partial-success and fail-fast behavior.
  • Black, isort, flake8, mypy, and git diff --check passed for the changed files.
  • The CPU tests used mocked inference with CUDA disabled. The available vLLM installation lacks the vllm.inputs.MultiModalDataDict type alias imported by current main; the test process supplied that type-only alias before collecting tests. No repository code or inference implementation was replaced for this workaround.
  • No GPU training or new end-to-end experiment was run on this main-based change.

Checklist

  • Focused tests and static checks passed as described above.
  • Docstrings/comments describe the new repeat context.
  • Code is ready for review.
  • Full repository and GPU integration tests.

@nashjojo

Copy link
Copy Markdown
Collaborator Author

/unittest-module-common

@nashjojo

Copy link
Copy Markdown
Collaborator Author

/unittest-module-explorer

@github-actions

Copy link
Copy Markdown

unittest: Run #1879

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Pending ⏳ Other ❓ Flaky 🍂 Duration ⏱️
150 149 0 1 0 0 0 39m 17s

🎉 All tests passed!

Github Test Reporter by CTRF 💚

@nashjojo

Copy link
Copy Markdown
Collaborator Author

/unittest-pattern-RolloutSeedTest or test_workflow_runner_partial_success_non_repeatable or test_workflow_runner_fail_fast_without_partial_collection

@github-actions

Copy link
Copy Markdown

unittest: Run #1885

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Pending ⏳ Other ❓ Flaky 🍂 Duration ⏱️
9 9 0 0 0 0 0 17.5s

🎉 All tests passed!

Github Test Reporter by CTRF 💚

@pan-x-c
pan-x-c merged commit fc5acbb into agentscope-ai:main Sep 29, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants