Conversation
requirements.txt is generated from uv.lock by the export-requirements pre-commit hook, but it drifted: allenai#1867 bumped datasets in pyproject.toml and uv.lock without regenerating it. The result is that requirements.txt pins the one version the bump existed to escape. allenai#1867's own changelog entry describes datasets 4.5.0 doing an unguarded `from torchvision.io import VideoReader`, which torchvision 0.26 removed, so every `dataset.set_format("pt")` raised ImportError -- in one case killing an 11.5h tokenization job at its final step. pyproject.toml datasets>=4.8.4,<5 uv.lock 4.8.5 requirements.txt 4.5.0 So the file both lags the lock and violates the constraint pyproject declares. Anyone installing from it gets a version the project forbids. Regenerated with the hook's exact command: uv export --format requirements-txt --no-hashes --all-extras \ --group cuda12 --no-emit-project --output-file requirements.txt That changes exactly one line, and re-running it is a no-op, so this is the file's correct generated state and there is no other drift. The hook has `files: ^uv\.lock$`, so it only fires for contributors who have pre-commit installed; CI runs `make style-check` and `make quality-check`, neither of which regenerates this file. That gap is why the drift went unnoticed, and is tracked separately in allenai#1906. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
requirements.txtis generated fromuv.lockby theexport-requirementspre-commit hook. It has drifted, and the drift landed on the worst possible line:pyproject.toml>=4.8.4,<5uv.lock4.8.5requirements.txt4.5.0So the file both lags the lock and pins a version
pyproject.tomlexplicitly forbids.4.5.0 is the exact version #1867 existed to escape. From its changelog entry:
#1867 changed
CHANGELOG.md,pyproject.tomlanduv.lock, but notrequirements.txt:Anyone installing from
requirements.txttherefore gets the broken version the fix was about.Change
Regenerated with the hook's exact command from
.pre-commit-config.yaml:uv export --format requirements-txt --no-hashes --all-extras \ --group cuda12 --no-emit-project --output-file requirements.txtThat changes exactly one line. I re-ran the export against the result and it is a no-op, so this is the file's correct generated state and there is no other drift hiding in it.
Why it went unnoticed
The hook is declared with
files: ^uv\.lock$, so it fires only for contributors who have pre-commit installed locally. CI (pr_checks.yml) runsmake style-checkandmake quality-check; neither regenerates this file. So auv.lockchange can land withrequirements.txtuntouched and nothing complains.That enforcement gap affects other hooks too —
maincurrently also violates theban-keywordshook — so I filed it separately as #1906 rather than bundling a CI change into this fix. This PR is just the regenerated artifact.Testing
pyproject.toml'sdatasets>=4.8.4,<5.Generated file only; no files under
open_instruct/, so the CHANGELOG check does not apply and no GPU code paths are touched.GPU_TESTS=bypass
🤖 Generated with Claude Code