Skip to content

Regenerate requirements.txt from uv.lock - #1908

Open
VishnuR23 wants to merge 1 commit into
allenai:mainfrom
VishnuR23:vishnu/regenerate-requirements-txt
Open

VishnuR23 wants to merge 1 commit into
allenai:mainfrom
VishnuR23:vishnu/regenerate-requirements-txt

Conversation

@VishnuR23

Copy link
Copy Markdown

Problem

requirements.txt is generated from uv.lock by the export-requirements pre-commit hook. It has drifted, and the drift landed on the worst possible line:

source datasets
pyproject.toml >=4.8.4,<5
uv.lock 4.8.5
requirements.txt 4.5.0

So the file both lags the lock and pins a version pyproject.toml explicitly forbids.

4.5.0 is the exact version #1867 existed to escape. From its changelog entry:

datasets 4.5.0 does an unguarded from torchvision.io import VideoReader whenever torchvision is already imported, and torchvision 0.26 (required by torch 2.11 stacks) removed it, so every dataset.set_format("pt") raised ImportError — in one case killing an 11.5h tokenization job at its final step.

#1867 changed CHANGELOG.md, pyproject.toml and uv.lock, but not requirements.txt:

$ git show --name-only b9269782a
CHANGELOG.md
pyproject.toml
uv.lock

Anyone installing from requirements.txt therefore gets the broken version the fix was about.

Change

Regenerated with the hook's exact command from .pre-commit-config.yaml:

uv export --format requirements-txt --no-hashes --all-extras \
  --group cuda12 --no-emit-project --output-file requirements.txt

That changes exactly one line. I re-ran the export against the result and it is a no-op, so this is the file's correct generated state and there is no other drift hiding in it.

Why it went unnoticed

The hook is declared with files: ^uv\.lock$, so it fires only for contributors who have pre-commit installed locally. CI (pr_checks.yml) runs make style-check and make quality-check; neither regenerates this file. So a uv.lock change can land with requirements.txt untouched and nothing complains.

That enforcement gap affects other hooks too — main currently also violates the ban-keywords hook — so I filed it separately as #1906 rather than bundling a CI change into this fix. This PR is just the regenerated artifact.

Testing

  • Re-running the export is a no-op (idempotent).
  • The single changed line now satisfies pyproject.toml's datasets>=4.8.4,<5.

Generated file only; no files under open_instruct/, so the CHANGELOG check does not apply and no GPU code paths are touched.

GPU_TESTS=bypass

🤖 Generated with Claude Code

requirements.txt is generated from uv.lock by the export-requirements
pre-commit hook, but it drifted: allenai#1867 bumped datasets in pyproject.toml
and uv.lock without regenerating it.

The result is that requirements.txt pins the one version the bump
existed to escape. allenai#1867's own changelog entry describes datasets 4.5.0
doing an unguarded `from torchvision.io import VideoReader`, which
torchvision 0.26 removed, so every `dataset.set_format("pt")` raised
ImportError -- in one case killing an 11.5h tokenization job at its
final step.

    pyproject.toml     datasets>=4.8.4,<5
    uv.lock            4.8.5
    requirements.txt   4.5.0

So the file both lags the lock and violates the constraint pyproject
declares. Anyone installing from it gets a version the project forbids.

Regenerated with the hook's exact command:

    uv export --format requirements-txt --no-hashes --all-extras \
      --group cuda12 --no-emit-project --output-file requirements.txt

That changes exactly one line, and re-running it is a no-op, so this is
the file's correct generated state and there is no other drift.

The hook has `files: ^uv\.lock$`, so it only fires for contributors who
have pre-commit installed; CI runs `make style-check` and
`make quality-check`, neither of which regenerates this file. That gap
is why the drift went unnoticed, and is tracked separately in allenai#1906.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant