Use lm-eval 0.4.11 to solve issue with transformers v5

[dhuangnm]

Try bumping up lm-eval version to 0.4.11 to solve following error with transformers v5:

.venv/lib64/python3.12/site-packages/lm_eval/models/hf_vlms.py:37: in HFMultimodalLM
    AUTO_MODEL_CLASS = transformers.AutoModelForVision2Seq
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module transformers has no attribute AutoModelForVision2Seq

[coderabbitai]

📝 Walkthrough

Walkthrough

Updated lm-eval dependency versions in pyproject.toml: main dependencies pinned lm-eval[api] from 0.4.9.2 to 0.4.11, and leaderboard optional dependencies raised lm-eval[ifeval,math,sentencepiece] minimum from 0.4.8 to 0.4.11.

Changes

Dependency version updates

Layer / File(s)	Summary
lm-eval version bumps pyproject.toml	lm-eval[api] pinned to 0.4.11 in main dependencies, and lm-eval[ifeval,math,sentencepiece] minimum raised to 0.4.11 in the leaderboard optional group.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Possibly related PRs

• openshift-psap/llm-eval-test#43: Both PRs update pyproject.toml's lm-eval[api] version constraint across different minor versions.
• openshift-psap/llm-eval-test#44: This PR bumps lm-eval[api] to 0.4.9.2; the current PR continues the version progression to 0.4.11.

Suggested reviewers

• sjmonson
• tarukumar

Poem

🐰 A hop, a bump, a version tale,
From four-point-nine, we now prevail!
lm-eval bounces to 0.4.11,
Leaderboard metrics reach new heaven! 📊

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarizes the main change: bumping lm-eval to version 0.4.11 to resolve a transformers v5 compatibility issue.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}