Conversation
…template
HFSystemFormat failed on every instance with the generic transformers
ValueError when the model's tokenizer does not define a chat template.
Check this once in prepare() and raise a UnitxtError naming the model and
how to proceed: use a model with a chat template, pass one explicitly via
chat_kwargs_dict={'chat_template': ...}, or use SystemFormat.
Signed-off-by: Waleed Khalid <wak327@gmail.com>
Author
|
@elronbandel @yoavkatz could you take a look when you have a moment? Small change: |
yoavkatz
approved these changes
Oct 5, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
HFSystemFormatrenders prompts withtokenizer.apply_chat_template. When the model's tokenizer has nochat_template(e.g.ibm-granite/granite-20b-code-basefrom #1144, orgpt2), every instance fails while the dataset is being generated with transformers' generic error:That message doesn't name the model, and it doesn't mention the options unitxt already supports.
Changes
HFSystemFormat.prepare()checks the tokenizer right after loading it. If there is no chat template and none was passed throughchat_kwargs_dict, it raises aUnitxtErroronce, when the format is created:The
HFSystemFormatdocstring now mentions passing a template throughchat_kwargs_dict={"chat_template": ...}. That already works today, sincechat_kwargs_dictis forwarded toapply_chat_template.UnitxtErrorisn't in the retry list ofretry_connection_with_exponential_backoff, so the new check is not retried.Fixes #1144
Test plan
test_hf_system_format_without_chat_template: creating the format withopenai-community/gpt2raises theUnitxtError, and passing an explicitchat_templateinchat_kwargs_dictformats correctly. Fails onmain(UnitxtError not raised), passes here.test_hf_system_format(zephyr) still passes.pre-commit runon the changed files.