Skip to content

fix: Raise a clear error when HFSystemFormat's tokenizer has no chat template - #1983

Open
wak327 wants to merge 4 commits into
IBM:mainfrom
wak327:fix/hf-system-format-missing-chat-template
Open

wak327 wants to merge 4 commits into
IBM:mainfrom
wak327:fix/hf-system-format-missing-chat-template

Conversation

@wak327

@wak327 wak327 commented Oct 3, 2026

Copy link
Copy Markdown

Summary

HFSystemFormat renders prompts with tokenizer.apply_chat_template. When the model's tokenizer has no chat_template (e.g. ibm-granite/granite-20b-code-base from #1144, or gpt2), every instance fails while the dataset is being generated with transformers' generic error:

ValueError: Cannot use chat template functions because tokenizer.chat_template is not set and no template argument was passed! ...

That message doesn't name the model, and it doesn't mention the options unitxt already supports.

Changes

  • HFSystemFormat.prepare() checks the tokenizer right after loading it. If there is no chat template and none was passed through chat_kwargs_dict, it raises a UnitxtError once, when the format is created:

    HFSystemFormat cannot be used with model 'openai-community/gpt2' because its tokenizer does not define a chat template (no 'chat_template' in its tokenizer_config.json). Either use a model whose tokenizer has a chat template, pass a Jinja chat template explicitly with chat_kwargs_dict={'chat_template': '<template>'}, or use a format that does not depend on the tokenizer, such as SystemFormat.
    
  • The HFSystemFormat docstring now mentions passing a template through chat_kwargs_dict={"chat_template": ...}. That already works today, since chat_kwargs_dict is forwarded to apply_chat_template.

  • UnitxtError isn't in the retry list of retry_connection_with_exponential_backoff, so the new check is not retried.

Fixes #1144

Test plan

  • New test_hf_system_format_without_chat_template: creating the format with openai-community/gpt2 raises the UnitxtError, and passing an explicit chat_template in chat_kwargs_dict formats correctly. Fails on main (UnitxtError not raised), passes here.
  • Existing test_hf_system_format (zephyr) still passes.
  • pre-commit run on the changed files.

…template

HFSystemFormat failed on every instance with the generic transformers
ValueError when the model's tokenizer does not define a chat template.
Check this once in prepare() and raise a UnitxtError naming the model and
how to proceed: use a model with a chat template, pass one explicitly via
chat_kwargs_dict={'chat_template': ...}, or use SystemFormat.

Signed-off-by: Waleed Khalid <wak327@gmail.com>
@wak327

wak327 commented Oct 3, 2026

Copy link
Copy Markdown
Author

@elronbandel @yoavkatz could you take a look when you have a moment? Small change: HFSystemFormat now fails early with a clear message when the model's tokenizer has no chat template (#1144), instead of failing on every instance with the generic transformers error.

@wak327 wak327 mentioned this pull request Oct 3, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HFSystemFormat Exception

2 participants