Skip to content

feat: add extra_body parameter to LLM for provider-specific options#130

Merged
Moskize91 merged 1 commit into
oomol-lab:mainfrom
xujialiu:feat/extra-body-support
Feb 28, 2026
Merged

feat: add extra_body parameter to LLM for provider-specific options#130
Moskize91 merged 1 commit into
oomol-lab:mainfrom
xujialiu:feat/extra-body-support

Conversation

@xujialiu

Copy link
Copy Markdown
Contributor

Summary

  • Adds extra_body parameter to LLM and LLMExecutor, passed through to client.chat.completions.create()
  • This enables users to send provider-specific parameters (e.g. {"enable_thinking": False} for Zhipu GLM-4.7)

Problem

Zhipu's GLM-4.7 is a thinking/reasoning model that enables reasoning mode by default. This causes two issues:

  1. The model spends ~10x more tokens on internal reasoning (reasoning_content) before producing actual content (content), wasting API credits
  2. If the token budget is exhausted during reasoning, delta.content can be empty, causing translation failures

Solution

Pass-through extra_body dict to the OpenAI client so users can disable thinking mode:

llm = LLM(
    key="your-key",
    url="https://open.bigmodel.cn/api/paas/v4",
    model="glm-4.7",
    token_encoding="o200k_base",
    extra_body={"enable_thinking": False},
)

This is a generic solution that works for any provider-specific parameter, not just Zhipu.

Changes

  • epub_translator/llm/core.py: Add extra_body parameter to LLM.__init__
  • epub_translator/llm/executor.py: Add extra_body to LLMExecutor.__init__ and pass to chat.completions.create()

Zhipu GLM-4.7 enables thinking/reasoning mode by default, which wastes
~10x tokens on internal reasoning before producing actual content. The
library only reads delta.content from streaming chunks, so when the
token budget is exhausted during reasoning, the response can be empty.

Adding extra_body pass-through to the OpenAI client allows users to
send provider-specific parameters like {"enable_thinking": false} to
disable reasoning mode and get direct responses.
@coderabbitai

coderabbitai Bot commented Feb 27, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 67b87d6 and 17533fa.

📒 Files selected for processing (2)
  • epub_translator/llm/core.py
  • epub_translator/llm/executor.py

Summary by CodeRabbit

Release Notes

  • New Features
    • Added extensible configuration support for language models. Users can now pass custom parameters during initialization to fine-tune model behavior, unlock advanced features, and achieve greater control and flexibility in language model configuration. This enhancement enables more sophisticated use cases and better adaptation to specific application requirements.

Walkthrough

This pull request introduces an optional extra_body parameter to the LLM request pipeline. The parameter is added to the LLM class constructor, forwarded to the LLMExecutor constructor, stored as an instance attribute, and passed through to the underlying OpenAI chat completions API call. The change involves no modifications to error handling, control flow, or core logic.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~4 minutes

🚥 Pre-merge checks | ✅ 2
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title follows the required format 'feat(): ' and accurately describes the main change—adding an extra_body parameter to LLM for provider-specific options.
Description check ✅ Passed The description is well-detailed and directly related to the changeset, explaining the problem, solution, and example usage for the extra_body parameter addition.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

Tip

Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs).
Share your feedback on Discord.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@Moskize91 Moskize91 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Moskize91 Moskize91 merged commit 1acea3d into oomol-lab:main Feb 28, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants