AzureOpenAiChatClient langchain reasoning models o*, gpt-5* will fail if passed in max_tokens, temperature and even api versions are problematic

[krodyrobi]

Describe the Bug

It seems for these models a breaking change happened sillently and the correct key would be max_completion_tokens

https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-openai/src/chat_models.ts#L2355
(permanent link: https://github.com/langchain-ai/langchainjs/blob/b8ffc1e613cf5a469127ed57195053f18d29b297/libs/langchain-openai/src/chat_models.ts#L2363)

Also noticed that these models seem to treat roles differently
https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-openai/src/chat_models.ts#L363
(permanent link: https://github.com/langchain-ai/langchainjs/blob/b8ffc1e613cf5a469127ed57195053f18d29b297/libs/langchain-openai/src/chat_models.ts#L366C4-L369C1)

Steps to Reproduce

n/a

Expected Behavior

max_tokens to be forwarded under the correct key

Screenshots

No response

Used Versions

• Node version via node -v: ...
• NPM version via `npm -v: ...
• SAP Cloud SDK for AI version:
  • ...
  • ...

Code Examples

No response

Log File

No response

Region

EU10

Affected Development Phase

Getting Started

Impact

No Impact

Timeline

No response

Additional Context

No response

[ZhongpinWang]

@krodyrobi Thanks for the useful information! We will discussed a potential alignment on the behaviour.

[krodyrobi]

@ZhongpinWang

It seems bigger than initially reported

• o3-mini o4-mini expect api version 2024-12-01-preview at the minimum and AzureOpenAiChatClient has no config for this: currently using 2024-10-21, so all calls fail, maybe affects other o* as well
• gpt-5 will fail if passed in a temperature other than default of 1.0, o3 seems to not care but maybe all applies to all 5*(some blog posts mention other flags as well but oficial docs don't say this even for temp..)

As for developer vs system messages it seems they are mutually exclusive in the same request and both behave the same for now as a backward compatability in the api - as specified by their docs

[ZhongpinWang]

Hi @krodyrobi ,

Thanks for adding more details on this topic. Regarding API version, the current official stable API version of Azure OpenAI is still 2024-10-21 which is quite annoying.

As a workaround, the API version can be overwritten (also in the LangChain client) as part of the custom request config.

We are aware that in LangChain OpenAI client, the max_completion_token currently cannot be set due to the lack of the mapping logic, and we probably should align with the official OpenAI LangChain as you pointed out.

For temperature.. I am not entirely sure if there is a better way, perhaps official OpenAI ignore / override this param if set? We need to investigate.

[krodyrobi]

Found the source of the unsuported params that I mentioned in azure docs

Indeed the api version can be set using withConfig but then the return types change from a model to a runnable, livable but not ideal.

Also seeing that the requestConfig type indicates that "data" can be passed in to the request body tried to pass along max_completion_tokens, unfortunatelly upon further inspection only headers and api-version are forwarded to the actual request, not sure if this typing information is inherited from the base type or if correctly set but to me it is missleading.

When checking new parameters, can you have a look over reasoning effort, completion / max / reasoning tokens as these seem to become more important as new models pop up

[florian-richter]

This is also an annoying problem in my current project. I would be very happy if we had some snippet in the documentation how to find the list of all supported params or even better filter out unsupported params for each of the supported models.

[krodyrobi]

@ZhongpinWang can you re-evaluate the priority of this ticket?

With the planned retirement of gpt-4o* planned by ai core for end of March this year and their recommendation to swap to gpt-5* (4.1 will follow shortly after) this feature feels more an more of a must (no other way to enforce token limits, or reasoning efforts)

Would definetly benefit from this as the default reasoning level is medium and through our observations it adds a 2-3 times delay to responses.

[krodyrobi]

Seems orchestration config testing tool allows passing in Records<string, any> on params and does seem to forward, looking into the sdk maybe the same thing happens there.