Checklist
Describe the Problem
Hello,
It's a follow up of #1453.
SAP AI Core orchestration API is currently enforcing prompt templating parsing rules on toolcalls, and it's not compliant with OpenAI specs. That's the root cause of the issue previously reported and redirected to LLM API team.
The targeted use case is: AI Coding agents following strictly the OpenAI specs (like Cline, Claude Code, OpenCode, ...) using models with specific knowledge (RAG, like SAP tech stack, SAP development policies, etc.) through the orchestration API.
Propose a Solution
Since the prompt templating rules will not change, the following suggestions were made:
- add escaping for specific prompt templating syntax at toolcall results handling. If it works, an option could be added to the SAP AI SDK orchestration.
- move toolcall results to
messages_history to bypass the prompt templating rules. The issue here is the semantic change: how model will interpret that move? The model request a toolcall, the model expect a toolcall answer.
In the vercel ai provider I'm developing as a building block to implement the previously described use case, I will try the escaping approach or transform on toolcall results to avoid triggering the prompt templating parsing on pure content. I think the SAP AI SDK should natively offer such options.
Describe Alternatives
No response
Affected Development Phase
Getting Started
Impact
Blocked
Timeline
No response
Additional Context
No response
Checklist
Describe the Problem
Hello,
It's a follow up of #1453.
SAP AI Core orchestration API is currently enforcing prompt templating parsing rules on toolcalls, and it's not compliant with OpenAI specs. That's the root cause of the issue previously reported and redirected to LLM API team.
The targeted use case is: AI Coding agents following strictly the OpenAI specs (like Cline, Claude Code, OpenCode, ...) using models with specific knowledge (RAG, like SAP tech stack, SAP development policies, etc.) through the orchestration API.
Propose a Solution
Since the prompt templating rules will not change, the following suggestions were made:
messages_historyto bypass the prompt templating rules. The issue here is the semantic change: how model will interpret that move? The model request a toolcall, the model expect a toolcall answer.In the vercel ai provider I'm developing as a building block to implement the previously described use case, I will try the escaping approach or transform on toolcall results to avoid triggering the prompt templating parsing on pure content. I think the SAP AI SDK should natively offer such options.
Describe Alternatives
No response
Affected Development Phase
Getting Started
Impact
Blocked
Timeline
No response
Additional Context
No response