You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
You can send a single chat completion request with multiple images by defining multiple content blocks with `image_url` type in the template.
763
763
The model will process each image and use the information from all of them to respond.
764
764
765
+
### Prompt Caching
766
+
767
+
:::note
768
+
This section covers explicit cache control for AWS Bedrock-hosted models (Anthropic Claude and Amazon Nova).
769
+
Other model providers, such as OpenAI and Gemini, use implicit context caching that is enabled by default and requires no additional configuration.
770
+
For a full overview, refer to the [Prompt Caching documentation](https://help.sap.com/docs/sap-ai-core/generative-ai/prompt-caching?locale=en-US).
771
+
:::
772
+
773
+
Cache control improves performance by caching requests at the model provider level.
774
+
Add a `cache_control` breakpoint to a content block to mark a cache breakpoint in the prompt.
775
+
Availability of different time-to-live (TTL) options may depend on the model, refer to the [orchestration prompt caching documentation](https://help.sap.com/docs/sap-ai-core/generative-ai/prompt-caching?locale=en-US) for details.
776
+
777
+
:::note
778
+
Prompt caching requires a minimum number of tokens in the cached content.
779
+
If the prompt is too short, the cache breakpoint may not take effect.
780
+
:::
781
+
782
+
Set the `cache_control` property as part of message contents.
783
+
Some models can also apply `cache_control` to tool calls.
784
+
Use the `getTokenUsage()` method to inspect cache activity.
785
+
For supported models, the `prompt_tokens_details` property on the tokens usage object will contain the amount of tokens read and written to the cache.
0 commit comments