Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
181 changes: 181 additions & 0 deletions docs-java/foundation-models/openai/responses.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
---
id: responses

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Major)

Please add the new file to the sidebarsDocsJava.js file so that is appears in the sidebar of the docs under Foundation Models > OpenAI.

title: Responses
hide_title: false
hide_table_of_contents: false
description: How to use the SAP AI SDK for Java to perform chat completion tasks using OpenAI models deployed on SAP AI Core.
keywords:
- sap
- cloud
- sdk
- ai
---

## Introduction

This guide demonstrates how to use the SAP AI SDK for Java to perform chat completion tasks using OpenAI models deployed on SAP AI Core.

:::warning
All classes under any of the `...model` packages are generated from an OpenAPI specification.
This means that these model classes are not guaranteed to be stable and may change with future releases.
They are safe to use, but may require updates even in minor releases.
:::

<details>
<summary id="new-user-interface-v140">New User Interface (v1.4.0)</summary>

We're excited to introduce a new user interface for OpenAI responses calls starting with **version 1.4.0**. This update is designed to improve the SAP AI SDK by:

- **Decoupling Layers:** Separating the convenience layer from the model classes to deliver a more stable and maintainable experience.
- **Staying Current:** Making it easier for the SAP AI SDK to adapt to the latest changes in the OpenAI API specification.
- **Consistent Design:** Aligning with the Orchestrator convenience API for a smoother transition and easier adoption.

**Please Note:**

- The new interface is gradually being rolled out across the SAP AI SDK.
- We welcome your feedback to help us refine this interface.
- The existing interface (v1.0.0) is deprecated but available.

</details>
Comment on lines +24 to +39

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Minor)

I don't think we need this info here anymore.


## Prerequisites

Before using the AI Core module, ensure that you have met all the general requirements outlined in the [overview](../../overview-cloud-sdk-for-ai-java#general-requirements).
Additionally, include the necessary Maven dependency in your project.

### Maven Dependencies

Add the following dependency to your `pom.xml` file:

```xml
<dependencies>
<dependency>
<groupId>com.sap.ai.sdk.foundationmodels</groupId>
<artifactId>openai</artifactId>
<version>${ai-sdk.version}</version>
</dependency>
</dependencies>
```

See [an example pom in our Spring Boot application](https://github.com/SAP/ai-sdk-java/tree/main/sample-code/spring-app/pom.xml)

## Usage

In addition to the prerequisites above, we assume you have already set up the following to carry out the examples in this guide:

<!-- prettier-ignore-start-->

- **A Deployed OpenAI Model in SAP AI Core**

- Refer
to [How to deploy a model to AI Core](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core)
for setup instructions.
- In case the model is deployed in a custom resource group, refer to [this section](#using-a-custom-resource-group).
- <details>
<summary>Example deployed model from the AI Core <code>/deployments</code> endpoint</summary>

```json
{
"id": "d123456abcdefg",
"deploymentUrl": "https://api.ai.region.aws.ml.hana.ondemand.com/v2/inference/deployments/d123456abcdefg",
"configurationId": "12345-123-123-123-123456abcdefg",
"configurationName": "gpt-4o-mini",
"scenarioId": "foundation-models",
"status": "RUNNING",
"statusMessage": null,
"targetStatus": "RUNNING",
"lastOperation": "CREATE",
"latestRunningConfigurationId": "12345-123-123-123-123456abcdefg",
"ttl": null,
"details": {
"scaling": {
"backendDetails": {}
},
"resources": {
"backendDetails": {
"model": {
"name": "gpt-4o-mini",
"version": "latest"
}
}
}
},
"createdAt": "2024-07-03T12:44:22Z",
"modifiedAt": "2024-07-16T12:44:19Z",
"submissionTime": "2024-07-03T12:44:51Z",
"startTime": "2024-07-03T12:45:56Z",
"completionTime": null
}
```

</details>

<!-- prettier-ignore-end-->

## Create a non-persistent response

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Major)

Please add a short sentence explaining what the code block does to each of the sections, like you did it for the bottom three.


```java
var responseClient = AiCoreOpenAiClient.forModel(GPT_5, "resource-group-name").responses();

var params = ResponseCreateParams.builder().input("Hi, how are you?").store(false).build(); // set store to true when creating a persistent response

var response = responseClient.create(params);
```

## Create a persistent response

```java
var responseClient = AiCoreOpenAiClient.forModel(GPT_5, "resource-group-name").responses();

var params = ResponseCreateParams.builder().input("Hi, how are you?").store(true).background(true).build(); // set background to true for the response to run asynchronously

var response = responseClient.create(params);
```

## Create a streaming non-persistent response

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Question)

Would it make sense to also add an example for streaming+persistence?


```java
var responseClient = AiCoreOpenAiClient.forModel(GPT_5, "resource-group-name").responses();

var params = ResponseCreateParams.builder().input("Hi, how are you?").model(ChatModel.GPT_5).store(false).build(); // set store to true when creating a persistent response

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Question)

Is there a reason why you use .model() here but not in the other examples? Which one has precedence in the end? I.e., if I use GPT_4 in the AiCoreOpenAiClient.forModel() and .model(ChatModel.GPT_5) here, which model is used? This should probably be noted to make it clear for users.


var response = responseClient.createStreaming(params);
```

## Retrieve a response

This allows the retrieval of a previously created persistent response by its id.

```java
var responseClient = AiCoreOpenAiClient.forModel(GPT_5, "resource-group-name").responses();

var params = ResponseRetrieveParams.builder().responseId("response-id").build();

var response = responseClient.retrieve(params);
```

## Cancel a response

This cancels the previously created background response by its id.

```java
var responseClient = AiCoreOpenAiClient.forModel(GPT_5, "resource-group-name").responses();

var params = ResponseCancelParams.builder().responseId("response-id").build();

var response = responseClient.cancel(params);
```

## Delete a response

This deletes a stored response by its id.

```java
var responseClient = AiCoreOpenAiClient.forModel(GPT_5, "resource-group-name").responses();

var params = ResponseDeleteParams.builder().responseId("response-id").build();

var response = responseClient.delete(params);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delete() does not return a value.

Suggested change
var response = responseClient.delete(params);
responseClient.delete(params);

```

Refer to [OpenAI Docs](https://developers.openai.com/api/reference/resources/responses) for OpenAI SDK API usage.