You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/ollama.md
+34-36Lines changed: 34 additions & 36 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -18,31 +18,31 @@ This page describes how to run OpenChat Playground (OCP) with [Ollama](https://o
18
18
19
19
## Run on local machine
20
20
21
-
1. Make sure Ollama is installed and running on your local machine. If not, install Ollama from [ollama.com](https://ollama.com/) and start the service.
21
+
1. Make sure Ollama is installed and running on your local machine. If not, install Ollama from [ollama.com](https://ollama.com/search) and start the service.
22
22
23
23
```bash
24
24
ollama serve
25
25
```
26
26
27
-
1. Pull the model you want to use. The default model OCP uses is "llama3.2"
27
+
1. Pull the model you want to use. The default model OCP uses is [llama3.2](https://ollama.com/library/llama3.2).
28
28
29
29
```bash
30
30
ollama pull llama3.2
31
31
```
32
-
33
-
Alternatively, if you want to run with a different model, say [qwen](https://ollama.com/library/qwen) other than the default one, download it first by running the following command.
32
+
33
+
Alternatively, if you want to run with a different model, say [qwen3](https://ollama.com/library/qwen3) other than the default one, download it first by running the following command.
34
34
35
35
```bash
36
-
ollama pull qwen
36
+
ollama pull qwen3
37
37
```
38
38
39
-
2. Make sure you are at the repository root.
39
+
1. Make sure you are at the repository root.
40
40
41
41
```bash
42
42
cd$REPOSITORY_ROOT
43
43
```
44
44
45
-
3. Run the app.
45
+
1. Run the app.
46
46
47
47
```bash
48
48
# bash/zsh
@@ -56,44 +56,42 @@ This page describes how to run OpenChat Playground (OCP) with [Ollama](https://o
56
56
--connector-type Ollama
57
57
```
58
58
59
-
Alternatively, if you want to run with a different model, say [qwen](https://ollama.com/library/qwen) other than the default one, download it first by running the following command.
59
+
Alternatively, if you want to run with a different model, say [qwen3](https://ollama.com/library/qwen3), make sure you've already downloaded the model by running the `ollama pull qwen3` command.
60
60
61
61
```bash
62
62
# bash/zsh
63
63
dotnet run --project $REPOSITORY_ROOT/src/OpenChat.PlaygroundApp -- \
64
64
--connector-type Ollama \
65
-
--model qwen
65
+
--model qwen3
66
66
```
67
67
68
68
```powershell
69
69
# PowerShell
70
70
dotnet run --project $REPOSITORY_ROOT\src\OpenChat.PlaygroundApp -- `
71
71
--connector-type Ollama `
72
-
--model qwen
72
+
--model qwen3
73
73
```
74
74
75
-
4. Open your web browser, navigate to `http://localhost:5280`, and enter prompts.
75
+
1. Open your web browser, navigate to `http://localhost:5280`, and enter prompts.
76
76
77
77
## Run on local container
78
78
79
-
This approach runs OpenChat Playground in a container while connecting to Ollama running on the host machine.
80
-
81
-
1. Configure Ollama to accept connections from containers.
79
+
1. Make sure the Ollama server is up and running.
82
80
83
81
```bash
84
82
ollama serve
85
83
```
86
84
87
-
1. Pull the model you want to use, and verify Ollama is accessible
85
+
1. Download the Ollama model. The default model OCP uses is [llama3.2](https://ollama.com/library/llama3.2).
Alternatively, if you want to run with a different model, say [qwen3](https://ollama.com/library/qwen3), other than the default one, download it first by running the following command.
92
+
93
+
```bash
94
+
ollama pull qwen3
97
95
```
98
96
99
97
1. Make sure you are at the repository root.
@@ -126,54 +124,54 @@ This approach runs OpenChat Playground in a container while connecting to Ollama
126
124
127
125
```bash
128
126
# bash/zsh - from GitHub Container Registry
129
-
docker run -i --rm -p 8080:8080 ghcr.io/aliencube/open-chat-playground/openchat-playground:latest\
127
+
docker run -i --rm -p 8080:8080 ghcr.io/aliencube/open-chat-playground/openchat-playground:latest\
130
128
--connector-type Ollama \
131
129
--base-url http://host.docker.internal:11434
132
130
```
133
-
131
+
134
132
```powershell
135
133
# PowerShell - from GitHub Container Registry
136
134
docker run -i --rm -p 8080:8080 ghcr.io/aliencube/open-chat-playground/openchat-playground:latest `
137
135
--connector-type Ollama `
138
136
--base-url http://host.docker.internal:11434
139
137
```
140
138
141
-
Alternatively, if you want to run with a different model, say [qwen](https://ollama.com/library/qwen), make sure you've already downloaded the model by running the `ollama pull qwen` command.
139
+
Alternatively, if you want to run with a different model, say [qwen3](https://ollama.com/library/qwen), make sure you've already downloaded the model by running the `ollama pull qwen3` command.
142
140
143
141
```bash
144
-
ollama pull qwen
142
+
ollama pull qwen3
145
143
```
146
144
147
145
```bash
148
146
# bash/zsh - from locally built container
149
147
docker run -i --rm -p 8080:8080 openchat-playground:latest \
150
148
--connector-type Ollama \
151
149
--base-url http://host.docker.internal:11434 \
152
-
--model qwen
150
+
--model qwen3
153
151
```
154
152
155
153
```powershell
156
154
# PowerShell - from locally built container
157
155
docker run -i --rm -p 8080:8080 openchat-playground:latest `
158
156
--connector-type Ollama `
159
157
--base-url http://host.docker.internal:11434 `
160
-
--model qwen
158
+
--model qwen3
161
159
```
162
160
163
161
```bash
164
162
# bash/zsh - from GitHub Container Registry
165
-
docker run -i --rm -p 8080:8080 ghcr.io/aliencube/open-chat-playground/openchat-playground:latest\
163
+
docker run -i --rm -p 8080:8080 ghcr.io/aliencube/open-chat-playground/openchat-playground:latest\
166
164
--connector-type Ollama \
167
165
--base-url http://host.docker.internal:11434 \
168
-
--model qwen
166
+
--model qwen3
169
167
```
170
-
168
+
171
169
```powershell
172
170
# PowerShell - from GitHub Container Registry
173
171
docker run -i --rm -p 8080:8080 ghcr.io/aliencube/open-chat-playground/openchat-playground:latest `
174
172
--connector-type Ollama `
175
173
--base-url http://host.docker.internal:11434 `
176
-
--model qwen
174
+
--model qwen3
177
175
```
178
176
179
177
>**NOTE**: Use `host.docker.internal:11434` to connect to Ollama running on the host machine from inside the container.
@@ -216,21 +214,21 @@ This approach runs OpenChat Playground in a container while connecting to Ollama
216
214
azd env set CONNECTOR_TYPE "Ollama"
217
215
```
218
216
219
-
The default model OCP uses is [llama3.2](https://ollama.com/library/llama3.2). If you want to run with a different model, say [qwen](https://ollama.com/library/qwen) other than the default one, add it to azd environment variables.
217
+
The default model OCP uses is [llama3.2](https://ollama.com/library/llama3.2). If you want to run with a different model, say [qwen3](https://ollama.com/library/qwen3) other than the default one, add it to azd environment variables.
220
218
221
219
```bash
222
-
azd env set OLLAMA_MODEL "qwen"
220
+
azd env set OLLAMA_MODEL "qwen3"
223
221
```
224
222
225
-
2. As a default, the app uses a Serverless GPU with NVIDIA T4 (`NC8as-T4`). If you want to use NVIDIA A100, set the GPU profile.
223
+
1. As a default, the app uses a Serverless GPU with NVIDIA T4 (`NC8as-T4`). If you want to use NVIDIA A100, set the GPU profile.
226
224
227
225
```bash
228
226
azd env set GPU_PROFILE_NAME "NC24-A100"
229
227
```
230
228
231
229
If you want to know more about Serverless GPU, visit [Using serverless GPUs in Azure Container Apps](https://learn.microsoft.com/azure/container-apps/gpu-serverless-overview#use-serverless-gpus).
232
230
233
-
3. Run the following commands in order to provision and deploy the app.
231
+
1. Run the following commands in order to provision and deploy the app.
234
232
235
233
```bash
236
234
azd up
@@ -241,9 +239,9 @@ This approach runs OpenChat Playground in a container while connecting to Ollama
241
239
242
240
Once deployed, you will be able to see the deployed OCP app URL.
243
241
244
-
4. Open your web browser, navigate to the OCP app URL, and enter prompts.
242
+
1. Open your web browser, navigate to the OCP app URL, and enter prompts.
0 commit comments