Remote Ollama Server Not Respected (Still Using localhost)

[Bernic777]

Hello,

I am currently trying to use this project with an Ollama backend, but I am running into limitations due to local resource constraints.

Context

I am unable to run Ollama locally because of limited CPU/RAM, so I am attempting to use a remote Ollama server hosted on a VPS.

What I Have Tried

1. Modifying settings.json

I changed the endpoint from the default:

http://localhost:11434

to:

http://<VPS_IP>:11434

However, requests still appear to be directed to localhost, suggesting the endpoint may be hardcoded or internally overridden.

2. Port Forwarding / Redirect (Windows)

I used netsh interface portproxy to forward traffic:

localhost:11434 → <VPS_IP>:11434

Command used:

netsh interface portproxy add v4tov4 listenport=11434 listenaddress=127.0.0.1 connectport=11434 connectaddress=<VPS_IP>

Result:

• Some requests are successfully forwarded
• However, behavior is inconsistent, as if certain calls bypass the proxy or use a different binding

3. Ollama Binding on VPS

I verified that Ollama is bound to:

0.0.0.0:11434

And confirmed accessibility:

curl http://<VPS_IP>:11434/api/tags

Main Issue

The application seems to:

• Still rely on localhost internally
• Not fully respect the configured endpoint

Possible causes:

• Hardcoded localhost usage
• Internal client override
• Multiple configuration sources (e.g., environment variables vs JSON)

Questions

1. Is it currently possible to fully use a remote Ollama server with this project?
2. Is there an official way to override the endpoint (e.g., via environment variables, CLI flags, or config priority)?
3. Are there any parts of the codebase that are still hardcoded to localhost?

Suggestion

It would be very helpful if:

• The Ollama endpoint could be explicitly configurable (single source of truth)
• Remote inference support was more stable for low-resource environments

Thank you. and may God Bless You

[Kuberwastaken]

@Bernic777 Sadly I don't have access to an ollama instance, my machine can't really handle it
But I'll look into it and update you!

[sovereign313]

as a work-around until this is fixed, you could run a local proxy on 127.0.0.1:11434 that proxies through to <VPS_IP>:11434

[Bernic777]

Hello,

Just to share an update from my side — I previously experimented with a simple local proxy setup, and it worked successfully.

By routing traffic from:

127.0.0.1:11434 → <VPS_IP>:11434

I was able to:

• Forward requests to the remote Ollama server without issues
• Bypass the localhost limitation in the application
• Use the remote instance despite my local resource constraints

So from my testing, this approach is confirmed to work as a temporary solution.

Thanks.

[Kuberwastaken]

pushed a potential fix, would love for you you to test @Bernic777

[jalfarocode]

I might have found a possible cause.

It looks like there is a runtime path for local providers that constructs ollama() directly, which reads OLLAMA_HOST or defaults to http://localhost:11434, instead of consistently using the effective config from settings.json.

This could explain why providers.ollama.api_base is not being respected in some cases.

I’ll try to reproduce it locally and verify if this is indeed the issue. If so, I can work on a fix to ensure Ollama always uses the configured api_base, and add a regression test for remote hosts.

Let me know if this aligns with your understanding

[goodneamtakenbydogs]

Hi! I opened a draft PR for one remote Ollama configuration path: #170

It makes the native Ollama endpoints used for model discovery and health checks follow the configured providers.ollama.api_base, instead of keeping the original OLLAMA_HOST/localhost value. It also adds a regression test and clarifies the docs for remote Ollama hosts.

[Kuberwastaken]

Fixed in #192.

The earlier "potential fix" was not actually merged — the bug was still
present on main. There were two gaps:

1. ollama() read only the OLLAMA_HOST env var and never looked at
   providers.ollama.api_base in settings, so the runtime provider path
   ignored a configured remote host entirely.
2. with_base_url() updated only the chat-completions base URL and left
   ollama_native_host at localhost, so health checks and native model
   discovery (/api/tags) kept hitting localhost even when the base URL
   was overridden.

The fix resolves the host from settings first, then OLLAMA_HOST, then
the localhost default, and keeps the native host in sync with any base-URL
override. Added a unit test; cargo test -p claurst-api passes.

Note: full end-to-end confirmation needs a live remote Ollama server —
this was verified by code inspection and unit tests. Closing as resolved
by #192.