[Question] How to configure HTTP proxy for ai-proxy plugin to access external AI models?

[huanghuang0913]

Environment：
APISIX version: 3.14
Deployment method: Running from source code (started via make run, not Docker)
OS: Linux
Plugin: ai-proxy with provider: openai-compatible

Problem Description
I'm using APISIX's ai-proxy plugin to proxy requests to Alibaba Cloud DashScope (an external AI service at https://dashscope.aliyuncs.com). However, my environment can only access the internet via an HTTP proxy.
Other routes that don't require the proxy (pointing to internal IPs like 10.x.x.x or 172.x.x.x) work fine. But when I try to access the DashScope route via ai-proxy, I get a 500 Internal Server Error.

What I've Tried
I attempted to configure the proxy by adding environment variables in conf/config.yaml like this:

nginx_config:
env_vars:
- https_proxy=http://my-proxy-ip:port
- http_proxy=http://my-proxy-ip:port
- no_proxy=127.0.0.1,localhost,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16

But this configuration doesn't seem to take effect. After restarting APISIX by apisix reload, requests to the external AI endpoint still fail with 500, and there's no evidence that the proxy is being used.

My Route Configuration:

curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT
-H "X-API-KEY: ${admin_key}"
-d '{
"id": "GTXT-Qwen3.5-flash",
"name": "qwen3.5-flash",
"uri": "/v11/chat/completions",
"methods": ["POST"],
"plugins": {
"ai-proxy": {
"provider": "openai-compatible",
"auth": {
"header": {
"Authorization": "Bearer '"$GTXT_API_KEY"'"
}
},
"override": {
"endpoint": "https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions"
},
"timeout": 6000000
},
"prometheus": {}
}
}'

Questions

1. What is the correct way to configure an HTTP/HTTPS proxy for the ai-proxy plugin when APISIX is running from source code (not in Docker)?
2. Does the nginx_config.env_vars approach in config.yaml actually work for this purpose? If not, what is the recommended alternative?
3. Should I set system environment variables (HTTP_PROXY, HTTPS_PROXY, NO_PROXY) before starting APISIX? If so, how can I verify that they are being picked up by the APISIX worker processes?
4. Is there any plugin-specific proxy configuration within ai-proxy that I'm missing? The documentation doesn't seem to mention proxy support.

Additional Context
• Other internal routes (without proxy) work correctly, so APISIX core is functioning.
• I can confirm that the proxy server itself is working (e.g., curl -x my-proxy:port https://dashscope.aliyuncs.com succeeds from the same machine).
Any guidance would be greatly appreciated. Thanks!

[dosubot[bot]]

Unfortunately, the ai-proxy plugin currently does not support HTTP/HTTPS proxy configuration for its outbound requests. This is a confirmed limitation, not a misconfiguration on your side.

Why your attempts didn't work:

• The nginx_config.env_vars approach won't help here because the ai-proxy plugin uses lua-resty-http to make outbound requests via cosockets, and this library does not automatically read http_proxy/https_proxy environment variables [1]. Proxy support requires an explicit httpc:set_proxy_options() call, which the ai-proxy plugin never makes [2].
• Other APISIX plugins like openid-connect and authz-keycloak do have explicit proxy support via a proxy_opts schema field [3] [4], but this pattern was not implemented in ai-proxy.

Workarounds for your situation:

1. OS-level transparent proxy (recommended): Use redsocks + iptables to transparently redirect outbound traffic from the APISIX process through your HTTP proxy. This doesn't require any APISIX configuration changes.
2. Network-level routing: Set up a local forward proxy (e.g., Tinyproxy, Squid) as a transparent proxy on the machine, routing traffic destined for external IPs through your corporate proxy.
3. VPN or direct connectivity: If possible, establish a network route that allows the APISIX host to reach dashscope.aliyuncs.com directly.

This is a known gap that has been discussed in the community [5]. If this feature is important for your use case, consider opening a feature request issue to add proxy_opts support to the ai-proxy plugin — the underlying lua-resty-http library already supports it; the plugin just needs to expose and wire up the configuration.

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.

[jmrplens]

The "not supported" part is right, but there is a way through, and it is worth knowing why the environment variables did nothing before reaching for it.

APISIX makes these outbound calls with a cosocket HTTP client, and cosockets do not go through the OS networking layer, so they never read http_proxy or https_proxy from the environment. A proxy has to be set on the client object explicitly. Across the whole repository set_proxy_options appears exactly once, in apisix/plugins/authz-keycloak.lua, and proxy_opts is exposed only by openid-connect and authz-keycloak. So ai-proxy has no proxy option to configure, and putting the variables in nginx_config.env_vars was never going to reach the code that matters.

What ai-proxy does have is override.endpoint, and its own schema describes the case you are in: "Override the endpoint of the AI Instance. Typically used for custom hosts (e.g., AWS PrivateLink, reverse proxies)." Point it at something you can reach directly, and let that thing carry the request out through your corporate proxy.

I checked that this actually redirects the outbound call rather than assuming it. On a standalone APISIX with a route like:

plugins:
  ai-proxy:
    provider: openai-compatible
    auth:
      header:
        Authorization: "Bearer sk-..."
    options:
      model: qwen-test
    override:
      endpoint: "http://relay:8080/v1/chat/completions"

the request left APISIX to that address instead of the provider, and the receiving side logged path=/v1/chat/completions host=relay:8080. The reply came back to the client normally.

So the shape that works for DashScope behind an HTTP proxy is a small relay next to APISIX, an nginx or similar with proxy_pass to https://dashscope.aliyuncs.com, which itself has the proxy configured the ordinary way, and override.endpoint pointing at that relay. The relay speaks to your proxy through normal OS networking, so https_proxy works there.

Note the endpoint accepts either scheme plus host, in which case the plugin computes the provider path itself, or a full URL with path and query, which it then uses as given. If your relay expects the provider's own path, the first form keeps the config shorter.