Thanks for poking at this. The project values a small, readable surface over feature sprawl.
conda create -y -n selfhosted-chat-api python=3.12
conda activate selfhosted-chat-api
cd api && pip install -r requirements-dev.txtmake test # pytest
make lint # ruff
make dev # uvicorn with --reload (points at BACKEND_BASE_URL)See docs/BACKENDS.md. Edits needed:
api/app/backends.py— capability flags + display namedocker-compose.yml— service under a new compose profiledeploy/env/<name>.env— templateapi/tests/test_backends.py— coverage
No route changes required.
make lintandmake testboth green- Public behaviour changes reflected in
README.mdor a file underdocs/ - New envs documented in
.env.example - No secrets, no committed
.env
- The gateway is a thin, OpenAI-shaped adapter. Resist the urge to add business logic. If a feature is model- or backend-specific, let the OpenAI pass-through carry it.
- Features that don't work on a given backend should return a structured 501, not pretend to work.
- Tests use
httpx.MockTransport. Don't add integration tests that require a real GPU to the defaultpytesttarget.