Connect any OpenAI-compatible endpoint

Any endpoint that speaks the OpenAI API connects. Big-AGI accepts what the endpoint returns without strict validation, since dialects carry fields validation would strip. That covers vLLM, llama.cpp's server, LiteLLM, an inference startup nobody has heard of, a company gateway, and proxies in front.

Adding one

In the add-service menu, the OpenAI entry carries a small Add button on top - tooltip Add another OpenAI-compatible service. Press it and a new copy appears, labelled OpenAI #2, #3 - type over the label to rename it, up to 32 characters. Up to ten copies can exist side by side, each with its own endpoint, key, name and route.

1. API Endpoint is the required field, behind the panel's Advanced toggle: enter your custom host. A few example services and one gateway come pre-filled - picking one fills the URL, renames the service and links its documentation.
2. A key is optional: one is required only when the effective host is OpenAI's own, so a keyless internal endpoint works with the field empty.
3. Choose whether the service uses Direct Connection.
4. Press Models: the endpoint is scanned and its models join the list.

API Endpoint	What happens
https://host.example/v1/ -> https://host.example	trailing /v1 stripped
host.example -> https://host.example	scheme becomes https
appended for you	/v1/chat/completions · /v1/responses · /v1/models · /v1/images/generations · /v1/images/edits · /v1/audio/speech

Models without the bells and whistles

The built-in AI services ship their model metadata; a custom endpoint rarely publishes any: no context window, no price, no capability flags. The models work; they arrive plain, and stay plain until you set the numbers yourself. The gear on each model carries overrides for context window, maximum output and price, each with a Reset.

Proxies and company gateways

A proxy or gateway in front of the endpoint works. Two behaviours matter.

Only your credentials are used. With a custom endpoint set, the key and organisation you entered are the ones sent - the deployment's own key is never substituted, so a gateway issuing its own tokens works.

Reachability depends on the route. With Direct Connection off, the Big-AGI server must reach the endpoint, so a gateway published only inside an office network fails from the hosted site. With it on, your browser makes the call and the endpoint's CORS policy has to allow it.

One shape to get right: Cloudflare AI Gateway expects the account and gateway in the path - /v1/<account>/<gateway>/openai - and accepts only OpenAI upstream.

Related

• Connect an AI service
• Models on your own machines
• Keep your key safe
• Key and access errors