Connect LocalAI

Big-AGI ships a capability list for LocalAI, naming what this runtime covers and what it does not - plus the address, and an optional key.

LocalAI Supported Models ->

Browse documentation

LocalAI is the one connection where Big-AGI states, in the setup panel itself, which capabilities the runtime covers. That list is below, and it is the honest boundary of what this OpenAI-compatible server does for you. The connection is an address, plus a key only if you started the server with one, and the hosted app connects to it like any other AI service (models on your own machine).

Running the container

The project's own quickstart line is docker run -p 8080:8080 --name local-ai -ti localai/localai:latest (getting started).
It then serves at http://127.0.0.1:8080. Pull one model from the gallery; weights are large and speed follows your memory and GPU, so start small. http://127.0.0.1:8080/v1/models in a browser tab returns the JSON list of what it loaded, before Big-AGI is involved.
For your browser to dial it directly, start it with CORS on - the CORS and CORS_ALLOW_ORIGINS environment variables on the container.

Address, optional key, and what it covers

Open Models (Ctrl + Shift + M), press More Services if the Setup AI Models wizard opens, then Add -> LocalAI. The address goes in LocalAI URL; Optional API Key stays empty unless the server was started with one. Press Models to fetch the list (add and manage your keys).

	Supported features
✓	text generation · function calling · model gallery · vision API · JSON output · image generation
✗	speech transcription · text-to-speech · embeddings generation · voice cloning

The panel's Show more chip opens that list. A capability outside it is the only-mostly-compatible case, and it belongs to custom endpoints.

Installing models from the panel

Install Models sits beside the models refresh button and opens a modal titled LocalAI Administration. It lists what your server offers under Available Models in Gallery - the model gallery row of the table above. Each install runs as a job; the modal polls it and shows progress. A Show Voice Models switch and a Debug: show JSON expander sit below the table.

The gallery itself is configured on the LocalAI server. The modal states the requirement: Install models from your LocalAI Model Gallery. You need a properly configured LocalAI gallery.

Direct Connection

Always eligible. With it off, the fetch is made by the Big-AGI fast edge servers, which cannot see your 127.0.0.1. With it on, your LocalAI server needs CORS enabled to accept your browser's origin.

Note

Direct Connection works only with your API key stored in the browser, and with an AI service that permits direct browser calls (CORS). Where it cannot be used, requests route through the Big-AGI fast edge servers instead - everything still works, within the standard upload size and time limits.

When this doesn't work

What you see	What it means	What to do
`[OpenAI/Localai network issue]: Could not connect:`	The container is not running, or the address is wrong.	I don't see the model I want
It answers in the terminal and not in Big-AGI	CORS is off, so the server refuses your browser's origin.	Restart with `CORS=true`: my local models don't show up
`Model galleries do not seem to be configured (null response).`	Install Models asked the server for its galleries and got null back.	Configure a gallery on LocalAI, per its model documentation