Connect NVIDIA NIM, hosted or your own

An address on its own counts as a credential here. Big-AGI accepts either an nvapi- key for NVIDIA's hosted endpoint or a host for a NIM or vLLM server you run, which may need no key at all. No other cloud AI service in this set can be configured without a key.

Two ways to run it

The hosted endpoint is free with no payment method and rate-limited per account. Joining the developer programme is what yields the nvapi- key; confirm the current allowance at build.nvidia.com rather than from any figure quoted elsewhere. A server you run yourself bills NVIDIA nothing, and its cost is hardware and electricity.

Free here means trial terms. NVIDIA may use prompts and outputs to improve its services, and production use is disallowed. The setup panel states the data-use half under the key field. Read the trial terms before putting work data through it.

Getting an nvapi- key

1. Open build.nvidia.com/settings/api-keys, the page the app links, behind a sign-in.
2. Generate a key with the Public API Endpoints scope included, and pick its expiry date. It begins nvapi-.
3. Copy it. Running your own server instead? Skip the key and note its address.

A key, a host, or both

Ctrl + Shift + M opens Models; press More Services if the Setup AI Models wizard opens. Then Add -> NVIDIA NIM. For the hosted endpoint, paste into NVIDIA API Key. For your own server, fill API Host under Advanced instead - the pair validates on either one, so a keyless host is accepted. A NIM or vLLM box on your network goes in as its own address, http://spark-xxxx.local:8000 for a DGX Spark. Press Models to fetch the list (add and manage your keys).

Anything further from the standard shape belongs on custom endpoints.

The curated model list

NVIDIA retires hosted models about every two weeks, and its raw model listing keeps returning the dead ids. Big-AGI drops the ones its own harvest confirmed retired, so they never reach your list and never fail mid-chat. A model the harvest has not seen yet carries a [?] prefix on its label. That prefix means uncurated: capabilities and context window stay unverified until the next harvest.

The limit is a rate limit

On the hosted endpoint there is no bill to cap; the account's rate limit is what binds. The allowance counts requests, not tokens, so a long prompt costs the same as a short one. Big-AGI also spaces its own calls about 1.6 seconds apart there, to stay inside that limit, so multi-model runs feel slower on the hosted endpoint by design. A custom host is not throttled.

Browser-direct only on your own host

Direct Connection here is inverted from the rest of the set: it can be turned on once you set a custom API Host. The default hosted endpoint admits only its own origins in the browser, so there the request goes through the Big-AGI fast edge servers.

When this doesn't work

What you see	What it means	What to do
Upstream responded with HTTP 403 Forbidden	The hosted endpoint needs an nvapi- key and none was sent.	Key and access errors
404 Function not found for account	The key lacks the Public API Endpoints scope, or that model is served only to entitled accounts.	Recreate the key with the scope; if it persists on one model only, use another: key and access errors

Related

• Connect an AI service
• Connect any OpenAI-compatible endpoint
• Direct Connection