Chat Completions

Request body

{
  "model": "gpu-mesh",
  "messages": [
    {"role": "system", "content": "You are concise."},
    {"role": "user", "content": "What is decentralized inference?"}
  ],
  "max_tokens": 256,
  "temperature": 0.7,
  "stream": true
}

Auto-routing

Pass model: "gpu-mesh" for the network to pick any available provider/model. Pass a specific id (e.g. llama-3.2-3b) to require that exact model — if no provider has it cached, the orchestrator silently falls back to the closest model in the same family/tier and adds routed_from to the response.

Streaming format

With stream: true you get SSE chunks in the OpenAI delta shape. Terminator is the literal data: [DONE]:

data: {"choices":[{"index":0,"delta":{"content":"Hello"}}]}

data: {"choices":[{"index":0,"delta":{"content":" world"}}]}

data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Reasoning content

Some models (Qwen 3, DeepSeek-R1, etc.) emit <think>...</think> blocks. We parse those out and surface them as delta.reasoning_content, separate from the final answer in delta.content. The dashboard renders reasoning as a collapsible block.

Errors

• 401 auth — missing or invalid bearer token
• 400 invalid_request — empty messages[]
• 402 insufficient_credits — top up via faucet / convert / buy
• 502 provider — picked node errored mid-task
• 503 service_unavailable — no live providers AND no fallback
• 504 task_timeout — provider exceeded 120 s