Powered byLightning AI
Documentation

    Inference quickstart

    Getting started

Inference quickstart

GraphN serves open models behind an OpenAI-compatible API. If you already use an OpenAI SDK, change two things: the base URL and the API key. New accounts start with $1 of inference credit.

1. Create an API key

In the app, open API keys at the bottom of the sidebar and create a key. Keys start with gn_ and belong to one workspace; requests made with a key are billed to that workspace's organization.
Export it so the examples below can read it:
bash
export GRAPHN_API_KEY=gn_...

2. Send a request

Base URL: https://model.graphn.ai/v1

curl

bash
curl https://model.graphn.ai/v1/chat/completions \
  -H "Authorization: Bearer $GRAPHN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3-80b",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Python

Install the SDK with pip install openai, then:
python
import os

from openai import OpenAI

client = OpenAI(base_url="https://model.graphn.ai/v1", api_key=os.environ["GRAPHN_API_KEY"])

response = client.chat.completions.create(
    model="qwen3-80b",
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)

TypeScript

Install the SDK with npm install openai, then:
typescript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://model.graphn.ai/v1",
  apiKey: process.env.GRAPHN_API_KEY,
});

const response = await client.chat.completions.create({
  model: "qwen3-80b",
  messages: [{ role: "user", content: "Hello" }],
});
console.log(response.choices[0].message.content);

CLI

bash
brew tap voltagepark/graphn && brew install graphn
graphn login
graphn model chat qwen3-80b -m "Hello"
graphn model chat without -m starts an interactive session. See Developing with agents for other install options.

3. Pick a model

The model field takes the model's callable name, such as qwen3-80b or deepseek-v4.1-flash. Models lists every built-in model with its context window, and each links to a page with pricing. To list the models your key can call:
bash
curl https://model.graphn.ai/v1/models \
  -H "Authorization: Bearer $GRAPHN_API_KEY"
Models you import or deploy yourself are called the same way, as imported:<id> or custom:<id>. The Playground in the app runs any of them and shows the matching request in its Code view.

Streaming

Set stream to receive the reply as server-sent events, one chunk at a time:
python
stream = client.chat.completions.create(
    model="qwen3-80b",
    messages=[{"role": "user", "content": "Write a haiku about GPUs."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Images

Vision models take OpenAI-style content parts. An image can be an https URL or a data: URL:
python
response = client.chat.completions.create(
    model="qwen3-vl",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What is in this image?"},
                {"type": "image_url", "image_url": {"url": "https://example.com/photo.jpg"}},
            ],
        }
    ],
)

Parameters beyond the OpenAI SDK

Some models take parameters the OpenAI SDK does not define, such as top_k or a thinking switch. In Python, pass them through extra_body:
python
response = client.chat.completions.create(
    model="qwen3.8-27b",
    messages=[{"role": "user", "content": "Hello"}],
    extra_body={"top_k": 20, "chat_template_kwargs": {"enable_thinking": False}},
)
Reasoning tokens count toward max_tokens, so a thinking model given a small budget can stop before it answers. Turn thinking off, or raise the budget.

Errors

Scroll horizontally to compare
StatusMeaning
401The key is missing, revoked, or malformed.
402The organization is out of credit. Add credit under Billing.
404No model by that name is available to this workspace.
429Too many requests. Retry with backoff.
503A model you deployed is scaled to zero and is waking up. Retry after a short wait.

Next steps

  • Compare models side by side in the Playground's Compare view.
  • Deploy your own weights or attach LoRA adapters.
  • Build multi-step pipelines on top of these models with workflows.
Next

Getting started