Models reference

GraphN routes each agent to a language model you choose in the agent editor. Built-in models are ready to use; you can also import an OpenAI-compatible endpoint (for example vLLM, Ollama with the OpenAI API, or a third-party API) and select it like any other model.

Built-in models

When you open the Models page (under Resources in the sidebar), you see a card grid of every model available in your workspace. When configuring an Agent node on the workflow canvas, you pick a model from a dropdown in the node panel. Both views list the same chat and vision models.

Chat and multimodal models

The following models are available in the model picker. Each links to its catalog page with the full editorial write-up, specifications, and runnable API examples. Your deployment may also include additional models discovered from the inference gateway; these appear alongside the built-in entries with names derived from their model identifiers.

Scroll horizontally to compare

Model	Callable name	Type	Context	When to use
Qwen3 80B Instruct	`qwen3-80b`	Chat	131K	Everyday chat and agent work at near-flagship quality; the default chat model.
Qwen3 235B Instruct	`qwen3-235b`	Chat	262K	Highest-quality answers when accuracy matters more than cost.
Qwen3 Coder	`qwen3-coder`	Chat	131K	Coding agents and IDE workflows at repo-scale context.
gpt-oss 120B	`gpt-oss-120b`	Chat	131K	Reasoning with a cost dial: low, medium, or high effort per request.
Gemma 4 31B Instruct	`gemma-4`	Chat	131K	A non-Qwen alternative with unusually wide language coverage (140+ languages).
Nemotron 3 Super 120B	`nemotron-3-super`	Chat	262K	A hybrid Mamba-2 model whose failure modes differ from the transformers here.
Qwen3.8 27B (image+video)	`qwen3.8-27b`	Chat / Vision	262K	Coding, long-horizon agents, and image/video understanding in one dense model.
Qwen3.5 122B Vision (image+video)	`qwen3.5-122b-a10b-fp8`	Chat / Vision	262K	The strongest visual reasoning here, for dense documents, diagrams, and video.
Qwen3.5 9B Vision (image+video)	`qwen3-vl`	Vision	262K	The most economical way to put images and video in front of a model.
Gemma 4 E4B (text+image+video+audio)	`gemma-4-e4b`	Audio chat	66K	Speech and images in one conversational model, without a separate ASR step.

Note: The agent model selector offers chat and vision models. Your deployment may show additional models discovered from the inference gateway; they appear in the same picker with capabilities and context limits inferred from their model identifiers.

Model picker showing available chat and vision models as a card grid

Speech, OCR, and retrieval models (not in the agent model picker)

The platform registers speech-to-text, text-to-speech, OCR, embedding, and reranker models for features such as voice input and output, document reading, and retrieval. They are not offered as the main LLM in the agent model selector.

Scroll horizontally to compare

Model	Callable name	Type	When to use
Qwen3 ASR 1.7B (52 languages)	`qwen3-asr`	Speech-to-text	Speech to text for voice agents, streaming or batch, with word-level timestamps.
Qwen3 TTS (10 languages, 9 speakers)	`qwen3-tts`	Text-to-speech	Default speech synthesis: 9 voices, 10 languages, styled by instruction, voice cloning.
DeepSeek OCR	`deepseek-ocr`	OCR	Turns document images into markdown with layout preserved.
HunyuanOCR	`hunyuan-ocr`	OCR	Compact OCR that also spots text, extracts fields, and translates.
BGE-M3 Embeddings (1024 dim)	`bge-m3`	Embedding	Semantic search and retrieval across knowledge bases (default text embedding).
Qwen3-VL Embeddings (4096 dim, multimodal)	`qwen3-vl-embedding`	Embedding	Multimodal knowledge bases that index text and images together; selectable at KB creation time.
BGE Reranker v2	`bge-reranker`	Reranker	Reranking search hits for better relevance ordering.
Qwen3-VL Reranker (multimodal)	`qwen3-vl-reranker`	Reranker	Relevance scoring across modalities: text against images and back.

Decision guide

Scroll horizontally to compare

Your goal	Recommended model
General agents, tools, structured JSON	Qwen3 80B Instruct
Hardest tasks, long inputs, or long-horizon agents	Qwen3 235B Instruct
Code-heavy agents and IDE-style workflows	Qwen3 Coder
Adjustable reasoning effort per request	gpt-oss 120B
Broad multilingual coverage	Gemma 4 31B Instruct
Multimodal inputs (images, video + text)	Qwen3.5 9B Vision or Qwen3.8 27B
Voice input without a separate transcription step	Gemma 4 E4B
Fine-tuned or external API you control	Import an endpoint (next section) and pick that model

Custom model import

Import an OpenAI-compatible HTTP API whose chat path is /v1/chat/completions (same request shape as OpenAI). Typical sources include vLLM, Ollama (OpenAI compatibility mode), OpenAI, Google Gemini via OpenAI compatibility, and OpenRouter.

Tip: Want to import directly from HuggingFace or your own S3 bucket from a script instead of the wizard below? See the Python SDK tutorial for an end-to-end walk-through.

Import from the UI

In the sidebar, under Resources, click Models.
Click Import Model (top-right corner).
The import wizard has two tabs:

API Endpoint (connect to a running inference server):

Choose a provider:
- OpenAI, Google (Gemini), or OpenRouter — preset base URLs; you attach a secret that stores your API key, then use Fetch Available Models to list models from /v1/models (filtered to chat-relevant entries).
- Custom Provider — for any other OpenAI-compatible base URL (for example http://your-host:8000/v1 for a local vLLM server). Enter the endpoint URL, model id (exactly as the server expects in API calls), and optionally an API key secret if the server requires authentication.
(Optional) Click Configure to set type (chat, vision, or embedding), capabilities (for example tool calling, vision, streaming, json mode), context length, and description.
Click Import. You are taken to the new model’s detail page.

Tip: For self-hosted vLLM or Ollama, the base URL usually ends with /v1. The model id must match what you pass when calling that server (for example the served model name).

HuggingFace Weights (deploy a model from HuggingFace):

Enter the full HuggingFace model ID (for example meta-llama/Llama-3-8B-Instruct).
Optionally set a display name (auto-generated from the model ID if left blank).
If the model is gated or private, click Attach secret to provide your HuggingFace token.
Click Validate & Continue to proceed through the optimization and scaling steps.

Import Model wizard showing API Endpoint and HuggingFace Weights tabs

Test the connection before using an agent

After import, open the imported model’s page. In the Test Model section, enter a short prompt and run the test. The app streams a reply so you can confirm the endpoint, credentials, and model id work together.

Warning: If the test fails, fix the endpoint URL, secret, or model id before selecting this model on an agent—otherwise runs will fail at inference time.

Verify connectivity with the API (optional)

You can send a single chat completion through GraphN to validate an endpoint before or after saving an imported model, using the same workspace and auth as the app.

Replace the placeholders with your workspace id, bearer token, base URL (must include /v1), and upstream model id:

bash

curl -X POST "https://cp.graphn.ai/v1/${WORKSPACE_ID}/imported-models/test-connection" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '{
    "endpoint": "https://your-endpoint.example.com/v1",
    "model_id": "your-model-name",
    "api_key_secret_id": null,
    "message": "Reply with one short sentence."
  }'

Omit api_key_secret_id or set it to null when the server does not use a bearer token. When you store a key in Secrets, pass that secret’s id so the test can authenticate.

A successful response includes a response field with the generated text, the model name returned by the server, and usage token counts when the upstream returns them.

Register an imported model with the API

To create the imported model record programmatically (equivalent to finishing the import wizard):

bash

curl -X POST "https://cp.graphn.ai/v1/${WORKSPACE_ID}/imported-models" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '{
    "name": "my-endpoint-model",
    "display_name": "My vLLM Model",
    "endpoint": "https://your-endpoint.example.com/v1",
    "model_id": "your-model-name",
    "api_key_secret_id": null,
    "type": "chat",
    "capabilities": ["tool_calling"],
    "context_length": 128000,
    "description": "Imported from our vLLM cluster."
  }'

Use a unique name (URL-friendly identifier) within the workspace; agents reference this value as the model setting. display_name is what people see in lists and selectors.

List models from a provider endpoint (API)

To mirror Fetch Available Models in the UI (OpenAI-style GET /v1/models on the same base URL as chat):

bash

curl -X POST "https://cp.graphn.ai/v1/${WORKSPACE_ID}/imported-models/discover-models" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '{
    "endpoint": "https://api.openai.com/v1",
    "api_key_secret_id": "YOUR_SECRET_ID"
  }'

endpoint and api_key_secret_id are required; the secret must hold the provider API key.

Next steps

Function SDK Reference — use models in custom Python functions with chat.complete(model="...") and vision.analyze(model="...")
Concepts — how models relate to agents and workflows
API Reference — full endpoint reference including imported model CRUD

Documentation

Getting Started

CLI & IDE

Concepts

Reference

Tutorials

Models reference

Built-in models

Chat and multimodal models

Speech, OCR, and retrieval models (not in the agent model picker)

Decision guide

Custom model import

Import from the UI

Test the connection before using an agent

Verify connectivity with the API (optional)

Register an imported model with the API

List models from a provider endpoint (API)

Next steps