Models

Models are the routable entries that clients use through the gateway. A model belongs to a provider, declares supported endpoint families, and can have aliases, visibility state, runtime metadata, and local performance settings.

Open Models to search, install, import, sync, hide, unhide, alias, load, unload, or tune models.

Model catalog

The catalog stores model records such as:

• Model ID
• Provider ID
• Display name
• Scope
• Endpoint families
• Input and output modalities
• Local/online state
• Aliases
• Hidden/visible state
• Safe runtime metadata

Model IDs often include the provider prefix, such as openai/gpt-test, ollama/llama3.1:8b, or llamacpp/qwen3.gguf.

Endpoint families

Endpoint families determine which gateway routes a model can use:

• openai.chat_completions
• openai.embeddings
• openai.responses
• openai.images
• openai.audio_speech
• openai.audio_transcriptions
• anthropic.messages
• rerank.documents

Nexus validates endpoint families when catalog records are written and again when requests are routed. This prevents a model approved only for one protocol from being called through another.

Modalities

Model metadata can include:

• inputModalities: text, image, audio, video, or pdf
• outputModalities: text, image, audio, or embedding

Nexus uses modalities for search filters, model-card badges, route-pack eligibility hints, and SDK feature detection. Modalities do not hard-block gateway requests; provider and runtime APIs remain the final authority.

Aliases and visibility

Aliases give clients stable model names such as default-chat or coding-local. Direct model IDs always take precedence over aliases.

Use visibility controls to hide models that should not appear in /v1/models or be called by direct ID/alias. Hidden models remain manageable through trusted control surfaces.

Hidden models are also excluded from Smart Route and preset/route picker surfaces so an operator-hidden model does not reappear indirectly in connected clients.

Local model search and install

The local model search flow returns install-ready candidates only for installed Nexus runtimes.

Sources include:

• Ollama registry for Ollama pulls
• Hugging Face snapshot models for MLX
• Hugging Face GGUF files for llama.cpp

Search results can include warnings when one registry is unavailable. A partial-source warning does not fail the whole search when another source succeeds.

Runtime model actions

Model actions run as jobs:

• pull
• import
• delete
• load
• unload
• sync

Runtime-specific behavior:

• Ollama supports pull, delete, load, unload, and sync through native Ollama APIs.
• llama.cpp supports Hugging Face GGUF pull, local GGUF import, managed delete, sync, native-router load, and unload.
• MLX supports Hugging Face snapshot pull, managed cache delete, and sync on macOS.
• Online providers support provider model sync where provider APIs expose model lists.

If another job already targets the same runtime or provider, Nexus returns RESOURCE_IN_USE with safe active job details.

GGUF import and delete safety

llama.cpp import accepts an absolute local file path as write-only input from the native app, copies the GGUF into Nexus-managed storage, and then stores only relative model metadata.

Delete is deliberately narrow:

• Nexus deletes only Nexus-managed model files.
• Absolute paths are not returned to the frontend or API clients.
• Path traversal, symlink escape, unmanaged files, and non-GGUF targets are rejected.
• Multimodal projector files are attached to the same catalog model and removed with that model.

Loaded models

The loaded-models view reports what is resident in local runtime memory now. It uses runtime-native APIs where available, such as Ollama's process list and llama.cpp's router model list.

Loaded entries can include:

• Service ID
• Provider ID
• Catalog model ID
• Runtime model name
• Loaded/loading state
• Model size
• VRAM use
• Keep-alive expiry

Entries without a matching catalog record are still shown so memory use is never hidden.

Library health

Library health scans managed model storage and runtime-reported model state so local model drift is visible before gateway calls fail.

It finds issues such as:

• Missing files
• File-size mismatches
• Orphan files
• Stray files
• Runtime models missing from the catalog
• Catalog models missing from the runtime

The route is read-only. Suggested repairs map to existing actions like sync, pull, or delete.

Loaded model memory

Loaded model memory shows what local runtimes have resident right now. Nexus reads runtime-native APIs such as Ollama's process list and llama.cpp's router model list.

Use it to answer:

• Which models are loaded or loading
• Which service owns the loaded model
• How much memory or VRAM the runtime reports
• Whether keep-alive expiry is approaching
• Whether a runtime is unreachable versus simply empty

Entries without matching catalog records still appear so memory usage is not hidden behind catalog drift.

GPU and performance

GPU placement is saved per local model. Nexus uses public-safe hardware telemetry to build plans and avoids vendor UUIDs, PCI addresses, local paths, or raw runtime flags.

Use GPU & Performance to:

• Choose balanced, max-speed, max-context, CPU-only, or custom placement goals
• Select automatic or manual GPU placement where supported
• Preview memory impact before applying
• Apply and verify placement
• Roll back to the previous verified setup
• Run bounded local benchmarks

Warnings such as GPU_FALLBACK_TO_CPU and GPU_PLACEMENT_VERIFICATION_LIMITED mean the setup completed but needs review.

Practical model strategy

• Start with one reliable visible chat model.
• Add aliases only when clients need stable names.
• Use presets for reusable defaults.
• Use route packs when one model value should choose among multiple targets.
• Use GPU tuning after the model is installed and routable.
• Use Health & Memory after external model changes or interrupted downloads.