Presets and Routes

Presets and route packs give clients stable model values without hard-coding every provider, model, and parameter decision into each app.

Use Models for route packs and Presets surfaces inside the app to manage reusable targets.

Presets

A preset is a named model configuration. Clients call it with:

@preset/<slug>

A preset can define:

• Display name
• Target model
• Scope
• Supported default params such as temperature and max tokens
• Provider-keyed options
• Thinking effort and display
• Anthropic prompt caching
• Local acceleration settings where supported

Gateway requests merge preset defaults first and the incoming request body last, so callers can override one field for a single request without editing the preset.

Generation controls

Current Nexus presets support richer generation controls than a flat parameter list:

• Params are provider-neutral defaults such as temperature and max tokens.
• Provider options are keyed by provider kind, with * for defaults that apply to every provider.
• Thinking effort uses off, minimal, low, medium, high, or max.
• Thinking display uses summarized or omitted.
• Prompt cache can ask Anthropic models to cache stable prompt prefixes for 5m or 1h.

Nexus translates these semantic settings to the selected provider at request time. For example, OpenAI-shaped providers receive reasoning fields, Anthropic receives its thinking/cache wire shapes, and llama.cpp can receive a chat-template thinking toggle where supported.

When a provider cannot honor a saved setting, Nexus reports warnings such as unsupported_param, unsupported_thinking, unsupported_thinking_display, unsupported_prompt_cache, or unknown_provider_bucket. Unsupported fields are dropped before the upstream request instead of being forwarded blindly.

Use preset validation before saving substantial preset changes. Validation dry-runs the draft against the target model and reports warnings without changing the stored preset.

When to use presets

Use presets for:

• Friendly model names for clients
• Stable team or workflow defaults
• Provider-specific defaults that should not be repeated in every request
• Local runtime acceleration settings
• Safer model handoff when the underlying model ID may change later

If a model delete fails with RESOURCE_IN_USE, check whether a preset still references it.

Preset acceleration

Compatible local presets can use a smaller helper model to speed up generation. The app validates whether the helper is compatible with the target model and runtime.

Acceleration flow:

1. Create or open a local preset.
2. Choose Speed up this preset.
3. Select a compatible helper model.
4. Run validation.
5. Apply the change.
6. Restart the runtime when required.

Usage stats expose acceleration counters separately, including speedup, accepted draft tokens, fallback requests, and estimated time saved when enough baseline traffic exists.

Route packs

A route pack is a single model value that can choose among multiple target models. Clients call it with:

@route/<slug>

Each route pack has:

• Slug
• Name and description
• Enabled state
• Fallback model
• Optional routes that steer matching request types to target models

Route packs are useful when clients should call one stable model value but Nexus should route different classes of requests differently. Fallback models can be chat-capable or image-generation-capable depending on the route pack's intended traffic.

Smart Route

Smart Route uses a compatible local runtime to prepare a small routing model. If Smart Route is not ready, route packs stay on their safe fallback path.

Setup pattern:

1. Configure at least one compatible local runtime: MLX, llama.cpp, or Ollama.
2. Add a chat-capable fallback model.
3. Open route packs and prepare Smart Route.
4. Create a route pack with a fallback.
5. Add routes only when there is a clear routing policy.

Nexus records route decisions as stable IDs and counters. It does not store prompts, classifier output, or provider response text in usage aggregates.

Hidden models are excluded from route-pack and Smart Route picker surfaces. If a model should not be callable by connected clients, hide it in Models before assigning routes.

Gauge-aware capacity routing

Route packs can opt into Gauge capacity signals when Gauge is enabled and configured. If a selected target is constrained, Nexus uses the configured fallback and records gauge_capacity_constrained as the decision reason.

Gauge never switches accounts or providers automatically outside route-pack policy. The route pack owns the fallback behavior.

Practical recommendations

• Use aliases for simple stable model names.
• Use presets when params or provider options matter.
• Use route packs when one client-facing value should route across models.
• Keep every route pack fallback valid before adding advanced routes.
• Validate preset thinking/cache settings before depending on them in clients.
• Prepare Smart Route only after a local runtime and fallback model are stable.