Conversations and follow-ups

A conversation has many messages. If each one went to a different model, every switch would lose the model’s cache: the new model has to read the whole conversation again, which is slower and, on paid models, costs more. So a Smart Route usually keeps a conversation on one model, and moves it only when there’s a good reason.

Choose how follow-ups are routed

In the route editor, In a conversation has three settings:

Setting	What happens	Use it when
Switch when the task changes (default)	Follow-ups stay on the same model. A message that fits another choice, like an image request, moves there. Sensitive conversations never move to the cloud.	Most routes.
Keep the first model	The conversation stays on the model that answered its first message, unless switching costs almost nothing.	Long chats and coding agents, where the cache matters most.
Route every message	Every message is routed on its own. Switching models often loses the cache, so long chats can cost more.	Short, unrelated requests, such as one-off API calls.

How Nexus knows it’s the same conversation

Nexus doesn’t store the conversation. It recognizes it in one of two ways:

The session header. If the app sends X-Msty-Nexus-Session-Id, that wins. Use it when your app knows its own conversation IDs.
The start of the prompt. Otherwise, Nexus looks at the system prompt, the tool list and the first user message. Turn 1 and turn 40 of one chat start the same way; two different chats don’t. Nexus keeps only a digest of that start, in memory, never the text. It’s also scoped to the API key, so two apps that open with the same words never share a conversation.

The second way needs nothing from the app, so a plain OpenAI SDK call is followed too.

A conversation that’s idle for 30 minutes starts fresh.

When a conversation moves

With Switch when the task changes, a follow-up stays on its model unless:

the router clearly picks another choice: the task changed, such as from writing to translating;
no choice fits anymore, so it goes to the fallback;
the message needs something the model can’t do, such as reading an image;
the model isn’t available.

With Keep the first model, the router still reads every message, but Nexus moves the conversation only when reading it again on the new model is cheap:

to a paid model, when it would cost about 2¢ or less;
to a local model, when it’s 4,096 tokens or less to read again (a few seconds);
never to a model whose price Nexus doesn’t know.

So a short chat that turns hard early on still gets the strong model. A chat that’s 30 turns deep keeps its cache.

Rules, guards and sensitive data

Rules: a conversation a rule sends somewhere stays there, like a choice.
Guards: a guard moves one message, not the conversation. The next message goes back once the model recovers.
Sensitive data: once Data Protection finds sensitive data in a conversation, it stays on this machine. It never moves to an online model.

Why a message went where it did

Each request in the route’s history, and the X-Msty-Nexus-Route-Reason response header, says why:

Reason	Code
First message of the conversation	`first_message`
Follow-up, stayed on the same model	`follow_up`
Same type of request, stayed on the same model	`same_choice`
New task, moved to a better choice	`task_changed`
The earlier model was not available	`target_unavailable`
The earlier model cannot handle this request	`needs_capability`
Kept on this machine: the conversation has sensitive data	`privacy_lock`
Kept on the conversation’s first model	`kept_first_model`
Routed on its own	`per_message`
Not part of a known conversation	`no_conversation`
The router did not answer in time, so the conversation stayed	`router_unavailable`
A rule picked this target	`rule`
A guard moved this away from a struggling target	`guard`

X-Msty-Nexus-Route-Affinity says how the conversation was treated: pinned (stayed), switched (moved), classified (routed fresh) or no_key (no conversation found). See Use from apps.

If something goes wrong

You see	Do this
Every message says `no_conversation`	The request has no user text to recognize, or uses `previous_response_id`. Send `X-Msty-Nexus-Session-Id`.
A long chat keeps switching models	Use Keep the first model.
A chat stays on a model that no longer fits	Use Switch when the task changes, or start a new chat.
Two users’ chats seem to share a model choice	They may send the same session ID. Give each conversation its own.