NNyquest docs

Streaming

Replies stream in token-by-token — you see words appear as the model generates them, not all at once at the end. This is intentional and works the same across every chassis.

Why streaming

  • Faster perceived response time. Even if a 1,000-word reply takes 10 seconds total, you start reading at second 1. Time to first word matters.
  • Early stop. If the model is going off track, you can stop the stream and re-prompt without waiting for it to finish. (Look for the Stop button next to the chat input while streaming.)
  • Live cost feedback. Token counts and cost tick up live in some chassis (notably Quant and Apsis).

What's actually happening

Server-Sent Events (SSE) over HTTPS:

client opens GET /chat/stream over HTTP/2
   ↓
server pipes through to the model provider
   ↓
each token arrives → server forwards as `data: {...}\n\n`
   ↓
client appends to the message bubble character-by-character

No WebSockets, no polling. Just one long-lived HTTP request.

Stop button

While a reply is streaming, the chat input shows a Stop button. Click it to:

  • Cancel the in-flight request
  • Keep whatever was streamed so far in the conversation
  • Free up the resources

You're billed only for the tokens that streamed before stop (input tokens + the partial output).

When streaming doesn't apply

A few cases skip streaming:

  • Reasoning models (o1, GPT-5 with reasoning enabled) sometimes return the entire response at once after thinking silently. The chat shows a thinking indicator until the response arrives.
  • Tool-use turns in Agent Mode stream the tool call but the response after the tool result usually streams normally.
  • Very short replies can complete before you see streaming. That's fine.

Common questions

Why does streaming sometimes stutter or pause? The model is "thinking" — generating but in a way that produced a long token (or a tool call boundary). Brief pauses are normal. If it pauses for 30+ seconds with no progress, the model might be stuck — hit Stop and try again.

Can I disable streaming? Not currently. Every chassis renders streamed replies. If you want a non-streaming UX (full reply at once), you can wait for the stream to complete and then read the final message — same end result.

Where to next