Long context without the wait

Big windows, on serverless

Long context is a standard serverless call on Nyx. There is no waitlist, no separate long-context SKU, and no surcharge past some threshold: qwen3-235b-a22b-2507 and gemma-4-26b-a4b-it both hold 262,144 tokens at the same per-token price whether the prompt is four hundred tokens or two hundred thousand, on the same OpenAI-compatible endpoint. kimi-k3 raises the window to 1,000,000 tokens when it goes live, at $10.50 per million output tokens under the same rule.

Sliced prefill keeps the fleet honest

The failure mode of long context elsewhere is what a giant prompt does to everyone behind it. Nyx runs long-context pods that cut prefill into 8,192-token slices, so a million-token document is scheduled as thousands of small units instead of one monolith that owns the GPU for a minute. A long prompt takes a capped share of any scheduling tick, short prompts interleave between the slices, and a document arriving mid-session never pushes a one-line question to the back. How that scheduling works is written up in the kimi-k3 launch post.

Who runs at this scale

Whole-repository work is the common case: a coding agent that reads the entire codebase before planning a migration, instead of retrieving fragments and guessing at the rest. The other is transcription-heavy products, where a day of meetings or a full deposition lands in one prompt and the model answers with every exchange still in view.

Agent sessions get the same benefit over time. Tool output, file contents, and conversation history accumulate for hours without a summarization step throwing away the detail the next decision needs. The prefix cache carries the growing context across turns, which removes 31 percent of prefill work on agent traces.

# the whole deposition goes in the prompt
jq -n --rawfile doc deposition-full.txt '{
    model: "qwen3-235b-a22b-2507", stream: true,
    messages: [{role: "user", content:
      ($doc + "\n\nWhere do the two witnesses
      contradict each other?")}]
  }' |
curl https://api.nyxprovider.com/v1/chat/completions \
  -H "Authorization: Bearer $NYX_API_KEY" \
  -d @-

Recommended models