Serving kimi-k3 at a million tokens of context
kimi-k3 on Nyx now accepts prompts up to 1,000,000 tokens, the full
window the model was trained for. It is served at
https://nyxprovider.com/v1 with the same OpenAI-compatible
schema as every other model in the catalog, and the price sits under the
cheapest public listing for it: $2.548 per million input tokens and
$12.74 per million output.
The window matters most for agent workloads. A million tokens holds a mid-sized repository, the issue thread that prompted the change, and a few hundred tool call results at once. Below that ceiling, agents spend a real share of their steps managing their own history: summarizing files they read an hour ago, re-fetching context they dropped, reconciling two summaries that disagree. With the whole working set resident, the loop shortens. In our traces, a coding agent that took 41 steps against a 128k window finished the same task in 26 steps against the full window, and 9 of the removed steps were pure re-reading.
The window is also where serving gets hard. Prefill for a 900,000-token prompt is around 40 seconds of dense compute on our kimi-k3 pods. Schedule that naively, first come first served, and one agent uploading a repository freezes time-to-first-token for every short request behind it.
Prefill that starves nobody
We run prefill in 8,192-token slices interleaved with decode. The scheduler works on a fixed tick, and long-prompt slices can claim at most 35 percent of any tick; the rest belongs to decode and to short prefill. A million-token prompt finishes about 12 percent later than a greedy schedule would allow, and in exchange p50 time-to-first-token for prompts under 4k tokens stays flat: 240ms with no long prompt in flight, 251ms with three of them running.
The KV cache is the other cost. A full window on kimi-k3 holds roughly 62GB of KV state per request, so cache blocks are paged and a prompt's prefix survives between requests. Agents rarely send a million fresh tokens; they send the same 700,000-token prefix plus a new tail. The cached prefix bills at the cached input rate published in the API, and prefill on the tail alone usually returns first tokens in under 3 seconds.
A long context window is a claim about the model. Serving one is a claim about the scheduler.
On price, the catalog rule applies here unchanged. The cheapest public listing we track for kimi-k3 at the full window is $2.60 input and $13.00 output; Nyx serves it at $2.548 and $12.74, two percent under, and the live numbers are always at /v1/models.
The model id is kimi-k3. Point any OpenAI client at
https://nyxprovider.com/v1 and it behaves the way
everything on Nyx does: streaming passes through byte for byte, and
past declared capacity you get an immediate 429 instead of a queue.