Real-time products that feel instant

Streaming, byte for byte

A chat product lives on the moment between send and the first visible character, so nothing on the Nyx side holds that character back. Tokens leave the server as the model produces them. There is no chunk buffering between the engine and your socket, so the cadence your user sees is the cadence the GPU produces.

Throughput you can design an interface around

Streaming arrives as ordinary server-sent deltas on the OpenAI-compatible chat completions route, so the client code you already have keeps working and the throughput on each model page tell you how fast text fills a bubble. Nothing between the engine and the response body reshapes the stream, so what your client measures is what the GPU produced.

Overload arrives as a 429

Past declared capacity the API returns 429 immediately with a Retry-After header. Requests do not sit in a hidden queue waiting for a slot, so a saturated minute reaches your retry logic as an error you can act on rather than a connection that hangs open. Your backoff decides what happens next.

Dedicated pods when traffic cannot share a queue

Shared capacity is priced per token and sized against declared limits. When your launch traffic cannot share a queue with anyone, dedicated pods run your model alone at a flat hourly price, with the whole GPU decoding for you.