Posts
All the articles I've posted.
-
An FP8 KV cache for mini-sglang: what I learned shipping the v1
Shipping FP8 KV cache for mini-sglang: 2× capacity, up to 27% faster decode, quality intact.
All the articles I've posted.
Shipping FP8 KV cache for mini-sglang: 2× capacity, up to 27% faster decode, quality intact.