Tag: inference

All the articles with the tag "inference".

• An FP8 KV cache for mini-sglang: what I learned shipping the v1

  17 May, 2026

  Shipping FP8 KV cache for mini-sglang: 2× capacity, up to 27% faster decode, quality intact.