KVBoost: Chunk-Level Key-Value Cache Reuse with Deviation-Guided Recomputation for Efficient Large Language Model Inference

arXiv:2608.21362v1 Announce Type: new Abstract: Transformerbased large language models LLMs incur high prefill latency because keyvalue KV tensors must be recomputed for each request.