#vllm

LMCache/LMCache

Python

89

health

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

vllm-project/semantic-router

Go

89

health

System Level Intelligent Router for Mixture-of-Models at Cloud, Data Center and Edge

kvcache-ai/Mooncake

C++

88

health

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

vllm-project/vllm-ascend

C++

83

health

Community maintained hardware plugin for vLLM on Ascend