Open source repositories tagged with #llm-serving, ranked by health score.
Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
Community maintained hardware plugin for vLLM on Ascend
