Open source repositories tagged with #inference, ranked by health score.
OpenVINO™ is an open source toolkit for optimizing and deploying AI inference
A unified inference and post-training framework for accelerated video generation.
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
Pure C inference engine for Qwen3-TTS text-to-speech. No Python, no PyTorch — just C and BLAS. Supports 0.6B and 1.7B models, 9 voices, 10 languages.
Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
A fast, lightweight, and extensible RWKV chat UI powered by Flutter. Offline-ready, multi-backend support, ideal for local RWKV inference.
Community maintained hardware plugin for vLLM on Ascend
On-device Speech AI for Apple Silicon
