Open source repositories tagged with #inference-server, ranked by health score.
LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
ONNX Runtime Server: The ONNX Runtime Server is a server that provides TCP and HTTP/HTTPS REST APIs for ONNX inference.
