#llmops

comet-ml/opik

Python

89

health

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

fkiene/llmtrim

Rust

88

health

Local proxy that compresses your LLM API requests so you pay less, with no change to the answers. Trims wasted tokens from prompts, history, tool output, and code before they're sent: -31% input / -74% output, measured live. Any provider, no extra model calls. Also an MCP server and embeddable library (Rust, Python, Ruby, Kotlin, Swift, JS/TS).

maximhq/bifrost

Go

87

health

Fastest enterprise AI gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k RPS.

traceloop/openllmetry

Python

86

health

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

vllm-project/vllm-ascend

C++

83

health

Community maintained hardware plugin for vLLM on Ascend