#gguf

timtoole02/Camelid

Rust

89

health

Camelid: a Rust-native local inference backend with evidence-gated model compatibility.

zhongkaifu/TensorSharp

C#

89

health

A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, iPhone App, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/iOS/Linux with full GPU capability

intel/auto-round

Python

88

health

A simple and effective post training quantization toolkit for high-accuracy low-bit LLM inference|简洁且高效的后训练量化工具包

enapt/SwarmLLM

Rust

85

health

Free, open-source app that runs AI chat models on your own computer — and lets computers team up over the internet to run models too big for one machine. No account, no crypto. OpenAI- and Anthropic-compatible API.

ariannamethod/notorch

C

82

health

neural networks in pure C

prabanta-dev/janas

C

80

health

Janas-LLM: large mixture-of-experts language models on ordinary computers, in C. Experts streamed from NVMe, with or without a GPU.

ariannamethod/ariannamethod.ai

C

80

health

Arianna Method Programming Language

leehack/llamadart

Dart

77

health

Cross-platform on-device LLM inference for Dart & Flutter: GGUF via llama.cpp and LiteRT-LM on Android, iOS, macOS, Windows, Linux and web.

anthony-chaudhary/fak

Go

73

health

Agentic Runtime