README

Xberg

liter-llm

A lighter, faster, safer universal LLM API client — one Rust core, 14 native language bindings, 143 providers.

What and Why?

liter-llm is a universal LLM API client compiled from the ground up in Rust: one core, 14 native language bindings, and 143 providers. No interpreter, no transitive dependency tree, no supply-chain surface area — and a drop-in OpenAI-compatible proxy plus an MCP server in a single 35 MB binary.

Compiled Rust core — no pip install supply chain, no .pth auto-execution hooks, no runtime dependency tree to compromise.
Secrets stay secret — API keys are wrapped in secrecy::SecretString: zeroed on drop, redacted in logs, never serialized.
Polyglot from day one — Python, TypeScript, Go, Java, Kotlin, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, and WebAssembly, all thin wrappers over the same Rust core, plus a C/FFI surface for everything else.
Observability built in — production-grade OpenTelemetry with GenAI semantic conventions, not an afterthought callback system.
Composable middleware — rate limiting, caching, cost tracking, health checks, and fallback as Tower layers you stack like building blocks.

We credit litellm for proving the category; our provider registry was bootstrapped from theirs. See ATTRIBUTIONS.md.

Features

Feature	Description
143 providers	OpenAI, Anthropic, Google, AWS Bedrock, Groq, Mistral, Together, Fireworks, DeepSeek, Cohere, and 130+ more — compiled at build time
14 native bindings	Rust, Python, Node.js, Go, Java, Kotlin, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, WebAssembly — plus a shared C/FFI surface
First-class streaming	SSE and AWS EventStream binary protocol with zero-copy buffers
Proxy & MCP server	Drop-in OpenAI-compatible proxy (22 endpoints) and MCP tool server in a 35 MB Docker image
Tower middleware	Rate limiting, caching (40+ OpenDAL backends), cost tracking, budget enforcement, health checks, and fallback — all composable
Observability	OpenTelemetry with GenAI semantic conventions, cost-tracking spans, and HTTP-level tracing
Tool calling	Parallel tools, structured outputs, and JSON-schema validation
Search & OCR	Web search across 12 providers, document OCR across 4
TOML configuration	`liter-llm.toml` auto-discovery, custom providers, cache backends, and middleware config
Local LLM support	Ollama, LM Studio, vLLM, llama.cpp, LocalAI, and llamafile via OpenAI-compatible APIs

⭐ Star this repo to show your support — it helps others discover liter-llm.

Quick Start

Language Packages

Python

pip install liter-llm

See Python README for full documentation.

Node.js

pnpm add @xberg-io/liter-llm

See Node.js README for full documentation.

Rust

cargo add liter-llm

See Rust crate for full documentation.

go get github.com/xberg-io/liter-llm/packages/go

See Go README for full documentation.

Java

Available on Maven Central as io.xberg.literllm:liter-llm. See Java README for the dependency snippet and current version.

Ruby

gem install liter_llm

See Ruby README for full documentation.

PHP

composer require xberg-io/liter-llm

See PHP README for full documentation.

dotnet add package LiterLlm

See .NET README for full documentation.

Elixir

Add {:liter_llm, "~> 1.6"} to your mix.exs dependencies. See Elixir README for full documentation.

Dart / Flutter

dart pub add liter_llm

See Dart README for full documentation.

Swift

Add via Swift Package Manager. See Swift README for full documentation.

Kotlin (Android)

Available on Maven Central as io.xberg.literllm:liter-llm-android. See Kotlin README for the dependency snippet and current version.

Zig

See Zig README for installation and usage.

WebAssembly

pnpm add @xberg-io/liter-llm-wasm

See WebAssembly README for full documentation.

C/C++ (FFI)

Build from source as part of this workspace. See FFI crate for full documentation.

CLI, Proxy & MCP Server

The liter-llm CLI ships both the OpenAI-compatible proxy and the MCP tool server. Install it any of these ways:

brew install xberg-io/tap/liter-llm
cargo install liter-llm-cli                 # from crates.io
npx @xberg-io/liter-llm-cli --help         # npm (self-installs the binary)
docker run -p 4000:4000 -e LITER_LLM_MASTER_KEY=sk-your-key ghcr.io/xberg-io/liter-llm

Then run the proxy or the MCP server:

liter-llm api --config liter-llm-proxy.toml   # OpenAI-compatible proxy (22 endpoints)
liter-llm mcp --transport stdio               # MCP tool server (stdio)
liter-llm mcp --transport http --port 3001    # MCP tool server (Streamable HTTP)

See the MCP server guide and the proxy guide for transports, routing, virtual keys, and budgets. To use the MCP server inside a coding agent, install the liter-llm plugin (below) — it auto-registers the server, no manual config required.

AI Coding Assistants

Install the liter-llm plugin from the xberg-io/plugins marketplace. It ships the liter-llm agent skills (chat, streaming, tools, embeddings across 143 providers) and works with every major coding agent — expand your harness below.

Claude Code

/plugin marketplace add xberg-io/plugins
/plugin install liter-llm@xberg

Codex CLI

/plugins add https://github.com/xberg-io/plugins

Then search for liter-llm and select Install Plugin.

Cursor

Settings → Plugins → Add from URL → https://github.com/xberg-io/plugins, then select liter-llm.

Gemini CLI

gemini extensions install https://github.com/xberg-io/plugins

Factory Droid

droid plugin marketplace add https://github.com/xberg-io/plugins
droid plugin install liter-llm@xberg

GitHub Copilot CLI

copilot plugin marketplace add https://github.com/xberg-io/plugins
copilot plugin install liter-llm@xberg

opencode

Not yet published as an opencode package. Install via any harness above (self-hosted marketplace); opencode support is tracked in xberg-io/plugins.

Documentation

Full guides, the unified chat() API for every binding, multimodal I/O, the proxy/gateway, and the complete provider list live at docs.liter-llm.xberg.io.

Contributing

Contributions are welcome! See CONTRIBUTING.md for guidelines.

Join our Discord community for questions and discussion.

Part of Xberg.io

Xberg — document intelligence: text, tables, metadata from 91+ formats with optional OCR.
Xberg Enterprise — managed extraction API with SDKs, dashboards, and observability.
crawlberg — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
html-to-markdown — fast, lossless HTML→Markdown engine.
liter-llm — universal LLM API client with native bindings for 14 languages and 143 providers.
tree-sitter-language-pack — tree-sitter grammars and code-intelligence primitives.
alef — the polyglot binding generator that produces every per-language binding across the 5 polyglot repos.

License

MIT — see LICENSE for details.

xberg-io/liter-llm

包简介

关键字：

README 文档