Skip to content
#

tensorrtllm

Star

Here is 1 public repository matching this topic...

lightseekorg / smg

Star 127

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across SGLang, vLLM, TRT-LLM, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

chat mcp routing gemini openai claude llm anthropic vllm sglang anthropic-api inference-gateway responses-api tensorrtllm trtllm lightseek
  • Updated Mar 27, 2026
  • Rust

Improve this page

Add a description, image, and links to the tensorrtllm topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the tensorrtllm topic, visit your repo's landing page and select "manage topics."

Learn more