llama.cpp Config

Configuration files and shell functions for running local LLMs with llama.cpp, including MCP tool support and a web search server powered by Gemini.

Prerequisites

• llama.cpp installed and on PATH (llama-server, llama-cli)
• uv (for running the MCP search server and tools)
• Claude Code (optional, for llama-claude)
• Python 3.11+
• GGUF model files (see Models)

Setup

macOS (zsh)

1. Clone the repo:

   git clone <repo-url> ~/Documents/llama.cpp-config

2. Create your mcp.json from the example and add your Gemini API key:

   cp mcp.json.example mcp.json

   Update the path to web-search-mcp.py to match your setup.

3. Source the profile script by adding this line to ~/.zshrc:

   source "$HOME/Documents/llama.cpp-config/llama-profile.sh"

4. Download models — llama.cpp stores them in ~/Library/Caches/llama.cpp/.

Windows (PowerShell)

1. Clone the repo:

   git clone <repo-url> "$HOME\Documents\llamacppconfig"

2. Create your mcp.json from the example and add your Gemini API key:

   cp mcp.json.example mcp.json

   Update the path to web-search-mcp.py to match your username.

3. Source the profile script by adding this line to your PowerShell $PROFILE:

   . "$HOME\Documents\llamacppconfig\llama-profile.ps1"

4. Download models to ~/AppData/Local/llama.cpp/.

Commands

Command	Platform	Description
llama-chat	Both	Starts llama-server with the chat model, MCP proxy, vision support, and opens the web UI
llama-code	Windows	Starts llama-server with the code model (thinking enabled) and opens the web UI
llama-claude	Windows	Points Claude Code at your local llama-server as an OpenAI-compatible backend
llama-test	Windows	Quick CLI test of a model with optional context size and reasoning budget params

Models

Edit the model paths at the top of the profile scripts to match your own models.

• macOS (llama-profile.sh): Qwen 3.5-4B Q8_0 + mmproj
• Windows (llama-profile.ps1): Qwen 3.5-9B Q8_0 + mmproj, OmniCoder-9B

File Overview

File	Description
llama-profile.sh	Zsh functions for macOS — sourced from ~/.zshrc
llama-profile.ps1	PowerShell functions for Windows — dot-sourced from $PROFILE
mcp.json.example	Template for MCP server config (copy to mcp.json)
web-search-mcp.py	MCP server providing web_search and web_fetch tools via Gemini
webui-config.json	System prompt for the llama.cpp web UI