Project J.A.I.son

Core server for building AI Companion applications.

About • Features • Applications • Quick Start • Developer Guide • Community • Contributors • License

---

About This Project

Project J.A.I.son is a fully customizable AI companion server designed for streaming, private companionship, or building interactive AI applications. Run it entirely locally or leverage cloud services—the choice is yours.

This software uses libraries from the FFmpeg project under the LGPLv2.1.

---

Key Features

• Real-time AI Personality - Text and speech input with customizable character prompts
• MCP Support - Integrate Model Context Protocol servers for extended capabilities
• REST API & WebSocket Server - Build custom applications on top of the core server
• Flexible Deployment - Run fully local or use cloud services (Azure, OpenAI, Fish Audio, etc.)
• Modular Pipeline - Mix and match STT, TTS, LLM, and filter operations

---

Official Applications

Extend J.A.I.son with these official integrations:

• Discord Bot - Chat with your AI in Discord servers
• VTube Studio Emotions - Trigger VTube Studio expressions based on AI emotions
• Twitch Integration - React to chat and channel events on Twitch

Build your own applications using our Developer Guide!

---

Quick Start

Basic Installation

Windows need to enable Developer Mode

This project uses uv and Make (Windows | Linux (usually provided by the distro)) for setup and development.

# Regular setup
make setup

# Development setup
make dev

Hardware Acceleration

Enable GPU usage by installing PyTorch 2.5.1 for your hardware and OS.

Configuration

1. Copy .env-template to .env and add your API keys for the services you plan to use.

   cp .env-template .env

   If running everything locally, you may leave the keys blank:

   OPENAI_API_KEY=
   FISH_API_KEY=
   AZURE_REGION=
   AZURE_API_KEY=

2. Create new text files under prompts\characters, prompts\instructions, and prompts\scenes, as required. Describe the character/instruction/scene as you'd like. Remember that you can create as many as you'd like and easily switch between them. An example for each is provided.

3. Copy configs\example.yaml to configs\config.yaml (you can name it whatever you'd like). This is the name you'll use when running the main server! Select the desired operations (refer to Development Guide) and set the desired prompt filenames to the ones you created in Step 2:

   prompter:
   instruction_prompt_filename: 'example.txt'
   character_prompt_filename: 'example.txt'
   scene_prompt_filename: 'example.txt'

See the Development Guide for detailed configuration instructions, including:

• Setting up local services (KoboldCPP, MeloTTS, RVC)
• Configuring cloud providers (Azure, OpenAI, Fish Audio)
• Customizing prompts and operations
• Choosing the right services for your use case

---

Running the Server

Inside a terminal inside the project directory

# Example for configs/example.yaml
uv run ./src/main.py -c example

For all available options:

uv run ./src/main.py --help

---

Developer Guide

Build applications, create custom integrations, or extend J.A.I.son:

• Development Guide - Complete configuration reference, service setup, and operations documentation
• REST API Specification - OpenAPI 3.1.0 spec for the REST API and WebSocket events
• Contributing Guidelines - How to contribute to the project

---

Community

Join the community and get help:

• Discord - Project discussions and support
• YouTube - Tutorials and showcases
• Twitch - Live development streams

---

Contributors

Thank you to everyone who has contributed to Project J.A.I.son!

Become a contributor

---

License

MIT