Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/speko-plugin.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@livekit/agents-plugin-speko": patch
---

Add the Speko STT, LLM, and TTS plugin.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ Currently, only the following plugins are supported:
| [@livekit/agents-plugin-lemonslice](https://www.npmjs.com/package/@livekit/agents-plugin-lemonslice) | Avatar |
| [@livekit/agents-plugin-liveavatar](https://www.npmjs.com/package/@livekit/agents-plugin-liveavatar) | Avatar |
| [@livekit/agents-plugin-mistralai](https://www.npmjs.com/package/@livekit/agents-plugin-mistralai) | LLM, STT, TTS |
| [@livekit/agents-plugin-speko](https://www.npmjs.com/package/@livekit/agents-plugin-speko) | STT, LLM, TTS |
| [@livekit/agents-plugin-xai](https://www.npmjs.com/package/@livekit/agents-plugin-xai) | LLM, TTS |
| [@livekit/agents-plugin-phonic](https://www.npmjs.com/package/@livekit/agents-plugin-phonic) | Realtime |
| [@livekit/agents-plugin-fishaudio](https://www.npmjs.com/package/@livekit/agents-plugin-fishaudio) | TTS |
Expand Down
101 changes: 101 additions & 0 deletions plugins/speko/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
<!--
SPDX-FileCopyrightText: 2026 LiveKit, Inc.

SPDX-License-Identifier: Apache-2.0
-->

# Speko plugin for LiveKit Agents

The Agents Framework is designed for building realtime, programmable
participants that run on servers. Use it to create conversational, multi-modal
voice agents that can see, hear, and understand.

This package contains the Speko plugin for LiveKit Agents. Speko provides a
single STT, LLM, and TTS router for voice agents, selecting providers and
handling failover server-side so your LiveKit worker does not need separate
provider credentials.

## Installation

```sh
pnpm add @livekit/agents @livekit/agents-plugin-speko \
@livekit/agents-plugin-silero @livekit/rtc-node
```

Set `SPEKO_API_KEY` in the environment before starting your worker.
If you need a non-default Speko API host, pass `baseURL` or set
`SPEKO_BASE_URL`.

## Usage

```ts
import {
type JobContext,
type JobProcess,
ServerOptions,
cli,
defineAgent,
voice,
} from '@livekit/agents';
import * as silero from '@livekit/agents-plugin-silero';
import * as speko from '@livekit/agents-plugin-speko';
import { fileURLToPath } from 'node:url';

export default defineAgent({
prewarm: async (proc: JobProcess) => {
proc.userData.vad = await silero.VAD.load();
},
entry: async (ctx: JobContext) => {
const vad = ctx.proc.userData.vad as silero.VAD;
const intent = {
language: 'en-US',
optimizeFor: 'balanced',
} as const;

const session = new voice.AgentSession({
vad,
stt: new speko.STT({ intent }),
llm: new speko.LLM({ intent }),
tts: new speko.TTS({ intent }),
});

await session.start({
agent: new voice.Agent({
instructions: 'You are a helpful voice assistant. Be concise.',
}),
room: ctx.room,
});

await ctx.connect();

session.generateReply({
instructions: 'Greet the user and offer your assistance.',
});
},
});

cli.runApp(
new ServerOptions({
agent: fileURLToPath(import.meta.url),
agentName: 'speko-demo',
}),
);
```

`intent` is required on each raw component because Speko uses it to route every
STT, LLM, and TTS call by language, region, and optimization preference.
`voice.AgentSession` automatically wraps non-streaming STT and TTS plugins with
LiveKit stream adapters when needed.

## Limitations

- STT is utterance-bounded. `speko.STT` uploads one VAD-bounded WAV per
recognition call. Use `voice.AgentSession` with a VAD such as Silero, or wrap
manually with `stt.StreamAdapter` if you implement a custom STT node.
- TTS is sentence-bounded. `voice.AgentSession` wraps `speko.TTS` with a
sentence tokenizer by default, or you can wrap it manually with
`tts.StreamAdapter` if you implement a custom TTS node.
- A Speko API key is required. Pass `apiKey`, set `SPEKO_API_KEY`, or pass a
preconfigured SDK `client`.
- TTS output must be PCM or WAV. The plugin accepts `audio/pcm;rate=NNNN` and
`audio/wav`; compressed formats such as MP3 are rejected.
20 changes: 20 additions & 0 deletions plugins/speko/api-extractor.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
/**
* Config file for API Extractor. For more info, please visit: https://api-extractor.com
*/
{
"$schema": "https://developer.microsoft.com/json-schemas/api-extractor/v7/api-extractor.schema.json",

/**
* Optionally specifies another JSON config file that this file extends from. This provides a way for
* standard settings to be shared across multiple projects.
*
* If the path starts with "./" or "../", the path is resolved relative to the folder of the file that contains
* the "extends" field. Otherwise, the first path segment is interpreted as an NPM package name, and will be
* resolved using NodeJS require().
*
* SUPPORTED TOKENS: none
* DEFAULT VALUE: ""
*/
"extends": "../../api-extractor-shared.json",
"mainEntryPointFilePath": "./dist/index.d.ts"
}
128 changes: 128 additions & 0 deletions plugins/speko/etc/agents-plugin-speko.api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
## API Report File for "@livekit/agents-plugin-speko"

> Do not edit this file. It is a report generated by [API Extractor](https://api-extractor.com/).

```ts

import { APIConnectOptions } from '@livekit/agents';
import type { AudioBuffer as AudioBuffer_2 } from '@livekit/agents';
import type { ChatMessage } from '@spekoai/sdk';
import { llm } from '@livekit/agents';
import type { OptimizeFor } from '@spekoai/sdk';
import type { PipelineConstraints } from '@spekoai/sdk';
import type { RoutingIntent } from '@spekoai/sdk';
import { Speko } from '@spekoai/sdk';
import { stt } from '@livekit/agents';
import type { SynthesizeResult } from '@spekoai/sdk';
import { tts } from '@livekit/agents';

// @public
export function chatContextToSpeko(ctx: llm.ChatContext): ChatMessage[];

// @public
export function decodeSynthesisResult(result: SynthesizeResult): {
pcm: Uint8Array;
sampleRate: number;
channels: number;
};

// @public
export function framesToWav(buffer: AudioBuffer_2): Uint8Array;

// @public
export type Intent = RoutingIntent;

// @public
export class LLM extends llm.LLM {
constructor(options: LLMOptions);
chat(params: {
chatCtx: llm.ChatContext;
toolCtx?: llm.ToolContext;
connOptions?: APIConnectOptions;
parallelToolCalls?: boolean;
toolChoice?: llm.ToolChoice;
extraKwargs?: Record<string, unknown>;
}): llm.LLMStream;
label(): string;
get model(): string;
get provider(): string;
}

// @public
export interface LLMOptions extends SpekoClientOptions {
constraints?: PipelineConstraints;
intent: Intent;
maxTokens?: number;
temperature?: number;
}

export { OptimizeFor }

// @public
export function parseWav(bytes: Uint8Array): {
pcm: Uint8Array;
sampleRate: number;
channels: number;
};

// @public
export function pcmSampleRateFromContentType(contentType: string, fallback: number): number;

// @public
export interface SpekoClientOptions {
apiKey?: string;
baseURL?: string;
client?: Speko;
timeout?: number;
}

// @public
export class SpekoPluginError extends Error {
constructor(message: string, code: string);
readonly code: string;
}

// @public
export class STT extends stt.STT {
constructor(options: STTOptions);
label: string;
get model(): string;
get provider(): string;
protected _recognize(frame: AudioBuffer_2, abortSignal?: AbortSignal): Promise<stt.SpeechEvent>;
stream(_options?: {
connOptions?: APIConnectOptions;
}): stt.SpeechStream;
}

// @public
export interface STTOptions extends SpekoClientOptions {
constraints?: PipelineConstraints;
intent: Intent;
keywords?: readonly string[];
}

// @public
export class TTS extends tts.TTS {
constructor(options: TTSOptions);
label: string;
get model(): string;
get provider(): string;
stream(_options?: {
connOptions?: APIConnectOptions;
}): tts.SynthesizeStream;
synthesize(text: string, connOptions?: APIConnectOptions, abortSignal?: AbortSignal): tts.ChunkedStream;
}

// @public
export interface TTSOptions extends SpekoClientOptions {
constraints?: PipelineConstraints;
intent: Intent;
sampleRate?: number;
speed?: number;
voice?: string;
}

// @public
export function validateIntent(intent: Intent): void;

```
52 changes: 52 additions & 0 deletions plugins/speko/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
{
"name": "@livekit/agents-plugin-speko",
"version": "1.4.4",
"description": "Speko plugin for LiveKit Node Agents",
"main": "dist/index.js",
"require": "dist/index.cjs",
"types": "dist/index.d.ts",
"exports": {
"import": {
"types": "./dist/index.d.ts",
"default": "./dist/index.js"
},
"require": {
"types": "./dist/index.d.cts",
"default": "./dist/index.cjs"
}
},
"author": "LiveKit",
"type": "module",
"repository": "git@github.com:livekit/agents-js.git",
"license": "Apache-2.0",
"files": [
"dist",
"src",
"README.md"
],
"scripts": {
"build": "tsup --onSuccess \"pnpm build:types\"",
"build:types": "tsc --declaration --emitDeclarationOnly && node ../../scripts/copyDeclarationOutput.js",
"clean": "rm -rf dist",
"clean:build": "pnpm clean && pnpm build",
"lint": "eslint -f unix \"src/**/*.{ts,js}\"",
"api:check": "api-extractor run --typescript-compiler-folder ../../node_modules/typescript",
"api:update": "api-extractor run --local --typescript-compiler-folder ../../node_modules/typescript --verbose"
},
"devDependencies": {
"@livekit/agents": "workspace:*",
"@livekit/agents-plugin-silero": "workspace:*",
"@livekit/agents-plugins-test": "workspace:*",
"@livekit/rtc-node": "catalog:",
"@microsoft/api-extractor": "^7.35.0",
"tsup": "^8.3.5",
"typescript": "^5.0.0"
},
"dependencies": {
"@spekoai/sdk": "^0.4.1"
},
"peerDependencies": {
"@livekit/agents": "workspace:*",
"@livekit/rtc-node": "catalog:"
}
}
Loading
Loading