feat(web): improve agent performance, MCP tools, and much more

.env.development

@@ -76,3 +76,5 @@ SOURCEBOT_TELEMETRY_DISABLED=true # Disables telemetry collection
 # CONFIG_MAX_REPOS_NO_TOKEN=
 NODE_ENV=development
 # SOURCEBOT_TENANCY_MODE=single
+
+DEBUG_WRITE_CHAT_MESSAGES_TO_FILE=true

CHANGELOG.md

@@ -9,6 +9,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Changed
 - Changed language detection to resolve file extensions with multiple language resolutions (e.g., .md) to the most common resolution. [#1026](https://github.com/sourcebot-dev/sourcebot/pull/1026)
+- Changed the `webUrl` property of the `/api/repos` api to return a URL rather than just a path. [#1014](https://github.com/sourcebot-dev/sourcebot/pull/1014)
+- Changed the ask search scope selector to allow submitting questions with no search scope selected. When no selection is made, the agent will be able to search over all repos the user has access to. [#1014](https://github.com/sourcebot-dev/sourcebot/pull/1014)
+- Renamed the `search_code` tool to `grep` for ask and mcp. [#1014](https://github.com/sourcebot-dev/sourcebot/pull/1014)
+
+### Added
+- Added `glob`, `find_symbol_definitions`, and `find_symbol_references` tools to the ask agent and MCP server. [#1014](https://github.com/sourcebot-dev/sourcebot/pull/1014)
+- Added `list_tree` tool to the ask agent. [#1014](https://github.com/sourcebot-dev/sourcebot/pull/1014)
+- Added input & output token breakdown in ask details card. [#1014](https://github.com/sourcebot-dev/sourcebot/pull/1014)
+- Added `path` parameter to the `/api/commits` api to allow filtering commits by paths. [#1014](https://github.com/sourcebot-dev/sourcebot/pull/1014)
+
+### Fixed
+- Fixed issue where ask responses would sometimes appear in the details panel while generating. [#1014](https://github.com/sourcebot-dev/sourcebot/pull/1014)
+- Fixed reference panel overflow issue in the ask UI. [#1014](https://github.com/sourcebot-dev/sourcebot/pull/1014)
+- Fixed homepage scrolling issue in the ask UI. [#1014](https://github.com/sourcebot-dev/sourcebot/pull/1014)
 
 ## [4.15.11] - 2026-03-20
 

CLAUDE.md

@@ -38,6 +38,25 @@ Exceptions:
 - Special files like `README.md`, `CHANGELOG.md`, `LICENSE`
 - Next.js conventions: `page.tsx`, `layout.tsx`, `loading.tsx`, etc.
 
+## Code Style
+
+Always use curly braces for `if` statements, with the body on a new line — even for single-line bodies:
+
+```ts
+// Correct
+if (!value) {
+    return;
+}
+if (condition) {
+    doSomething();
+}
+
+// Incorrect
+if (!value) return;
+if (!value) { return; }
+if (condition) doSomething();
+```
+
 ## Tailwind CSS
 
 Use Tailwind color classes directly instead of CSS variable syntax:

docs/docs/features/mcp-server.mdx

@@ -304,22 +304,19 @@ Pass the key as an `Authorization: Bearer <key>` header when connecting to the M
 
 ## Available Tools
 
-### `search_code`
+### `grep`
 
-Searches for code that matches the provided search query as a substring by default, or as a regular expression if `useRegex` is true.
+Searches for code matching a regular expression pattern across repositories, similar to `grep`/`ripgrep`. Always case-sensitive. Results are grouped by file and include line numbers.
 
 Parameters:
 | Name | Required | Description |
-|:----------------------|:---------|:---------------------------------------------------------------------------------------------------------------------|
-| `query` | yes | The search pattern to match against code contents. Do not escape quotes in your query. |
-| `useRegex` | no | Whether to use regular expression matching. When false, substring matching is used (default: false). |
-| `filterByRepos` | no | Scope the search to specific repositories. |
-| `filterByLanguages` | no | Scope the search to specific languages. |
-| `filterByFilepaths` | no | Scope the search to specific filepaths. |
-| `caseSensitive` | no | Whether the search should be case sensitive (default: false). |
-| `includeCodeSnippets` | no | Whether to include code snippets in the response (default: false). |
+|:----------|:---------|:--------------------------------------------------------------------------------------------------------------|
+| `pattern` | yes | The regex pattern to search for in file contents. |
+| `path` | no | Directory path to scope the search to. Defaults to the repository root. |
+| `include` | no | File glob pattern to include in the search (e.g. `*.ts`, `*.{ts,tsx}`). |
+| `repo` | no | Repository name to search in. If not provided, searches all repositories. Use the full name including host (e.g. `github.com/org/repo`). |
 | `ref` | no | Commit SHA, branch or tag name to search on. If not provided, defaults to the default branch. |
-| `maxTokens` | no | The maximum number of tokens to return (default: 10000). |
+| `limit` | no | Maximum number of matching files to return (default: 100). |
 
 ### `list_repos`
 
@@ -336,18 +333,20 @@ Parameters:
 
 ### `read_file`
 
-Reads the source code for a given file.
+Reads the source code for a given file, with optional line range control for large files.
 
 Parameters:
 | Name | Required | Description |
-|:-------|:---------|:-------------------------------------------------------------------------------------------------------|
+|:---------|:---------|:-------------------------------------------------------------------------------------------------------|
 | `repo` | yes | The repository name. |
 | `path` | yes | The path to the file. |
 | `ref` | no | Commit SHA, branch or tag name to fetch the source code for. If not provided, uses the default branch. |
+| `offset` | no | Line number to start reading from (1-indexed). Omit to start from the beginning. |
+| `limit` | no | Maximum number of lines to read (max: 500). Omit to read up to 500 lines. |
 
 ### `list_tree`
 
-Lists files and directories from a repository path. Can be used as a directory listing tool (`depth: 1`) or a repo-tree tool (`depth > 1`).
+Lists files and directories from a repository path. Directories are shown before files at each level.
 
 Parameters:
 | Name | Required | Description |
@@ -376,6 +375,39 @@ Parameters:
 | `page` | no | Page number for pagination (min 1, default: 1). |
 | `perPage` | no | Results per page for pagination (min 1, max 100, default: 50). |
 
+### `glob`
+
+Finds files whose paths match a glob pattern across repositories (e.g. `**/*.ts`, `src/**/*.test.{ts,tsx}`). Results are grouped by repository.
+
+Parameters:
+| Name | Required | Description |
+|:----------|:---------|:------------|
+| `pattern` | yes | Glob pattern to match file paths against (e.g. `**/*.ts`, `src/**/*.test.{ts,tsx}`). |
+| `path` | no | Restrict results to files under this subdirectory. |
+| `repo` | no | Repository name to search in. If not provided, searches all repositories. Use the full name including host (e.g. `github.com/org/repo`). |
+| `ref` | no | Commit SHA, branch or tag name to search on. If not provided, defaults to the default branch. |
+| `limit` | no | Maximum number of files to return (default: 100). |
+
+### `find_symbol_definitions`
+
+Finds where a symbol (function, class, variable, etc.) is defined in a repository.
+
+Parameters:
+| Name | Required | Description |
+|:---------|:---------|:------------|
+| `symbol` | yes | The symbol name to find definitions of. |
+| `repo` | yes | Repository name to scope the search to. Use the full name including host (e.g. `github.com/org/repo`). |
+
+### `find_symbol_references`
+
+Finds all usages of a symbol (function, class, variable, etc.) across a repository.
+
+Parameters:
+| Name | Required | Description |
+|:---------|:---------|:------------|
+| `symbol` | yes | The symbol name to find references to. |
+| `repo` | yes | Repository name to scope the search to. Use the full name including host (e.g. `github.com/org/repo`). |
+
 ### `list_language_models`
 
 Lists the available language models configured on the Sourcebot instance. Use this to discover which models can be specified when calling `ask_codebase`.

packages/shared/src/logger.ts

@@ -32,12 +32,11 @@ const datadogFormat = format((info) => {
     return info;
 });
 
-const humanReadableFormat = printf(({ level, message, timestamp, stack, label: _label }) => {
+const humanReadableFormat = printf(({ level, message, timestamp, stack, label: _label, ...rest }) => {
     const label = `[${_label}] `;
-    if (stack) {
-        return `${timestamp} ${level}: ${label}${message}\n${stack}`;
-    }
-    return `${timestamp} ${level}: ${label}${message}`;
+    const extras = Object.keys(rest).length > 0 ? ` ${JSON.stringify(rest)}` : '';
+    const base = `${timestamp} ${level}: ${label}${message}${extras}`;
+    return stack ? `${base}\n${stack}` : base;
 });
 
 const createLogger = (label: string) => {

packages/web/next.config.mjs

@@ -63,7 +63,14 @@ const nextConfig = {
         ]
     },
 
-    turbopack: {},
+    turbopack: {
+        rules: {
+            '*.txt': {
+                loaders: ['raw-loader'],
+                as: '*.js',
+            },
+        },
+    },
 
     // @see: https://github.com/vercel/next.js/issues/58019#issuecomment-1910531929
     ...(process.env.NODE_ENV === 'development' ? {

packages/web/package.json

@@ -144,6 +144,7 @@
     "escape-string-regexp": "^5.0.0",
     "fast-deep-equal": "^3.1.3",
     "fuse.js": "^7.0.0",
+    "glob-to-regexp": "^0.4.1",
     "google-auth-library": "^10.1.0",
     "graphql": "^16.9.0",
     "http-status-codes": "^2.3.0",
@@ -189,6 +190,7 @@
     "stripe": "^17.6.0",
     "tailwind-merge": "^2.5.2",
     "tailwindcss-animate": "^1.0.7",
+    "use-stick-to-bottom": "^1.1.3",
     "usehooks-ts": "^3.1.0",
     "vscode-icons-js": "^11.6.1",
     "zod": "^3.25.74",
@@ -202,6 +204,7 @@
     "@tanstack/eslint-plugin-query": "^5.74.7",
     "@testing-library/dom": "^10.4.1",
     "@testing-library/react": "^16.3.0",
+    "@types/glob-to-regexp": "^0.4.4",
     "@types/micromatch": "^4.0.9",
     "@types/node": "^20",
     "@types/nodemailer": "^6.4.17",
@@ -218,6 +221,7 @@
     "jsdom": "^25.0.1",
     "npm-run-all": "^4.1.5",
     "postcss": "^8",
+    "raw-loader": "^4.0.2",
     "react-email": "^5.1.0",
     "react-grab": "^0.1.23",
     "tailwindcss": "^3.4.1",

packages/web/src/app/[domain]/askgh/[owner]/[repo]/components/landingPage.tsx

@@ -77,7 +77,6 @@ export const LandingPage = ({
                             languageModels={languageModels}
                             selectedSearchScopes={selectedSearchScopes}
                             searchContexts={[]}
-                            onContextSelectorOpenChanged={setIsContextSelectorOpen}
                             isDisabled={isChatBoxDisabled}
                         />
                         <Separator />

packages/web/src/app/[domain]/chat/components/landingPageChatBox.tsx

@@ -42,7 +42,6 @@ export const LandingPageChatBox = ({
                     languageModels={languageModels}
                     selectedSearchScopes={selectedSearchScopes}
                     searchContexts={searchContexts}
-                    onContextSelectorOpenChanged={setIsContextSelectorOpen}
                     isDisabled={isChatBoxDisabled}
                 />
                 <Separator />

packages/web/src/app/[domain]/chat/page.tsx

@@ -73,7 +73,7 @@ export default async function Page(props: PageProps) {
     })() : undefined;
 
     return (
-        <div className="flex flex-col items-center min-h-screen overflow-hidden">
+        <div className="flex flex-col items-center h-screen overflow-hidden">
             <NavigationMenu
                 domain={params.domain}
             />
@@ -88,12 +88,13 @@ export default async function Page(props: PageProps) {
                     isCollapsedInitially={true}
                 />
                 <AnimatedResizableHandle />
-                <ResizablePanel 
+                <ResizablePanel
                     order={2}
                     id="chat-home-panel"
                     defaultSize={85}
+                    className="overflow-hidden"
                 >
-                <div className="flex flex-col justify-center items-center mt-8 mb-8 md:mt-16 w-full px-5">
+                <div className="flex flex-col items-center h-full overflow-y-auto pt-8 pb-8 md:pt-16 w-full px-5">
                     <div className="max-h-44 w-auto">
                         <SourcebotLogo
                             className="h-18 md:h-40 w-auto"

packages/web/src/app/api/(server)/commits/route.ts

@@ -13,6 +13,7 @@ const listCommitsQueryParamsSchema = z.object({
     until: z.string().optional(),
     author: z.string().optional(),
     ref: z.string().optional(),
+    path: z.string().optional(),
     page: z.coerce.number().int().positive().default(1),
     perPage: z.coerce.number().int().positive().max(100).default(50),
 });
@@ -57,6 +58,7 @@ export const GET = apiHandler(async (request: NextRequest): Promise<Response> =>
             ...(searchParams.until ? { until: searchParams.until } : {}),
             ...(searchParams.author ? { author: searchParams.author } : {}),
             ...(searchParams.ref ? { ref: searchParams.ref } : {}),
+            ...(searchParams.path ? { path: searchParams.path } : {}),
         },
     });
     if (linkHeader) headers.set('Link', linkHeader);

packages/web/src/features/chat/agent.ts

@@ -14,10 +14,10 @@ import {
 } from "ai";
 import { randomUUID } from "crypto";
 import _dedent from "dedent";
-import { ANSWER_TAG, FILE_REFERENCE_PREFIX, toolNames } from "./constants";
-import { createCodeSearchTool, findSymbolDefinitionsTool, findSymbolReferencesTool, listCommitsTool, listReposTool, readFilesTool } from "./tools";
+import { ANSWER_TAG, FILE_REFERENCE_PREFIX } from "./constants";
 import { Source } from "./types";
 import { addLineNumbers, fileReferenceToString } from "./utils";
+import { createTools } from "./tools";
 
 const dedent = _dedent.withOptions({ alignValues: true });
 
@@ -198,14 +198,7 @@ const createAgentStream = async ({
         providerOptions,
         messages: inputMessages,
         system: systemPrompt,
-        tools: {
-            [toolNames.searchCode]: createCodeSearchTool(selectedRepos),
-            [toolNames.readFiles]: readFilesTool,
-            [toolNames.findSymbolReferences]: findSymbolReferencesTool,
-            [toolNames.findSymbolDefinitions]: findSymbolDefinitionsTool,
-            [toolNames.listRepos]: listReposTool,
-            [toolNames.listCommits]: listCommitsTool,
-        },
+        tools: createTools({ source: 'sourcebot-ask-agent', selectedRepos }),
         temperature: env.SOURCEBOT_CHAT_MODEL_TEMPERATURE,
         stopWhen: [
             stepCountIsGTE(env.SOURCEBOT_CHAT_MAX_STEP_COUNT),
@@ -223,40 +216,7 @@ const createAgentStream = async ({
                     return;
                 }
 
-                if (toolName === toolNames.readFiles) {
-                    output.forEach((file) => {
-                        onWriteSource({
-                            type: 'file',
-                            language: file.language,
-                            repo: file.repository,
-                            path: file.path,
-                            revision: file.revision,
-                            name: file.path.split('/').pop() ?? file.path,
-                        });
-                    });
-                } else if (toolName === toolNames.searchCode) {
-                    output.files.forEach((file) => {
-                        onWriteSource({
-                            type: 'file',
-                            language: file.language,
-                            repo: file.repository,
-                            path: file.fileName,
-                            revision: file.revision,
-                            name: file.fileName.split('/').pop() ?? file.fileName,
-                        });
-                    });
-                } else if (toolName === toolNames.findSymbolDefinitions || toolName === toolNames.findSymbolReferences) {
-                    output.forEach((file) => {
-                        onWriteSource({
-                            type: 'file',
-                            language: file.language,
-                            repo: file.repository,
-                            path: file.fileName,
-                            revision: file.revision,
-                            name: file.fileName.split('/').pop() ?? file.fileName,
-                        });
-                    });
-                }
+                output.sources?.forEach(onWriteSource);
             });
         },
         experimental_telemetry: {
@@ -312,6 +272,10 @@ const createPrompt = ({
         <selected_repositories>
         The user has explicitly selected the following repositories for analysis:
         ${repos.map(repo => `- ${repo}`).join('\n')}
+
+        When calling tools that accept a \`repo\` parameter (e.g. \`read_file\`, \`list_commits\`, \`list_tree\`, \`grep\`), use these repository names exactly as listed above, including the full host prefix (e.g. \`github.com/org/repo\`).
+
+        When using \`grep\` to search across ALL selected repositories (e.g. "which repos have X?"), omit the \`repo\` parameter entirely — the tool will automatically search across all selected repositories in a single call. Do NOT call \`grep\` once per repository when a single broad search would suffice.
         </selected_repositories>
     ` : ''}