Skip to content

feat: full-text search with SQLite FTS5#64

Merged
rsbh merged 9 commits into
mainfrom
feat/fts-search
May 13, 2026
Merged

feat: full-text search with SQLite FTS5#64
rsbh merged 9 commits into
mainfrom
feat/fts-search

Conversation

@rsbh
Copy link
Copy Markdown
Member

@rsbh rsbh commented May 13, 2026

Summary

Replace MiniSearch with SQLite FTS5 via Nitro database for full-text search across docs and API pages.

Commits

  1. Enable Nitro SQLite database — configure experimental database with SQLite connector
  2. Replace MiniSearch with FTS5 — FTS5 virtual table, ranked search, fresh index per deploy
  3. Remove minisearch dependency
  4. Index page body content and headings — read raw MDX files, extract headings + body text, separate FTS columns with bm25 ranking (title 10x, headings 5x, body 1x)
  5. Match type and snippets — results include match field (title/heading/body) with contextual snippet
  6. Readiness endpoint + UI/api/ready returns 503 until index built, search UI shows highlighted snippets with # prefix for headings

Search features

  • Full-text search across page titles, headings, and body content
  • API endpoint search (summary, description, path, method)
  • bm25 ranking: title matches > heading matches > body matches
  • Snippet extraction with match highlighting in accent color
  • Readiness probe (/api/ready) for k8s — lock file in /tmp, deleted on restart
  • Race condition protection with index mutex
  • Client-side filtering disabled (server-side FTS handles it)

Test plan

  • Search for a word in page title — shows as title match
  • Search for a heading — shows with # prefix snippet
  • Search for body text — shows contextual snippet with highlight
  • Search for API operation — finds by summary/description
  • /api/ready returns 503 before first search, 200 after
  • Restart server — /api/ready returns 503 again until re-indexed
  • Search UI renders results with snippets and highlights

🤖 Generated with Claude Code

rsbh and others added 6 commits May 13, 2026 14:08
Configure experimental database with SQLite connector for FTS.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- FTS5 virtual table for ranked full-text search
- Indexes page titles, descriptions, and API operations
- Fresh index on every deploy (drops and recreates tables)
- FTS MATCH with prefix queries for partial matching
- Results ranked by relevance via FTS5 rank function

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replaced by SQLite FTS5 via Nitro database.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Read raw MDX files and extract headings + body text
- Separate FTS columns: title (10x boost), headings (5x), body (1x)
- bm25 ranking for relevance-based results
- Use fs/promises for async file reading

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Response includes match field (title/heading/body)
- Snippet shows the matched text with surrounding context
- Headings stored newline-separated for individual matching
- Body snippets show ~120 chars around the match

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- /api/ready returns 503 until search index is built, 200 after
- Lock file in /tmp deleted on startup for pod restart detection
- Search results show match type (title/heading/body) with snippets
- Matched text highlighted in accent color in both title and snippet
- # prefix for heading matches
- Fixed race condition with index mutex
- Fixed table creation with IF NOT EXISTS
- Pass items to Command to disable client-side filtering

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@vercel
Copy link
Copy Markdown

vercel Bot commented May 13, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
chronicle Ready Ready Preview, Comment May 13, 2026 9:30am

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 13, 2026

Warning

Rate limit exceeded

@rsbh has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 4 minutes and 41 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 22529154-b16b-4f35-91c4-54cc6f7f8bc2

📥 Commits

Reviewing files that changed from the base of the PR and between 7ab1885 and 1df546f.

⛔ Files ignored due to path filters (1)
  • bun.lock is excluded by !**/*.lock
📒 Files selected for processing (8)
  • examples/basic/content/docs/guides/deployment.mdx
  • packages/chronicle/package.json
  • packages/chronicle/src/components/ui/search.module.css
  • packages/chronicle/src/components/ui/search.tsx
  • packages/chronicle/src/lib/source.ts
  • packages/chronicle/src/server/api/ready.ts
  • packages/chronicle/src/server/api/search.ts
  • packages/chronicle/src/server/vite-config.ts
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/fts-search

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Covers build, Docker multi-stage, Docker Compose, Kubernetes
deployment with health/readiness probes, service, ingress,
Vercel and Netlify configs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove unused getFirstApiUrl import
- Suppress useDatabase false positive (Nitro DI, not React hook)
- Suppress empty catch block on lock file cleanup

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@rsbh rsbh merged commit 28eae37 into main May 13, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants