Skip to content

feat(saas): SaaS database migration - PostgreSQL backend + visualizat…#303

Open
LinShiG0ng wants to merge 3 commits intovolcengine:mainfrom
LinShiG0ng:claude/saas-database-migration-YohvE
Open

feat(saas): SaaS database migration - PostgreSQL backend + visualizat…#303
LinShiG0ng wants to merge 3 commits intovolcengine:mainfrom
LinShiG0ng:claude/saas-database-migration-YohvE

Conversation

@LinShiG0ng
Copy link

…ion dashboard

Changes

New PostgreSQL + pgvector Storage Backend

  • openviking/storage/postgresql_backend.py: Full implementation of VikingDBInterface using asyncpg + PostgreSQL pgvector extension. Supports all CRUD, vector similarity search, scalar filter DSL translation to SQL, scroll pagination, and batch operations.
  • openviking/storage/postgresql_manager.py: Manager wrapper with queue injection interface matching VikingDBManager, enabling transparent swap in OpenVikingService.

Configuration

  • openviking_cli/utils/config/vectordb_config.py: Added postgresql backend type and PostgreSQLConfig model (host/port/db/user/password/dsn).
  • pyproject.toml: Added asyncpg>=0.29.0 dependency.

Service Integration

  • openviking/service/core.py: Auto-selects PostgreSQLManager when vectordb.backend == 'postgresql', falls back to local VikingDBManager.

SaaS Deployment

  • docker-compose.saas.yml: Full SaaS stack with PostgreSQL+pgvector, MinIO (S3-compatible AGFS backend), and OpenViking server.
  • ov.conf.saas.example: Reference SaaS configuration template.

Visualization Dashboard

  • openviking/server/routers/dashboard.py: Single-page HTML dashboard served at GET /dashboard with dark-themed UI, context browser, semantic search, resource ingestion, and session viewing.
  • openviking/server/routers/debug.py: Added GET /api/v1/debug/storage/stats and GET /api/v1/debug/storage/list endpoints for dashboard use.
  • openviking/server/app.py + routers/init.py: Register dashboard router.

https://claude.ai/code/session_01XUxVTAos6k6N3idBx2a7kP

Description

Related Issue

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Performance improvement
  • Test update

Changes Made

Testing

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have tested this on the following platforms:
    • Linux
    • macOS
    • Windows

Checklist

  • My code follows the project's coding style
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

Screenshots (if applicable)

Additional Notes

…ion dashboard

## Changes

### New PostgreSQL + pgvector Storage Backend
- `openviking/storage/postgresql_backend.py`: Full implementation of
  VikingDBInterface using asyncpg + PostgreSQL pgvector extension.
  Supports all CRUD, vector similarity search, scalar filter DSL
  translation to SQL, scroll pagination, and batch operations.
- `openviking/storage/postgresql_manager.py`: Manager wrapper with
  queue injection interface matching VikingDBManager, enabling
  transparent swap in OpenVikingService.

### Configuration
- `openviking_cli/utils/config/vectordb_config.py`: Added `postgresql`
  backend type and PostgreSQLConfig model (host/port/db/user/password/dsn).
- `pyproject.toml`: Added asyncpg>=0.29.0 dependency.

### Service Integration
- `openviking/service/core.py`: Auto-selects PostgreSQLManager when
  vectordb.backend == 'postgresql', falls back to local VikingDBManager.

### SaaS Deployment
- `docker-compose.saas.yml`: Full SaaS stack with PostgreSQL+pgvector,
  MinIO (S3-compatible AGFS backend), and OpenViking server.
- `ov.conf.saas.example`: Reference SaaS configuration template.

### Visualization Dashboard
- `openviking/server/routers/dashboard.py`: Single-page HTML dashboard
  served at GET /dashboard with dark-themed UI, context browser,
  semantic search, resource ingestion, and session viewing.
- `openviking/server/routers/debug.py`: Added GET /api/v1/debug/storage/stats
  and GET /api/v1/debug/storage/list endpoints for dashboard use.
- `openviking/server/app.py` + routers/__init__.py: Register dashboard router.

https://claude.ai/code/session_01XUxVTAos6k6N3idBx2a7kP
Comprehensive Chinese-language guide covering:
- Architecture overview (PostgreSQL+pgvector vs MinIO vs local)
- Docker Compose quick start (5 steps)
- Full ov.conf.saas configuration reference with inline comments
- Deployment verification (health check, storage stats, MinIO console)
- Visual dashboard usage guide (all 6 tabs documented)
- REST API reference with curl examples
- Local development mode (without Docker)
- Production hardening (passwords, API keys, CORS, real S3)
- Troubleshooting (common errors, PostgreSQL inspection queries)

https://claude.ai/code/session_01XUxVTAos6k6N3idBx2a7kP
@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

…a-only compose

Problem:
  ghcr.io/volcengine/openviking:main is a private image that cannot be
  pulled without authentication, causing 'unauthorized' errors on
  docker compose up.

Changes:
  docker-compose.saas.yml
    - Replace `image: ghcr.io/volcengine/openviking:main` with
      `build: .` + `image: openviking:saas-local` so the image is
      built from the local Dockerfile on first run.
    - Update `start_period` to 60s to accommodate build time.
    - Add explanatory comments about first-build duration.

  docker-compose.infra.yml (new)
    - Minimal compose for PostgreSQL+pgvector and MinIO only.
    - Intended for local development: start infra with Docker,
      run openviking-server directly on the host machine.
    - No image build required; starts in ~20 seconds.

  ov.conf.local.example (new)
    - Companion config for docker-compose.infra.yml.
    - Uses localhost for postgres/minio endpoints instead of
      Docker container names.

  SAAS_DEPLOYMENT.md
    - Section 3 rewritten with two deployment options:
      A) Full Docker (build from source, ~10-30 min first time)
      B) Infra-only Docker + local server (fast, recommended for dev)
    - Windows PowerShell commands added.
    - First-build duration warning added.

https://claude.ai/code/session_01XUxVTAos6k6N3idBx2a7kP
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

3 participants