Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/workflows/preview-seed.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ jobs:
CONVEX_DEPLOY_KEY: ${{ secrets.CONVEX_DEPLOY_KEY_PREVIEW }}
BROWSERBASE_API_KEY: ${{ secrets.BROWSERBASE_API_KEY }}
BROWSERBASE_PROJECT_ID: ${{ secrets.BROWSERBASE_PROJECT_ID }}
VERCEL_AUTOMATION_BYPASS_SECRET: ${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}
steps:
- name: Resolve pull request context
id: pr
Expand Down Expand Up @@ -158,6 +159,14 @@ jobs:
exit 1
fi

- name: Check Vercel automation bypass credential
if: steps.pr.outputs.skip != 'true'
run: |
if [ -z "${VERCEL_AUTOMATION_BYPASS_SECRET:-}" ]; then
echo "VERCEL_AUTOMATION_BYPASS_SECRET secret is not configured."
exit 1
fi

- name: Checkout
if: steps.pr.outputs.skip != 'true'
uses: actions/checkout@v4
Expand Down
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,7 @@ Prioritize those over micro-style debates.

- Use `.memory/` directory to store any temporary artifacts.
- This directory is gitignored, so it will not be committed to the repository, but it is intentionally configured to be visible to codex.
- For preview-debug sessions against Vercel deployment protection, load `VERCEL_AUTOMATION_BYPASS_SECRET` from a local gitignored file such as `.memory/vercel-automation.env` (or another local `.env` outside git).


## web browsing and running dev server
Expand Down
7 changes: 7 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Copyright 2026 | SHPIT LLC

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
146 changes: 44 additions & 102 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,73 +1,29 @@
# Cable Intel

Cable Intel helps people with large cable collections identify the right cable fast, apply a consistent physical label system, and stop relying on memory, random tests, or spreadsheets.
Cable Intel helps you identify cables quickly, label them consistently, and avoid trial-and-error.

## Why This Exists
[![Runtime: Bun](https://img.shields.io/badge/runtime-Bun-black?logo=bun)](https://bun.sh/)
[![Frontend: SvelteKit](https://img.shields.io/badge/frontend-SvelteKit-ff3e00?logo=svelte)](https://kit.svelte.dev/)
[![Backend: Convex](https://img.shields.io/badge/backend-Convex-111111)](https://www.convex.dev/)
[![CI: GitHub Actions](https://img.shields.io/badge/ci-GitHub_Actions-2088FF?logo=githubactions)](https://docs.github.com/actions)
[![Deploy: Vercel](https://img.shields.io/badge/deploy-Vercel-000000?logo=vercel)](https://vercel.com/)
[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](./LICENSE)

When you have dozens or hundreds of cables on a wall, day-to-day cable choice is slow and error-prone.
## What It Does

| Pain | Typical workaround | What Cable Intel changes |
|---|---|---|
| "I cannot tell what this cable can do from a quick glance." | Search product pages manually or test against devices. | Identify by catalog row or markings, then get a capability profile immediately. |
| "My labels are inconsistent." | Invent a personal color system and hope everyone follows it. | Generate deterministic `velcro strap + holder` color recommendations from cable capability. |
| "I lose track of duplicates and what needs to be re-printed." | Keep ad hoc notes/spreadsheets. | Moves toward a virtual wall and inventory-first workflow (planned). |
- Identifies cables from catalog data or free-text printed markings.
- Infers connector/power/data/video capabilities and summarizes expected behavior.
- Suggests deterministic physical label colors for strap and holder classes.
- Supports catalog filtering (brand, connector type, length, color, price bucket).
- Includes a TUI for ingest discovery, seeding, and quality checks.

Works well with physical systems like Multiboard + 3D printed holders, where consistency matters.
## How It Works

## What Exists Now

- Web workspace for cable identification and label assignment.
- Two identification paths:
- Catalog path (Anker-first ingest today).
- Manual markings path (works even when catalog has no match).
- Automatic color output:
- `Velcro strap color` for data/video class.
- `Holder color` for charging/power class.
- Capability summary with practical guidance for power, data, and video expectations.
- Catalog search and filtering (brand, connector type, length, color; price facet is scaffolded but price ingest is pending).
- Convex ingest pipeline that stores evidence-backed normalized specs and quality state.
- TUI ingest manager for discovery, seed runs, and quality reporting.

See `docs/ARCHITECTURE_SPEC.md` for architecture details.

## Future Features Under Consideration

These are open TODO/spike items already tracked in GitHub:

| Candidate | Pain solved | Status |
|---|---|---|
| Hybrid search + price ingest (facets + lexical + semantic rerank) | Improves "find the right cable" speed for natural-language queries | https://github.com/shpitdev/cable-intel/issues/2 |

## Product Planning (Working Draft)

Current constraint: there is no auth, no multi-user model, and no per-user saved workspace yet. That is the major platform unlock for personalization and collaboration.

### Big Dependencies

| Dependency | Why it matters | Unlocks |
|---|---|---|
| Need-profile engine (task requirements + matching explanations) | Converts "will this cable work for what I do all the time?" into a repeatable check | Use-case fit advisor before buying/using a cable; saved recurring checks |
| Auth + workspace model (user/team identity and ownership) | Lets data belong to a person or team instead of one global state | Custom color systems, personal inventory, shared wall management, collaboration |
| Physical inventory model (cable instances separate from catalog variants) | Required to track duplicates and real-world state of each cable | Virtual wall, in-use vs on-wall status, low-stock and "print more" alerts |

### Sequenced Backlog (Value vs Complexity)

| Order | Candidate | Pain solved | Depends on | Value | Complexity | Tracking |
|---|---|---|---|---|---|---|
| 1 | Use-case fit advisor (check a cable against repeated workflows) | Reduces uncertainty like "will this reliably handle my laptop + display workflow?" | Need-profile engine + existing capability profile | High | Medium | Idea (no issue yet) |
| 2 | Hybrid catalog search + price ingest | Faster catalog lookup with natural phrasing and budget awareness | Ingest/schema updates + ranking pipeline | High | Medium-High | https://github.com/shpitdev/cable-intel/issues/2 |
| 3 | Auth + multi-user workspaces | Enables persistent per-user/team state and collaboration | Identity/session + data ownership model | Very High | High | Idea (no issue yet) |
| 4 | Custom color systems by user/workspace (strap + holder profiles) | Supports personalized labeling systems and real material choices | Auth/workspaces | Medium-High | Medium | Idea (no issue yet) |
| 5 | Virtual wall + cable inventory tracking | Replaces spreadsheets for duplicates, location, and usage state | Auth/workspaces + physical inventory model | Very High | High | Idea (no issue yet) |
| 6 | "Print more" planner and stock alerts | Reduces holder/label stock surprises and planning overhead | Inventory tracking + holder/label metadata | Medium-High | Medium | Idea (no issue yet) |

## Repository Layout

- `apps/web`: SvelteKit frontend workspace
- `apps/tui`: terminal ingest manager
- `packages/backend`: Convex schema/functions and ingest workflow
- `packages/env`: shared environment validation
- `packages/shopify-cable-source`: Shopify template/source extraction
- `apps/web`: SvelteKit workspace UI (catalog + manual flows).
- `packages/backend`: Convex schema/actions/queries and ingest pipeline.
- `packages/shopify-cable-source`: Shopify source discovery/extraction.
- `apps/tui`: terminal ingest manager.
- Preview deployments run `preview-validation` (seed + runtime QA + browser smoke with artifacts/replay links).

## Quick Start

Expand All @@ -79,57 +35,43 @@ bun run dev

Web app: `http://localhost:5173`

## Useful Commands

- `bun run dev`: run backend + web dev servers
- `bun run build`: run workspace build tasks
- `bun run check-types`: run type checks
- `bun run check`: run Ultracite checks
- `bun run fix`: apply Ultracite fixes
- `bun run seed:realistic -- --deployment-name <name>`: discover and seed a multi-brand realistic dataset for local search QA
- `bun run seed:vercel-deployment -- --vercel-url <deployment-url>`: map Vercel URL to Convex deployment, run seed ingest, print quality summary
- `bun run --cwd apps/tui start`: interactive ingest manager
- `bun run --cwd apps/tui build:bin`: build standalone TUI binary at `apps/tui/dist/cable-intel-tui`

## Testing And CI

| Layer | Present | Tooling | Runs in CI |
|---|---|---|---|
| unit | yes | `bun test` (example: `apps/web/src/lib/*.test.ts`, `packages/backend/convex/*.test.ts`) | no |
| integration | yes | `bun test` integration suites | yes |
| unit | yes | `bun test` (core suites) | yes |
| integration | yes | `bun test` integration suites | no |
| e2e api | yes | `bun test packages/backend/convex/ingest.e2e.test.ts` | no |
| e2e web | no | none | no |
| e2e web | yes | `agent-browser` smoke scripts | yes |

CI (`.github/workflows/ci.yml`) currently runs:
- `ultracite check`
- deterministic manual inference + mapping tests
- workspace build
- TUI compile smoke test
Primary CI workflows:
- `.github/workflows/ci.yml` (quality + build)
- `.github/workflows/preview-seed.yml` (preview seed + browser QA)
- `.github/workflows/org-required-checks.yml` (required merge gate)

## Deployment And External Services

- Deployment target: Vercel (`vercel.json`)
- Backend/data: Convex
- Fallback ingest providers: Firecrawl + AI Gateway/OpenAI-compatible model via `ai` SDK
- Deploy target: Vercel (`vercel.json`)
- Data/backend: Convex
- LLM path: AI Gateway/OpenAI-compatible via `ai` SDK
- Optional analytics: `PUBLIC_VERCEL_ANALYTICS_DSN` (already wired in `apps/web/src/routes/+layout.ts`)

Preview automation secrets:
- `CONVEX_DEPLOY_KEY_PREVIEW`
- `BROWSERBASE_API_KEY`
- `BROWSERBASE_PROJECT_ID`
- `VERCEL_AUTOMATION_BYPASS_SECRET`

Environment notes:
- `PUBLIC_CONVEX_URL` is required by the web app.
- `AI_GATEWAY_API_KEY` and `FIRECRAWL_API_KEY` are required only for fallback ingest mode.
- Shopify-only extraction path works without those fallback provider keys.
- Optional web analytics: `PUBLIC_VERCEL_ANALYTICS_DSN`.
- Preview runtime validation exists in `.github/workflows/preview-seed.yml` (`preview-validation`):
- triggered by Vercel `deployment_status` events
- resolves the preview deployment URL automatically
- seeds Convex preview catalog
- runs manual inference runtime validation and Browserbase QA on deployed preview
## API Surface

## Release Automation
There is no public REST API in this repo. App/backend integration is via Convex actions and queries.

- Pushes to `main` create a version bump PR (`chore: bump version to x.y.z`).
- Auto-merge is enabled on that PR after review.
- When merged, GitHub Release `vX.Y.Z` is created.
- Tag pushes trigger TUI binary bundles via `.github/workflows/tui-release.yml`.
## Active Backlog

- Search quality phase 2: semantic rerank + measured eval ([#92](https://github.com/shpitdev/cable-intel/issues/92))

## License

No repository-level license file is currently committed.
MIT License. See [LICENSE](./LICENSE).

Copyright 2026 SHPIT LLC.
17 changes: 17 additions & 0 deletions scripts/browser/run-local-browser-qa.sh
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,19 @@ if [[ -z "$BASE_URL" ]]; then
BASE_URL="http://${HOST}:${PORT}"
fi

append_vercel_bypass_query() {
local raw_url="$1"
local bypass_secret="$2"
node -e '
const rawUrl = process.argv[1];
const bypassSecret = process.argv[2];
const url = new URL(rawUrl);
url.searchParams.set("x-vercel-protection-bypass", bypassSecret);
url.searchParams.set("x-vercel-set-bypass-cookie", "true");
process.stdout.write(url.toString());
' "$raw_url" "$bypass_secret"
}

is_local_base_url() {
local url="$1"
[[ "$url" =~ ^https?://(localhost|127\.0\.0\.1|0\.0\.0\.0|\[::1\])(:|/|$) ]]
Expand Down Expand Up @@ -89,6 +102,10 @@ if [[ "$START_PREVIEW" == "1" ]]; then
wait_for_url "$BASE_URL"
fi

if [[ -n "${VERCEL_AUTOMATION_BYPASS_SECRET:-}" ]]; then
BASE_URL="$(append_vercel_bypass_query "$BASE_URL" "$VERCEL_AUTOMATION_BYPASS_SECRET")"
fi

SMOKE_ARGS=(
--base-url "$BASE_URL"
--out-dir "$OUT_DIR"
Expand Down
39 changes: 32 additions & 7 deletions scripts/seed-vercel-deployment.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ interface CliOptions {
discoverMax: number;
seedMax: number;
templateId: string;
vercelBypassSecret: string | null;
vercelUrl: string | null;
}

Expand Down Expand Up @@ -61,8 +62,9 @@ const printUsage = (): void => {
" --deployment-name <name> Convex deployment name (skip URL discovery)",
" --template-id <id> Shopify template id (default: anker-us)",
" --discover-max <n> Max URLs to discover (default: 30)",
" --seed-max <n> Max URLs to seed (default: 20)",
" --seed-max <n> Max URLs to seed (default: 8)",
" --allowed-domain <host> Allowed ingest domain (default: anker.com)",
" --vercel-bypass-secret Vercel automation bypass secret",
" --backend-dir <path> Backend package dir (default: packages/backend)",
" --help Show this message",
].join("\n")
Expand Down Expand Up @@ -93,6 +95,7 @@ const parseArgs = (argv: string[]): CliOptions => {
vercelUrl: process.env.VERCEL_URL
? normalizeVercelUrl(process.env.VERCEL_URL)
: null,
vercelBypassSecret: process.env.VERCEL_AUTOMATION_BYPASS_SECRET ?? null,
deploymentName: null,
templateId: DEFAULT_TEMPLATE_ID,
discoverMax: DEFAULT_DISCOVER_MAX,
Expand Down Expand Up @@ -121,6 +124,10 @@ const parseArgs = (argv: string[]): CliOptions => {
const trimmed = value.trim();
options.allowedDomain = trimmed || null;
},
"--vercel-bypass-secret": (value) => {
const trimmed = value.trim();
options.vercelBypassSecret = trimmed || null;
},
"--backend-dir": (value) => {
options.backendDir = value.trim();
},
Expand Down Expand Up @@ -160,10 +167,18 @@ const parseArgs = (argv: string[]): CliOptions => {
return options;
};

const fetchText = async (url: string): Promise<string> => {
const fetchText = async (
url: string,
bypassSecret: string | null
): Promise<string> => {
const response = await fetch(url, {
headers: {
"user-agent": "cable-intel-seed-script/1.0",
...(bypassSecret
? {
"x-vercel-protection-bypass": bypassSecret,
}
: {}),
},
});
if (!response.ok) {
Expand Down Expand Up @@ -198,13 +213,20 @@ const extractConvexUrl = (nodeCode: string): string => {
};

const resolveDeploymentNameFromVercelUrl = async (
vercelUrl: string
vercelUrl: string,
bypassSecret: string | null
): Promise<{ convexUrl: string; deploymentName: string }> => {
const html = await fetchText(vercelUrl);
const html = await fetchText(vercelUrl, bypassSecret);
const appEntryPath = extractAppEntryPath(html);
const appCode = await fetchText(new URL(appEntryPath, vercelUrl).toString());
const appCode = await fetchText(
new URL(appEntryPath, vercelUrl).toString(),
bypassSecret
);
const nodeZeroPath = extractNodeZeroPath(appCode);
const nodeCode = await fetchText(new URL(nodeZeroPath, vercelUrl).toString());
const nodeCode = await fetchText(
new URL(nodeZeroPath, vercelUrl).toString(),
bypassSecret
);
const convexUrl = extractConvexUrl(nodeCode);
const deploymentName = new URL(convexUrl).hostname.split(".")[0];
if (!deploymentName) {
Expand Down Expand Up @@ -496,7 +518,10 @@ const main = async (): Promise<void> => {
if (!vercelUrl) {
throw new Error("Missing Vercel URL.");
}
const resolved = await resolveDeploymentNameFromVercelUrl(vercelUrl);
const resolved = await resolveDeploymentNameFromVercelUrl(
vercelUrl,
options.vercelBypassSecret
);
deploymentName = resolved.deploymentName;
convexUrl = resolved.convexUrl;
console.log(`Resolved Convex URL: ${convexUrl}`);
Expand Down
Loading