feat(trask): quality-first free LLM failover chain#92
Open
th3w1zard1 wants to merge 2 commits into
Open
Conversation
Curate OpenRouter :free model priority before vendor scan order, raise rewrite compose attempts to 8, and harden Discord brief fallbacks when LLM rewrite fails. Export TRASK_LLM_PROFILE=free in live stack scripts for Holocron/Discord parity.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Align Holocron grounding with cached-index retrieval: backfill anchored passages for multi-URL compose, preserve upstream passage pools, and skip LLM retries when the grounded template already meets citation bars so expert queries finish under the 30s budget. Add Cloudflare/GitHub weekly reindex schedulers and KB crosswalk for REQ-A/B/C.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| * vector index, so "in Cloudflare" means scheduling here, storage on the host. | ||
| */ | ||
|
|
||
| const trimTrailingSlashes = (value: string): string => value.replace(/\/+$/, ""); |
Comment on lines
+485
to
+486
| value | ||
| .replace(/!\[([^\]]*)\]\([^)]*\)/gu, "$1") |
| export const extractNumberedSourceUrls = (sourceLines: readonly string[]): Map<number, string> => { | ||
| const map = new Map<number, string>(); | ||
| for (const line of sourceLines) { | ||
| const match = line.match(/^\s*(\d+)\.\s+.+\s-\s+(https?:\S+)/u); |
| for (const line of sourceLines) { | ||
| const match = line.match(/^\s*(\d+)\.\s+.+\s-\s+(https?:\S+)/u); | ||
| if (!match) continue; | ||
| const url = match[2]!.replace(/[.,;:!?)]+$/u, ""); |
| }; | ||
|
|
||
| const json = (status: number, body: unknown): Response => | ||
| new Response(JSON.stringify(body), { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CURATED_OPENROUTER_FREE_PRIORITYin@openkotor/configso free-profile compose tries quality-ranked OpenRouter:freemodels before scanningvendor/llm_fallbackslist order.research-wizardandweb-research, matching primary + fallback budget.passagesAnchoredForQuerybackfill,preserveDistinctPassagePool, template-first fast path (REQ-C ≤30s), and markdown artifact stripping in answers.infra/trask-reindex-scheduler) + GitHub Actions fallback (REQ-A).TRASK_LLM_PROFILE=freeandTRASK_RESEARCH_BUDGET_MS=30000intrask_live_stack.shfor local Holocron/Discord parity.Manual verification (Holocron :4010, browser + HTTP)
All five expert queries from
data/trask/eval/verification-queries.json:Stack:
bash scripts/trask_live_stack.sh(Worker :8787, indexer :8790, live crawl off).Test plan
pnpm buildnode --test packages/trask/dist/grounded-evidence.test.js(20/20)4010,8787,8790)Done · grounded, 2 https sources)/api/trask/askpoll for all 5 expert queries ≤30spnpm holocron:e2egreen on CIpnpm verify:trask-discord(requires live bot token + stack)Residual Review Findings
packages/trask/src/grounded-evidence.ts:BRIEF_MAX_CLAIM_LINESraised 2→5 — confirm Discord brief UX still wants ~2 visible lines vs pool size 5.packages/trask/src/research-wizard.ts: LLM claim replacement guard uses URL count only; optional hardening to require grounding improvement.tryGroundedComposethin-citation → template fallback and brief fallback dual-URL path.