Skip to content

Commit 7b9f846

Browse files
committed
feat: exec engine, Kokoro TTS, OCR tag, model architecture
- Run All notebook engine — 11 runtime adapters, progress bar, abort, SQLite context store - Kokoro TTS hybrid engine — English/Chinese ONNX + Web Speech API fallback for 10+ languages - OCR tag ({{@ocr:}}) — image-to-text with Text/Math/Table modes, amber card styling - Architecture-aware ai-worker — qwen3 vs qwen3_5 model loading, auto-fallback to HuggingFace - Vision support flags on Qwen 3.5 Flash, 35B-A3B, DeepSeek V3.2 - Language Learning template with TTS pronunciation tips - SQLite-compatible SQL in Technical template - model-hosts.js for configurable ONNX model hosting - scripts/mirror-models.sh for self-hosted model mirroring - Deleted moonshine-medium-worker.js (replaced by speech-worker.js) - 12 new Playwright tests (191 total)
1 parent 1ec8b90 commit 7b9f846

39 files changed

Lines changed: 3623 additions & 613 deletions

CHANGELOG-exec-engine.md

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
# Exec Engine, Kokoro TTS, OCR, Model Hosts & Templates
2+
3+
- Added **"Run All" button** to the formatting toolbar — executes every code/tag block in document order
4+
- Built **Block Registry** (`exec-registry.js`) with FNV-1a content-hash stable IDs and central adapter discovery
5+
- Built **Execution Controller** (`exec-controller.js`) with progress bar, abort support, per-block status badges (pending/running/done/error), and event emitter
6+
- Added **SQLite-backed shared context store** (`_exec_results` table) — blocks can write results, downstream blocks read them via `$(blockId)` references or SQL queries
7+
- Registered **11 runtime adapters**: `bash`, `math`, `python`, `html`, `javascript`, `sql`, `docgen-ai`, `docgen-image`, `docgen-agent`, `api`, `linux-script`
8+
- DocGen adapters (AI/Image/Agent) use **auto-accept mode** during Run All — skip review panel, replace tag with generated content directly
9+
- API adapter **auto-accepts** fetch results — replaces `{{API:}}` tag with response markdown
10+
- Linux adapter submits code to **Judge0 CE** API directly and returns stdout/stderr
11+
- Added **deferred adapter queue** (`M._pendingRuntimeAdapters`) to handle module loading order
12+
- Added `css/exec-engine.css` — Run All button (green gradient), fixed-bottom progress bar, per-block status badges
13+
- Added **Kokoro TTS hybrid engine** (`textToSpeech.js`) — English/Chinese via Kokoro 82M ONNX, Japanese & others via Web Speech API fallback
14+
- Added **TTS WebWorker** (`tts-worker.js`) — runs `kokoro-js` off main thread for jank-free synthesis; voice auto-selection by language
15+
- Added `css/tts.css` — TTS button styling, loading states, speaker icon animations
16+
- Added **model-hosts.js** — configurable model hosting with `MODEL_HOST_FALLBACK` (HuggingFace) for automatic fallback when primary mirror fails
17+
- Updated **ai-worker.js** — architecture-aware model loading (`qwen3` vs `qwen3_5`), per-component dtype config, `setModelId` accepts `architecture` and `dtype` params, automatic fallback to HuggingFace on primary host failure
18+
- Added **Kokoro TTS** entry to `ai-models.js``kokoro-tts` with `isTtsModel: true` flag, ~80 MB download
19+
- Added `supportsVision: true` to Qwen 3.5 Flash, Qwen 3.5 35B-A3B, and DeepSeek V3.2 cloud models
20+
- Added **OCR tag type** to DocGen — `{{@OCR:}}` tag with amber-accented card, Qwen model default, OCR mode pills (Text/Math/Table), image upload with `@upload:` sync
21+
- Added **OCR card CSS** (`ai-docgen.css`) — amber color scheme, mode pill selector, dark/light theme support
22+
- Added `data-ocr-mode`, `data-mode`, `data-upload-index` to DOMPurify allowlist
23+
- Deleted `js/moonshine-medium-worker.js` — replaced by unified `speech-worker.js`
24+
- Updated `speech-worker.js` — accepts `MODEL_ARCH` and `MODEL_DTYPE` configuration for flexible model loading
25+
- Added **Language Learning template** (`agents.js`) — multilingual lesson generator with vocabulary, dialogue, grammar, quiz, TTS pronunciation tips
26+
- Fixed: SQLite-compatible SQL in **Technical template** — replaced `SERIAL`/`VARCHAR`/`NOW()` with `INTEGER`/`TEXT`/`datetime('now')`
27+
- Updated **Documentation template** — added Run All notebook section, updated test count to 191
28+
- Added `scripts/mirror-models.sh` — shell script for self-hosting ONNX models on GitLab
29+
- Updated `package.json` — added `kokoro-js` dependency
30+
- Added `public/assets/demos/25_run_all.png` — Run All demo screenshot
31+
- Added **Run All** entry to Help Mode (`help-mode.js`)
32+
- Added **Run All** entry to Feature Demos (`feature-demos.js`)
33+
- Added **12 Playwright tests** for the execution engine (`tests/feature/exec-engine.spec.js`)
34+
- All 191 existing tests pass with zero regressions
35+
36+
---
37+
38+
## Summary
39+
40+
This release adds three major features: (1) a unified "Run All" notebook engine that executes every code block, AI tag, API call, and Linux compile in document order with visual progress tracking; (2) a hybrid Kokoro TTS engine for high-quality text-to-speech in English, Chinese, and 10+ languages via Web Speech API fallback; and (3) an OCR tag type for image-to-text extraction. Supporting changes include configurable model hosting with automatic fallback, architecture-aware AI worker loading, vision support flags on cloud models, and an updated template library.
41+
42+
---
43+
44+
## 1. Block Registry
45+
**Files:** `js/exec-registry.js` [NEW]
46+
**What:** Central registry that scans the document for all executable blocks (fenced code blocks + DocGen/API/Linux tags). Assigns stable content-hash-based IDs using FNV-1a hashing. Maintains ordered block list and maps blocks to their runtime adapters. Drains a pending adapter queue (`M._pendingRuntimeAdapters`) on init.
47+
**Impact:** Every block type is discoverable and addressable by a stable ID, enabling the controller to orchestrate execution.
48+
49+
## 2. Execution Controller
50+
**Files:** `js/exec-controller.js` [NEW]
51+
**What:** Orchestrates "Run All" — scans via registry, executes sequentially, resolves `$(blockId)` cross-references, manages progress bar (fixed bottom), per-block status badges, and abort flow. Emits lifecycle events.
52+
**Impact:** One-click execution of entire document with visual progress tracking and abort capability.
53+
54+
## 3. SQLite Context Store
55+
**Files:** `js/exec-sandbox.js`
56+
**What:** Auto-creates `_exec_results` table in the existing sql.js instance. Each block's output is stored as key-value pair. SQL blocks can natively query previous results. Added HTML, JS, and SQL runtime adapters.
57+
**Impact:** Blocks share data within a single Run All session.
58+
59+
## 4. DocGen Runtime Adapters (Auto-Accept)
60+
**Files:** `js/ai-docgen-generate.js`
61+
**What:** Registered `docgen-ai`, `docgen-image`, and `docgen-agent` adapters. During Run All, AI/Image/Agent blocks auto-replace the tag with generated content directly.
62+
**Impact:** AI/Image/Agent tags execute during Run All without manual review.
63+
64+
## 5. API & Linux Runtime Adapters
65+
**Files:** `js/api-docgen.js`, `js/linux-docgen.js`
66+
**What:** API adapter parses config, executes `fetch()`, auto-replaces `{{API:}}` tag. Linux adapter submits to Judge0 CE.
67+
**Impact:** API calls and compiled code participate in Run All.
68+
69+
## 6. Code Runtime Adapters
70+
**Files:** `js/executable-blocks.js`, `js/exec-math.js`, `js/exec-python.js`
71+
**What:** Registered `bash`, `math`, and `python` adapters wrapping existing execution logic.
72+
**Impact:** All 6 in-browser code runtimes participate in Run All.
73+
74+
## 7. Kokoro TTS Hybrid Engine
75+
**Files:** `js/textToSpeech.js` [NEW], `js/tts-worker.js` [NEW], `css/tts.css` [NEW]
76+
**What:** Hybrid text-to-speech using Kokoro 82M v1.1-zh ONNX for English/Chinese (high quality, off-thread via WebWorker) and Web Speech API for Japanese and all other languages. Audio playback via AudioContext with proper cleanup. Voice auto-selection by language code.
77+
**Impact:** Pronunciation preview on any text in preview — hover and click 🔊 to hear it spoken.
78+
79+
## 8. Model Hosting & AI Worker Architecture
80+
**Files:** `js/model-hosts.js` [NEW], `ai-worker.js`, `public/ai-worker.js`
81+
**What:** Configurable model hosting with `MODEL_HOST_FALLBACK` to HuggingFace. AI worker now accepts `architecture` (qwen3 vs qwen3_5) and `dtype` params for flexible model loading. Automatic fallback on primary host failure.
82+
**Impact:** Models can be self-hosted with transparent fallback; supports both text-only and vision model architectures.
83+
84+
## 9. OCR Tag Type
85+
**Files:** `js/ai-docgen.js`, `css/ai-docgen.css`, `js/renderer.js`
86+
**What:** New `{{@OCR:}}` tag with amber-accented card, OCR mode pills (Text/Math/Table), image upload with `@upload:` sync back to editor. Qwen model default. DOMPurify allowlist expanded.
87+
**Impact:** Extract text from images directly in markdown documents.
88+
89+
## 10. Vision Support & Model Registry
90+
**Files:** `js/ai-models.js`
91+
**What:** Added `supportsVision: true` to Qwen 3.5 Flash, 35B-A3B, and DeepSeek V3.2. Added `kokoro-tts` model entry with `isTtsModel: true` flag.
92+
**Impact:** Vision-capable models shown in image/OCR card selectors; TTS model discoverable in model registry.
93+
94+
## 11. Templates & Documentation
95+
**Files:** `js/templates/agents.js`, `js/templates/ai.js`, `js/templates/documentation.js`, `js/templates/technical.js`
96+
**What:** New Language Learning template with vocabulary, dialogue, grammar, quiz, and TTS pronunciation tips. SQLite-compatible SQL in Technical template. Documentation template updated with Run All section and test count.
97+
**Impact:** Better template coverage; SQL templates now actually run in the SQLite sandbox.
98+
99+
## 12. Infrastructure
100+
**Files:** `scripts/mirror-models.sh` [NEW], `js/speech-worker.js`, `index.html`, `src/main.js`, `package.json`
101+
**What:** Model mirroring shell script for self-hosting ONNX models. Speech worker accepts architecture/dtype config. New modules loaded in Phase 3b. `kokoro-js` dependency added. Moonshine medium worker deleted.
102+
**Impact:** Self-hosting option for air-gapped deployments; cleaner speech module architecture.
103+
104+
---
105+
106+
## Files Changed (38 total)
107+
108+
| File | Lines Changed | Type |
109+
|------|:---:|------|
110+
| `js/exec-registry.js` | +239 | New: Block scanner, adapter registry, FNV-1a IDs |
111+
| `js/exec-controller.js` | +383 | New: Run All orchestration, progress UI, abort |
112+
| `js/textToSpeech.js` | +251 | New: Hybrid TTS engine (Kokoro + Web Speech) |
113+
| `js/tts-worker.js` | +136 | New: Kokoro TTS WebWorker |
114+
| `js/model-hosts.js` | +19 | New: Configurable model hosting with fallback |
115+
| `css/exec-engine.css` | +146 | New: Run All button, progress bar, status badges |
116+
| `css/tts.css` | +178 | New: TTS button styling, animations |
117+
| `tests/feature/exec-engine.spec.js` | +161 | New: 12 Playwright tests |
118+
| `scripts/mirror-models.sh` | +130 | New: ONNX model mirroring script |
119+
| `public/assets/demos/25_run_all.png` || New: Run All demo screenshot |
120+
| `js/ai-docgen-generate.js` | +280 | DocGen auto-accept adapters |
121+
| `js/ai-docgen.js` | +377 −5 | OCR tag type, block scanner exports |
122+
| `js/exec-sandbox.js` | +179 | SQLite context store, HTML/JS/SQL adapters |
123+
| `ai-worker.js` | +126 −86 | Architecture-aware model loading, fallback |
124+
| `public/ai-worker.js` | +126 −86 | Mirror of ai-worker.js |
125+
| `css/ai-docgen.css` | +114 | OCR card styling, mode pills |
126+
| `js/templates/agents.js` | +160 −118 | Language Learning template |
127+
| `js/templates/ai.js` | +121 | TTS pronunciation tips in templates |
128+
| `js/api-docgen.js` | +77 | API runtime adapter |
129+
| `js/linux-docgen.js` | +63 | Linux runtime adapter |
130+
| `js/speech-worker.js` | +61 −36 | Architecture/dtype config |
131+
| `js/ai-models.js` | +39 | Vision flags, Kokoro TTS entry |
132+
| `js/exec-math.js` | +33 | Math adapter |
133+
| `js/exec-python.js` | +32 | Python adapter |
134+
| `js/executable-blocks.js` | +22 | Bash adapter |
135+
| `js/templates/documentation.js` | +20 −3 | Run All docs, test count update |
136+
| `js/model-hosts.js` | +19 | Model hosting config |
137+
| `package-lock.json` | +17 | kokoro-js dependency |
138+
| `js/templates/technical.js` | +16 −10 | SQLite-compatible SQL |
139+
| `index.html` | +11 | Run All button, TTS module |
140+
| `src/main.js` | +7 | Module loading |
141+
| `js/help-mode.js` | +6 | Run All help entry |
142+
| `js/renderer.js` | +3 −1 | DOMPurify attr expansion |
143+
| `js/ai-assistant.js` | +2 −2 | Minor fix |
144+
| `js/feature-demos.js` | +1 | Run All demo entry |
145+
| `package.json` | +1 | kokoro-js dependency |
146+
| `README.md` | +5 −3 | Release notes (pre-existing) |
147+
| `js/moonshine-medium-worker.js` | −393 | Deleted: replaced by speech-worker |

0 commit comments

Comments
 (0)