|
| 1 | +# Exec Engine, Kokoro TTS, OCR, Model Hosts & Templates |
| 2 | + |
| 3 | +- Added **"Run All" button** to the formatting toolbar — executes every code/tag block in document order |
| 4 | +- Built **Block Registry** (`exec-registry.js`) with FNV-1a content-hash stable IDs and central adapter discovery |
| 5 | +- Built **Execution Controller** (`exec-controller.js`) with progress bar, abort support, per-block status badges (pending/running/done/error), and event emitter |
| 6 | +- Added **SQLite-backed shared context store** (`_exec_results` table) — blocks can write results, downstream blocks read them via `$(blockId)` references or SQL queries |
| 7 | +- Registered **11 runtime adapters**: `bash`, `math`, `python`, `html`, `javascript`, `sql`, `docgen-ai`, `docgen-image`, `docgen-agent`, `api`, `linux-script` |
| 8 | +- DocGen adapters (AI/Image/Agent) use **auto-accept mode** during Run All — skip review panel, replace tag with generated content directly |
| 9 | +- API adapter **auto-accepts** fetch results — replaces `{{API:}}` tag with response markdown |
| 10 | +- Linux adapter submits code to **Judge0 CE** API directly and returns stdout/stderr |
| 11 | +- Added **deferred adapter queue** (`M._pendingRuntimeAdapters`) to handle module loading order |
| 12 | +- Added `css/exec-engine.css` — Run All button (green gradient), fixed-bottom progress bar, per-block status badges |
| 13 | +- Added **Kokoro TTS hybrid engine** (`textToSpeech.js`) — English/Chinese via Kokoro 82M ONNX, Japanese & others via Web Speech API fallback |
| 14 | +- Added **TTS WebWorker** (`tts-worker.js`) — runs `kokoro-js` off main thread for jank-free synthesis; voice auto-selection by language |
| 15 | +- Added `css/tts.css` — TTS button styling, loading states, speaker icon animations |
| 16 | +- Added **model-hosts.js** — configurable model hosting with `MODEL_HOST_FALLBACK` (HuggingFace) for automatic fallback when primary mirror fails |
| 17 | +- Updated **ai-worker.js** — architecture-aware model loading (`qwen3` vs `qwen3_5`), per-component dtype config, `setModelId` accepts `architecture` and `dtype` params, automatic fallback to HuggingFace on primary host failure |
| 18 | +- Added **Kokoro TTS** entry to `ai-models.js` — `kokoro-tts` with `isTtsModel: true` flag, ~80 MB download |
| 19 | +- Added `supportsVision: true` to Qwen 3.5 Flash, Qwen 3.5 35B-A3B, and DeepSeek V3.2 cloud models |
| 20 | +- Added **OCR tag type** to DocGen — `{{@OCR:}}` tag with amber-accented card, Qwen model default, OCR mode pills (Text/Math/Table), image upload with `@upload:` sync |
| 21 | +- Added **OCR card CSS** (`ai-docgen.css`) — amber color scheme, mode pill selector, dark/light theme support |
| 22 | +- Added `data-ocr-mode`, `data-mode`, `data-upload-index` to DOMPurify allowlist |
| 23 | +- Deleted `js/moonshine-medium-worker.js` — replaced by unified `speech-worker.js` |
| 24 | +- Updated `speech-worker.js` — accepts `MODEL_ARCH` and `MODEL_DTYPE` configuration for flexible model loading |
| 25 | +- Added **Language Learning template** (`agents.js`) — multilingual lesson generator with vocabulary, dialogue, grammar, quiz, TTS pronunciation tips |
| 26 | +- Fixed: SQLite-compatible SQL in **Technical template** — replaced `SERIAL`/`VARCHAR`/`NOW()` with `INTEGER`/`TEXT`/`datetime('now')` |
| 27 | +- Updated **Documentation template** — added Run All notebook section, updated test count to 191 |
| 28 | +- Added `scripts/mirror-models.sh` — shell script for self-hosting ONNX models on GitLab |
| 29 | +- Updated `package.json` — added `kokoro-js` dependency |
| 30 | +- Added `public/assets/demos/25_run_all.png` — Run All demo screenshot |
| 31 | +- Added **Run All** entry to Help Mode (`help-mode.js`) |
| 32 | +- Added **Run All** entry to Feature Demos (`feature-demos.js`) |
| 33 | +- Added **12 Playwright tests** for the execution engine (`tests/feature/exec-engine.spec.js`) |
| 34 | +- All 191 existing tests pass with zero regressions |
| 35 | + |
| 36 | +--- |
| 37 | + |
| 38 | +## Summary |
| 39 | + |
| 40 | +This release adds three major features: (1) a unified "Run All" notebook engine that executes every code block, AI tag, API call, and Linux compile in document order with visual progress tracking; (2) a hybrid Kokoro TTS engine for high-quality text-to-speech in English, Chinese, and 10+ languages via Web Speech API fallback; and (3) an OCR tag type for image-to-text extraction. Supporting changes include configurable model hosting with automatic fallback, architecture-aware AI worker loading, vision support flags on cloud models, and an updated template library. |
| 41 | + |
| 42 | +--- |
| 43 | + |
| 44 | +## 1. Block Registry |
| 45 | +**Files:** `js/exec-registry.js` [NEW] |
| 46 | +**What:** Central registry that scans the document for all executable blocks (fenced code blocks + DocGen/API/Linux tags). Assigns stable content-hash-based IDs using FNV-1a hashing. Maintains ordered block list and maps blocks to their runtime adapters. Drains a pending adapter queue (`M._pendingRuntimeAdapters`) on init. |
| 47 | +**Impact:** Every block type is discoverable and addressable by a stable ID, enabling the controller to orchestrate execution. |
| 48 | + |
| 49 | +## 2. Execution Controller |
| 50 | +**Files:** `js/exec-controller.js` [NEW] |
| 51 | +**What:** Orchestrates "Run All" — scans via registry, executes sequentially, resolves `$(blockId)` cross-references, manages progress bar (fixed bottom), per-block status badges, and abort flow. Emits lifecycle events. |
| 52 | +**Impact:** One-click execution of entire document with visual progress tracking and abort capability. |
| 53 | + |
| 54 | +## 3. SQLite Context Store |
| 55 | +**Files:** `js/exec-sandbox.js` |
| 56 | +**What:** Auto-creates `_exec_results` table in the existing sql.js instance. Each block's output is stored as key-value pair. SQL blocks can natively query previous results. Added HTML, JS, and SQL runtime adapters. |
| 57 | +**Impact:** Blocks share data within a single Run All session. |
| 58 | + |
| 59 | +## 4. DocGen Runtime Adapters (Auto-Accept) |
| 60 | +**Files:** `js/ai-docgen-generate.js` |
| 61 | +**What:** Registered `docgen-ai`, `docgen-image`, and `docgen-agent` adapters. During Run All, AI/Image/Agent blocks auto-replace the tag with generated content directly. |
| 62 | +**Impact:** AI/Image/Agent tags execute during Run All without manual review. |
| 63 | + |
| 64 | +## 5. API & Linux Runtime Adapters |
| 65 | +**Files:** `js/api-docgen.js`, `js/linux-docgen.js` |
| 66 | +**What:** API adapter parses config, executes `fetch()`, auto-replaces `{{API:}}` tag. Linux adapter submits to Judge0 CE. |
| 67 | +**Impact:** API calls and compiled code participate in Run All. |
| 68 | + |
| 69 | +## 6. Code Runtime Adapters |
| 70 | +**Files:** `js/executable-blocks.js`, `js/exec-math.js`, `js/exec-python.js` |
| 71 | +**What:** Registered `bash`, `math`, and `python` adapters wrapping existing execution logic. |
| 72 | +**Impact:** All 6 in-browser code runtimes participate in Run All. |
| 73 | + |
| 74 | +## 7. Kokoro TTS Hybrid Engine |
| 75 | +**Files:** `js/textToSpeech.js` [NEW], `js/tts-worker.js` [NEW], `css/tts.css` [NEW] |
| 76 | +**What:** Hybrid text-to-speech using Kokoro 82M v1.1-zh ONNX for English/Chinese (high quality, off-thread via WebWorker) and Web Speech API for Japanese and all other languages. Audio playback via AudioContext with proper cleanup. Voice auto-selection by language code. |
| 77 | +**Impact:** Pronunciation preview on any text in preview — hover and click 🔊 to hear it spoken. |
| 78 | + |
| 79 | +## 8. Model Hosting & AI Worker Architecture |
| 80 | +**Files:** `js/model-hosts.js` [NEW], `ai-worker.js`, `public/ai-worker.js` |
| 81 | +**What:** Configurable model hosting with `MODEL_HOST_FALLBACK` to HuggingFace. AI worker now accepts `architecture` (qwen3 vs qwen3_5) and `dtype` params for flexible model loading. Automatic fallback on primary host failure. |
| 82 | +**Impact:** Models can be self-hosted with transparent fallback; supports both text-only and vision model architectures. |
| 83 | + |
| 84 | +## 9. OCR Tag Type |
| 85 | +**Files:** `js/ai-docgen.js`, `css/ai-docgen.css`, `js/renderer.js` |
| 86 | +**What:** New `{{@OCR:}}` tag with amber-accented card, OCR mode pills (Text/Math/Table), image upload with `@upload:` sync back to editor. Qwen model default. DOMPurify allowlist expanded. |
| 87 | +**Impact:** Extract text from images directly in markdown documents. |
| 88 | + |
| 89 | +## 10. Vision Support & Model Registry |
| 90 | +**Files:** `js/ai-models.js` |
| 91 | +**What:** Added `supportsVision: true` to Qwen 3.5 Flash, 35B-A3B, and DeepSeek V3.2. Added `kokoro-tts` model entry with `isTtsModel: true` flag. |
| 92 | +**Impact:** Vision-capable models shown in image/OCR card selectors; TTS model discoverable in model registry. |
| 93 | + |
| 94 | +## 11. Templates & Documentation |
| 95 | +**Files:** `js/templates/agents.js`, `js/templates/ai.js`, `js/templates/documentation.js`, `js/templates/technical.js` |
| 96 | +**What:** New Language Learning template with vocabulary, dialogue, grammar, quiz, and TTS pronunciation tips. SQLite-compatible SQL in Technical template. Documentation template updated with Run All section and test count. |
| 97 | +**Impact:** Better template coverage; SQL templates now actually run in the SQLite sandbox. |
| 98 | + |
| 99 | +## 12. Infrastructure |
| 100 | +**Files:** `scripts/mirror-models.sh` [NEW], `js/speech-worker.js`, `index.html`, `src/main.js`, `package.json` |
| 101 | +**What:** Model mirroring shell script for self-hosting ONNX models. Speech worker accepts architecture/dtype config. New modules loaded in Phase 3b. `kokoro-js` dependency added. Moonshine medium worker deleted. |
| 102 | +**Impact:** Self-hosting option for air-gapped deployments; cleaner speech module architecture. |
| 103 | + |
| 104 | +--- |
| 105 | + |
| 106 | +## Files Changed (38 total) |
| 107 | + |
| 108 | +| File | Lines Changed | Type | |
| 109 | +|------|:---:|------| |
| 110 | +| `js/exec-registry.js` | +239 | New: Block scanner, adapter registry, FNV-1a IDs | |
| 111 | +| `js/exec-controller.js` | +383 | New: Run All orchestration, progress UI, abort | |
| 112 | +| `js/textToSpeech.js` | +251 | New: Hybrid TTS engine (Kokoro + Web Speech) | |
| 113 | +| `js/tts-worker.js` | +136 | New: Kokoro TTS WebWorker | |
| 114 | +| `js/model-hosts.js` | +19 | New: Configurable model hosting with fallback | |
| 115 | +| `css/exec-engine.css` | +146 | New: Run All button, progress bar, status badges | |
| 116 | +| `css/tts.css` | +178 | New: TTS button styling, animations | |
| 117 | +| `tests/feature/exec-engine.spec.js` | +161 | New: 12 Playwright tests | |
| 118 | +| `scripts/mirror-models.sh` | +130 | New: ONNX model mirroring script | |
| 119 | +| `public/assets/demos/25_run_all.png` | — | New: Run All demo screenshot | |
| 120 | +| `js/ai-docgen-generate.js` | +280 | DocGen auto-accept adapters | |
| 121 | +| `js/ai-docgen.js` | +377 −5 | OCR tag type, block scanner exports | |
| 122 | +| `js/exec-sandbox.js` | +179 | SQLite context store, HTML/JS/SQL adapters | |
| 123 | +| `ai-worker.js` | +126 −86 | Architecture-aware model loading, fallback | |
| 124 | +| `public/ai-worker.js` | +126 −86 | Mirror of ai-worker.js | |
| 125 | +| `css/ai-docgen.css` | +114 | OCR card styling, mode pills | |
| 126 | +| `js/templates/agents.js` | +160 −118 | Language Learning template | |
| 127 | +| `js/templates/ai.js` | +121 | TTS pronunciation tips in templates | |
| 128 | +| `js/api-docgen.js` | +77 | API runtime adapter | |
| 129 | +| `js/linux-docgen.js` | +63 | Linux runtime adapter | |
| 130 | +| `js/speech-worker.js` | +61 −36 | Architecture/dtype config | |
| 131 | +| `js/ai-models.js` | +39 | Vision flags, Kokoro TTS entry | |
| 132 | +| `js/exec-math.js` | +33 | Math adapter | |
| 133 | +| `js/exec-python.js` | +32 | Python adapter | |
| 134 | +| `js/executable-blocks.js` | +22 | Bash adapter | |
| 135 | +| `js/templates/documentation.js` | +20 −3 | Run All docs, test count update | |
| 136 | +| `js/model-hosts.js` | +19 | Model hosting config | |
| 137 | +| `package-lock.json` | +17 | kokoro-js dependency | |
| 138 | +| `js/templates/technical.js` | +16 −10 | SQLite-compatible SQL | |
| 139 | +| `index.html` | +11 | Run All button, TTS module | |
| 140 | +| `src/main.js` | +7 | Module loading | |
| 141 | +| `js/help-mode.js` | +6 | Run All help entry | |
| 142 | +| `js/renderer.js` | +3 −1 | DOMPurify attr expansion | |
| 143 | +| `js/ai-assistant.js` | +2 −2 | Minor fix | |
| 144 | +| `js/feature-demos.js` | +1 | Run All demo entry | |
| 145 | +| `package.json` | +1 | kokoro-js dependency | |
| 146 | +| `README.md` | +5 −3 | Release notes (pre-existing) | |
| 147 | +| `js/moonshine-medium-worker.js` | −393 | Deleted: replaced by speech-worker | |
0 commit comments