Skip to content

Commit 1721d1b

Browse files
ijboijbo
committed
docs: add orchestration upgrade plan for future enhancements
1 parent b26e6df commit 1721d1b

2 files changed

Lines changed: 226 additions & 0 deletions

File tree

‎changelogs/CHANGELOG-upgrade-plan.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Changelog: Orchestration Upgrade Plan
2+
3+
## Added
4+
- `upgradePlan/orchestration-enhancements.md` — Future roadmap for advanced execution flow control:
5+
- Phase 1: Parallel block execution (`@parallel` groups)
6+
- Phase 2: Conditional branching (`@if`/`@else`)
7+
- Phase 3: Loop/iteration (`@each`)
8+
- Phase 4: Event-driven triggers (`@on`)
9+
10+
## Files Added
11+
- `upgradePlan/orchestration-enhancements.md`

‎upgradePlan/orchestration-enhancements.md‎

Lines changed: 215 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,215 @@
1+
# TextAgent Orchestration Enhancements
2+
3+
Add advanced execution flow control to TextAgent's block execution engine — parallel execution, conditional branching, loops, and event triggers.
4+
5+
## Current Architecture
6+
7+
TextAgent's execution engine has 3 layers:
8+
- [exec-registry.js](file:///Users/jyotibose/textagent.github.io/js/exec-registry.js) — Scans document, assigns stable IDs, maintains block list
9+
- [run-requirements.js](file:///Users/jyotibose/textagent.github.io/js/run-requirements.js) — 6-pass compiler: scan → enrich → resolve → validate → model plan → summarize
10+
- [exec-controller.js](file:///Users/jyotibose/textagent.github.io/js/exec-controller.js) — Sequential top-to-bottom execution loop with events, abort, progress UI
11+
12+
Current execution is a **flat sequential `for` loop** over document-ordered blocks. Variables flow via `M._vars` (set/get) and `M._execContext`.
13+
14+
## User Review Required
15+
16+
> [!IMPORTANT]
17+
> **Scope Decision**: These are 4 independent features of increasing complexity. I recommend implementing them **one phase at a time**, starting with Phase 1 (Parallel) which has the highest impact-to-effort ratio.
18+
>
19+
> **Which phases do you want to build?**
20+
> 1. ✅ Parallel execution only (quickest win)
21+
> 2. ✅ Parallel + Conditional (solid workflow upgrade)
22+
> 3. ✅ Parallel + Conditional + Loops (full pipeline control)
23+
> 4. ✅ All four (complete orchestration OS)
24+
25+
> [!WARNING]
26+
> **Syntax choice**: The proposed syntax uses annotations inside code block headers and markdown comments. This avoids breaking existing templates. However, if you prefer a different syntax (e.g., new DocGen tags like `{{@If:}}`), we should decide before implementation.
27+
28+
---
29+
30+
## Proposed Changes
31+
32+
### Phase 1: Parallel Block Execution
33+
34+
Users can group blocks that run concurrently using `@parallel` annotations.
35+
36+
**Syntax:**
37+
```markdown
38+
<!-- @parallel: group1 -->
39+
```python @var: data1
40+
fetch_data("source_a")
41+
```
42+
43+
```python @var: data2
44+
fetch_data("source_b")
45+
```
46+
<!-- @/parallel -->
47+
48+
```python
49+
# This runs after both data1 and data2 are ready
50+
combine($(data1), $(data2))
51+
```
52+
```
53+
54+
#### [MODIFY] [exec-registry.js](file:///Users/jyotibose/textagent.github.io/js/exec-registry.js)
55+
- Add `scanParallelGroups(markdown)` — detect `<!-- @parallel: name -->` / `<!-- @/parallel -->` comment pairs
56+
- Tag blocks with `_parallelGroup` property when they fall inside a parallel region
57+
- Add `_parallelGroupId` for barrier synchronization
58+
59+
#### [MODIFY] [exec-controller.js](file:///Users/jyotibose/textagent.github.io/js/exec-controller.js)
60+
- Replace flat `for` loop in `executeBlocks()` with a group-aware scheduler:
61+
- Blocks without `_parallelGroup` → sequential (current behavior)
62+
- Blocks with same `_parallelGroup` → `Promise.all()` concurrent execution
63+
- After parallel group completes → continue sequential flow
64+
- Update progress bar to show "Running 3 blocks in parallel" when applicable
65+
- Ensure variable writes from parallel blocks are safe (they write to different vars)
66+
67+
#### [MODIFY] [run-requirements.js](file:///Users/jyotibose/textagent.github.io/js/run-requirements.js)
68+
- **Pass 2 (Enrich)**: Detect and label parallel groups
69+
- **Pass 4 (Validate)**: Error if two parallel blocks write the same `@var:` name
70+
- **Pass 4 (Validate)**: Error if a parallel block reads a var produced by a block in the same parallel group
71+
- Display parallel group info in preflight dialog
72+
73+
---
74+
75+
### Phase 2: Conditional Branching
76+
77+
Blocks can be conditionally skipped based on variable values.
78+
79+
**Syntax:**
80+
```markdown
81+
```python @var: sentiment
82+
analyze_sentiment($(text))
83+
```
84+
85+
<!-- @if: $(sentiment) == "positive" -->
86+
{{AI: Write a congratulations message about: $(text) @var: response}}
87+
<!-- @else -->
88+
{{AI: Write an empathetic message about: $(text) @var: response}}
89+
<!-- @/if -->
90+
```
91+
92+
#### [NEW] [exec-flow.js](file:///Users/jyotibose/textagent.github.io/js/exec-flow.js)
93+
- New module (~150 lines) for flow control evaluation
94+
- `evaluateCondition(expr, vars)` — evaluates simple conditions:
95+
- String equality: `$(var) == "value"`
96+
- Truthiness: `$(var)` (truthy if non-empty)
97+
- Negation: `!$(var)`
98+
- Contains: `$(var) contains "word"`
99+
- Numeric comparisons: `$(var) > 5`
100+
- Export on `M._execFlow`
101+
102+
#### [MODIFY] [exec-registry.js](file:///Users/jyotibose/textagent.github.io/js/exec-registry.js)
103+
- Detect `<!-- @if: condition -->`, `<!-- @else -->`, `<!-- @/if -->` comment markers
104+
- Tag blocks with `_ifCondition`, `_ifBranch` ("then" / "else"), `_ifGroupId`
105+
106+
#### [MODIFY] [exec-controller.js](file:///Users/jyotibose/textagent.github.io/js/exec-controller.js)
107+
- Before each block: if it has `_ifCondition`, evaluate it using current vars
108+
- Skip blocks in the wrong branch
109+
- Update block status UI to show "⏭ Skipped (condition false)"
110+
111+
#### [MODIFY] [run-requirements.js](file:///Users/jyotibose/textagent.github.io/js/run-requirements.js)
112+
- Validate `@if` blocks have matching `@/if`
113+
- Show conditional branches in preflight dialog
114+
115+
---
116+
117+
### Phase 3: Loop / Iteration
118+
119+
Repeat blocks for each item in a list variable.
120+
121+
**Syntax:**
122+
```markdown
123+
```python @var: items
124+
["apple", "banana", "cherry"]
125+
```
126+
127+
<!-- @each: $(items) as item -->
128+
{{AI: Write a haiku about $(item) @var: haiku_$(item)}}
129+
<!-- @/each -->
130+
```
131+
132+
#### [MODIFY] [exec-flow.js](file:///Users/jyotibose/textagent.github.io/js/exec-flow.js)
133+
- Add `parseList(value)` — parse JSON array or newline-separated list from var
134+
- Add `expandEachBlock(block, items)` — create N copies of block, each with `$(item)` bound
135+
136+
#### [MODIFY] [exec-registry.js](file:///Users/jyotibose/textagent.github.io/js/exec-registry.js)
137+
- Detect `<!-- @each: $(var) as item -->` / `<!-- @/each -->`
138+
- Tag blocks with `_eachSource`, `_eachAlias`, `_eachGroupId`
139+
140+
#### [MODIFY] [exec-controller.js](file:///Users/jyotibose/textagent.github.io/js/exec-controller.js)
141+
- When hitting an `@each` group: resolve the list var, expand blocks, execute sequentially (or parallel if nested in `@parallel`)
142+
- Dynamically update progress total when loops expand
143+
144+
#### [MODIFY] [run-requirements.js](file:///Users/jyotibose/textagent.github.io/js/run-requirements.js)
145+
- Warn if `@each` source var isn't produced by an earlier block
146+
- Show loop info in preflight
147+
148+
---
149+
150+
### Phase 4: Event-Driven Triggers
151+
152+
Blocks that react to other blocks completing (for reactive pipelines).
153+
154+
**Syntax:**
155+
```markdown
156+
```python @var: data
157+
fetch_data()
158+
```
159+
160+
<!-- @on: data -->
161+
{{AI: Summarize this data: $(data) @var: summary}}
162+
<!-- @/on -->
163+
164+
<!-- @on: summary -->
165+
{{TTS: $(summary)}}
166+
<!-- @/on -->
167+
```
168+
169+
#### [MODIFY] [exec-flow.js](file:///Users/jyotibose/textagent.github.io/js/exec-flow.js)
170+
- Add event trigger registry: `_onTriggers[varName] → [blockGroup]`
171+
- Integration with existing event emitter in exec-controller
172+
173+
#### [MODIFY] [exec-registry.js](file:///Users/jyotibose/textagent.github.io/js/exec-registry.js)
174+
- Detect `<!-- @on: varName -->` / `<!-- @/on -->`
175+
- Tag blocks with `_onTrigger`, `_onGroupId`
176+
177+
#### [MODIFY] [exec-controller.js](file:///Users/jyotibose/textagent.github.io/js/exec-controller.js)
178+
- After each block sets a variable, check if any `@on` blocks are triggered
179+
- Execute triggered blocks asynchronously (fire-and-forget or awaited based on config)
180+
181+
182+
---
183+
184+
## Verification Plan
185+
186+
### Automated Tests
187+
188+
Extend existing [exec-engine.spec.js](file:///Users/jyotibose/textagent.github.io/tests/feature/exec-engine.spec.js) Playwright tests.
189+
190+
**Run command:**
191+
```bash
192+
cd /Users/jyotibose/textagent.github.io && npx playwright test tests/feature/exec-engine.spec.js
193+
```
194+
195+
**New test cases per phase:**
196+
197+
| Phase | Test | Description |
198+
|-------|------|-------------|
199+
| 1 | `parallel annotation scanning` | Registry detects `@parallel` comments and tags blocks |
200+
| 1 | `parallel execution completes` | Two math blocks in `@parallel` produce results faster than sequential |
201+
| 1 | `parallel var conflict detected` | Compiler errors on two parallel blocks writing same var |
202+
| 2 | `if/else scanning` | Registry detects `@if`/`@else`/`@/if` markers |
203+
| 2 | `condition evaluation` | `exec-flow.js` evaluates equality, truthiness, negation |
204+
| 2 | `conditional skip` | Blocks in wrong branch show "Skipped" status |
205+
| 3 | `each scanning` | Registry detects `@each` markers |
206+
| 3 | `loop expansion` | 3-item list produces 3 block executions |
207+
| 4 | `on trigger scanning` | Registry detects `@on` markers |
208+
| 4 | `event trigger fires` | Block runs when its trigger var is set |
209+
210+
### Manual Verification
211+
212+
1. Open TextAgent at `http://localhost:5174`
213+
2. Load a template with `@parallel` blocks and click "Run All"
214+
3. Verify progress bar shows "Running N blocks in parallel"
215+
4. Confirm both blocks produce results and downstream block receives both vars

0 commit comments

Comments
 (0)