File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -198,6 +198,16 @@ Workspace hygiene helper:
198198python scripts/clean_workspace.py --dry-run
199199```
200200
201+ ### 5.1 README/PyPI flow diagram rendering
202+
203+ Render Mermaid source files to high-resolution static images:
204+
205+ ``` bash
206+ python scripts/render_mermaid_diagrams.py --write-manifest
207+ ```
208+
209+ Sources live in ` docs/diagrams/*.mmd ` and outputs are written to ` docs/assets/diagrams/ ` .
210+
201211---
202212
203213## 6) Host Learning Lifecycle (auto domain routing)
Original file line number Diff line number Diff line change @@ -79,33 +79,15 @@ web-scraper --url https://example.com --format markdown --export
7979
8080### Simple flow
8181
82- ``` mermaid
83- flowchart TD
84- A[Input URL(s)] --> B[Runtime Config Resolution]
85- B --> C[Primary Fetch/Crawl Pipeline]
86- C --> D[Parse + Extract]
87- D --> E[Format + Write Output]
88- ```
82+ ![ Simple flow diagram] ( https://raw.githubusercontent.com/ImYourBoyRoy/WebScraperToolkit/main/docs/assets/diagrams/simple_flow.webp )
8983
9084### Advanced flow (dynamic routing)
9185
92- ``` mermaid
93- flowchart TD
94- A[Input URL(s)] --> B[Resolve Config: CLI > ENV > Files > Defaults]
95- B --> C[Host Profile Lookup]
96- C --> D[Playwright Primary Attempt]
97- D --> E{Blocked or Degraded?}
98- E -- No --> F[Parse + Extract + Save]
99- E -- Yes --> G[Native Browser Fallback Chain]
100- G --> H{Solved?}
101- H -- Yes --> I[Record Telemetry]
102- H -- No --> J[Optional Interactive Challenge Solve]
103- J --> I
104- I --> K[Host Learning Candidate/Promotion]
105- K --> F
106- ```
86+ ![ Advanced routing flow diagram] ( https://raw.githubusercontent.com/ImYourBoyRoy/WebScraperToolkit/main/docs/assets/diagrams/advanced_flow.webp )
10787
108- > GitHub renders Mermaid natively. Some package indexes may not render Mermaid; use ` INSTRUCTIONS.md ` for text flow details.
88+ > These diagrams are rendered from Mermaid source files for GitHub/PyPI compatibility.
89+ > Sources: ` docs/diagrams/*.mmd `
90+ > Rebuild command: ` python scripts/render_mermaid_diagrams.py --write-manifest `
10991
11092---
11193
Original file line number Diff line number Diff line change 1+ [
2+ {
3+ "source" : " C:/Users/Roy/Desktop/AI_tools/WebScraperToolkit/docs/diagrams/advanced_flow.mmd" ,
4+ "output" : " C:/Users/Roy/Desktop/AI_tools/WebScraperToolkit/docs/assets/diagrams/advanced_flow.webp" ,
5+ "output_width" : 880 ,
6+ "output_height" : 3104 ,
7+ "theme" : " neutral" ,
8+ "format" : " webp"
9+ },
10+ {
11+ "source" : " C:/Users/Roy/Desktop/AI_tools/WebScraperToolkit/docs/diagrams/simple_flow.mmd" ,
12+ "output" : " C:/Users/Roy/Desktop/AI_tools/WebScraperToolkit/docs/assets/diagrams/simple_flow.webp" ,
13+ "output_width" : 2706 ,
14+ "output_height" : 140 ,
15+ "theme" : " neutral" ,
16+ "format" : " webp"
17+ }
18+ ]
Original file line number Diff line number Diff line change 1+ %% ./docs/diagrams/advanced_flow.mmd
2+ flowchart TB
3+ A [ " Input URLs" ] --> B [ " Resolve config precedence" ]
4+ B --> C [ " Lookup host profile" ]
5+ C --> D [ " Primary Playwright attempt" ]
6+ D --> E { " Blocked or degraded?" }
7+ E -- "No" --> F [ " Parse, extract, and save" ]
8+ E -- "Yes" --> G [ " Native browser fallback chain" ]
9+ G --> H { " Solved?" }
10+ H -- "Yes" --> I [ " Record telemetry" ]
11+ H -- "No" --> J [ " Interactive challenge solve" ]
12+ J --> I
13+ I --> K [ " Host learning promotion or demotion" ]
14+ K --> F
Original file line number Diff line number Diff line change 1+ %% ./docs/diagrams/simple_flow.mmd
2+ flowchart LR
3+ A [ " Input URLs" ] --> B [ " Resolve runtime config" ]
4+ B --> C [ " Fetch and crawl pipeline" ]
5+ C --> D [ " Parse and extract content" ]
6+ D --> E [ " Format and write output" ]
You can’t perform that action at this time.
0 commit comments