Skip to content

Commit b675fc9

Browse files
prosdevprosdev
authored
docs: comprehensive documentation and changeset for v0.6.1 release (#141)
- Add missing 'dev update' command to CLI documentation (README and website) - Document performance tuning environment variables in configuration - Add timing expectations for indexing (5-10 minutes for large codebases) - Create v0.6.1 release notes covering reliability and performance improvements - Update website latest version to v0.6.1 - Add comprehensive changeset for patch release covering: * Performance configuration with environment variables * Enhanced Go scanner reliability and WASM validation * Better error handling and logging across scanners * 57 new tests and architectural improvements * Documentation enhancements and user experience improvements This properly documents all the improvements merged in the recent comprehensive Go support enhancement PR, categorized as a patch release (0.6.1) since it improves the Go feature introduced in 0.6.0 rather than adding new features.
1 parent c2a8cdc commit b675fc9

6 files changed

Lines changed: 162 additions & 9 deletions

File tree

‎.changeset/document-dev-update-command.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
---
2+
"@lytics/dev-agent-cli": patch
3+
"@lytics/dev-agent": patch
4+
"@lytics/dev-agent-core": patch
5+
---
6+
7+
fix: improve reliability, performance, and documentation for Go support
8+
9+
## Major Features
10+
- **Performance Configuration**: Environment variables for fine-tuning concurrency (DEV_AGENT_*_CONCURRENCY)
11+
- **Enhanced Go Scanner**: Runtime WASM validation, improved error handling, better reliability
12+
- **TypeScript Improvements**: Streamlined error handling, better type checking, enhanced progress reporting
13+
- **System Resource Detection**: Intelligent performance defaults based on CPU and memory
14+
- **Architectural Utilities**: Reusable modules for WASM resolution, concurrency, and file validation
15+
16+
## New Environment Variables
17+
- `DEV_AGENT_TYPESCRIPT_CONCURRENCY`: Control TypeScript scanner parallelism
18+
- `DEV_AGENT_INDEXER_CONCURRENCY`: Configure embedding batch processing
19+
- `DEV_AGENT_GO_CONCURRENCY`: Tune Go scanner performance
20+
- `DEV_AGENT_CONCURRENCY`: General fallback for all scanners
21+
22+
## Documentation & User Experience
23+
- Document missing `dev update` command for incremental indexing
24+
- Add timing expectations (5-10 minutes for large codebases)
25+
- Create LANGUAGE_SUPPORT.md contributor guide
26+
- Enhanced troubleshooting and configuration sections
27+
- Remove Renovate automation for manual dependency control
28+
29+
## Technical Improvements
30+
- 57 new tests with comprehensive coverage
31+
- Dependency injection for testable file system operations
32+
- Centralized error handling patterns across scanners
33+
- Build script reliability fixes (prevent silent failures)
34+
35+
This release significantly improves performance, reliability, and developer experience while maintaining backward compatibility.
36+

‎README.md‎

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -50,14 +50,17 @@ We benchmarked dev-agent against baseline Claude Code across 5 task types:
5050
# Install globally
5151
npm install -g dev-agent
5252

53-
# Index your repository
53+
# Index your repository (initial indexing can take 5-10 minutes for large codebases)
5454
cd /path/to/your/repo
5555
dev index .
5656

5757
# Install MCP integration
5858
dev mcp install --cursor # For Cursor IDE
5959
dev mcp install # For Claude Code
6060

61+
# Keep index up to date (fast incremental updates)
62+
dev update
63+
6164
# That's it! AI tools now have access to dev-agent capabilities.
6265
```
6366

@@ -186,6 +189,7 @@ Check MCP server and component health.
186189
### Production Ready
187190
- 🛡️ Rate limiting (100 req/min burst)
188191
- 🔄 Retry logic with exponential backoff
192+
- ⚡ Incremental indexing (only processes changed files)
189193
- 💚 Health monitoring
190194
- 🧹 Memory-safe (circular buffers)
191195
- ✅ 1300+ tests
@@ -223,10 +227,14 @@ npm link
223227
## CLI Commands
224228

225229
```bash
226-
# Index everything (code, git history, GitHub)
230+
# Index everything (code, git history, GitHub) - can take 5-10 min for large codebases
227231
dev index .
228232
dev index . --no-github # Skip GitHub indexing
229233

234+
# Incremental updates (only changed files) - much faster, typically seconds
235+
dev update # Fast incremental reindexing
236+
dev update -v # Verbose output showing what changed
237+
230238
# Semantic search
231239
dev search "how do agents communicate"
232240
dev search "error handling" --threshold 0.3
@@ -289,6 +297,7 @@ To add new languages, see [LANGUAGE_SUPPORT.md](LANGUAGE_SUPPORT.md).
289297

290298
**Indexing too slow:**
291299
```bash
300+
# Note: Initial indexing can take 5-10 minutes for mature codebases (4k+ files)
292301
# Try increasing concurrency (if you have enough memory)
293302
export DEV_AGENT_CONCURRENCY=20
294303
dev index .
@@ -303,6 +312,14 @@ export DEV_AGENT_INDEXER_CONCURRENCY=2
303312
dev index .
304313
```
305314

315+
**Search results are outdated:**
316+
```bash
317+
# Update index with recent file changes
318+
dev update
319+
# Or do a full reindex if needed
320+
dev index .
321+
```
322+
306323
**Go scanner not working:**
307324
```bash
308325
# Check if WASM files are bundled (after installation/build)

‎website/content/docs/cli.mdx‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,8 @@ Creates a `.dev-agent.json` configuration file with default settings.
2424

2525
Index your repository for semantic search. Indexes code, git history, and GitHub issues/PRs.
2626

27+
*Note: Initial indexing can take 5-10 minutes for large codebases (4k+ files).*
28+
2729
```bash
2830
dev index .
2931
dev index /path/to/repo
@@ -153,16 +155,17 @@ dev git stats
153155

154156
### `dev update`
155157

156-
Incrementally update the index with changed files.
158+
Incrementally update the index with changed files. Much faster than full indexing - typically completes in seconds.
157159

158160
```bash
159161
dev update
162+
dev update -v # Show what files changed
160163
```
161164

162165
**Options:**
163166
| Flag | Description |
164167
|------|-------------|
165-
| `-v, --verbose` | Show verbose output |
168+
| `-v, --verbose` | Show verbose output and which files changed |
166169

167170
### `dev stats`
168171

‎website/content/docs/configuration.mdx‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,43 @@ Configuration values can reference environment variables using `${VAR_NAME}` syn
9191
| `WORKSPACE_FOLDER_PATHS` | Cursor workspace paths | MCP server |
9292
| `REPOSITORY_PATH` | Explicit repository path | MCP server |
9393

94+
## Performance Tuning
95+
96+
Control scanning and indexing performance using environment variables:
97+
98+
| Variable | Description | Default | Recommended Range |
99+
|----------|-------------|---------|-------------------|
100+
| `DEV_AGENT_CONCURRENCY` | Global concurrency setting (applies to all operations) | Auto-detected | 5-30 |
101+
| `DEV_AGENT_TYPESCRIPT_CONCURRENCY` | TypeScript file processing concurrency | Auto-detected | 5-40 |
102+
| `DEV_AGENT_INDEXER_CONCURRENCY` | Vector embedding batch concurrency | Auto-detected | 2-10 |
103+
| `DEV_AGENT_GO_CONCURRENCY` | Go file processing concurrency | Auto-detected | 5-30 |
104+
105+
**Auto-detection:** If no environment variables are set, dev-agent automatically detects optimal concurrency based on your system's CPU and memory.
106+
107+
### Performance Recommendations
108+
109+
| System Type | Global | TypeScript | Indexer | Notes |
110+
|-------------|--------|------------|---------|-------|
111+
| Low memory (<4GB) | 5 | 5 | 2 | Prevents OOM errors |
112+
| Standard (4-8GB) | 15 | 15 | 3 | Balanced performance |
113+
| High-end (8GB+, 8+ cores) | 30 | 30 | 5 | Maximum speed |
114+
115+
### Usage Examples
116+
117+
**High-performance settings:**
118+
```bash
119+
export DEV_AGENT_TYPESCRIPT_CONCURRENCY=30
120+
export DEV_AGENT_INDEXER_CONCURRENCY=8
121+
dev index .
122+
```
123+
124+
**Memory-conservative settings:**
125+
```bash
126+
export DEV_AGENT_CONCURRENCY=5
127+
export DEV_AGENT_INDEXER_CONCURRENCY=2
128+
dev index .
129+
```
130+
94131
## Storage Locations
95132

96133
Dev-agent stores index data in `~/.dev-agent/indexes/`:

‎website/content/latest-version.ts‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,10 @@
44
*/
55

66
export const latestVersion = {
7-
version: '0.6.0',
8-
title: 'Go Language Support',
9-
date: 'December 10, 2025',
7+
version: '0.6.1',
8+
title: 'Reliability & Performance Improvements',
9+
date: 'December 11, 2025',
1010
summary:
11-
'Full Go language support with tree-sitter — functions, methods, structs, interfaces, and generics.',
12-
link: '/updates#v060--go-language-support',
11+
'Enhanced reliability, performance configuration, and better documentation for Go support.',
12+
link: '/updates#v061--reliability--performance-improvements',
1313
} as const;

‎website/content/updates/index.mdx‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,66 @@ What's new in dev-agent. We ship improvements regularly to help AI assistants un
99

1010
---
1111

12+
## v0.6.1 — Reliability & Performance Improvements
13+
14+
*December 11, 2025*
15+
16+
**Enhanced reliability, performance configuration, and better documentation for the Go support released in v0.6.0.**
17+
18+
### What's New
19+
20+
**⚡ Performance Configuration**
21+
22+
Fine-tune scanning and indexing performance with environment variables:
23+
24+
```bash
25+
# High-performance systems (8GB+ RAM, 8+ cores)
26+
export DEV_AGENT_TYPESCRIPT_CONCURRENCY=30
27+
export DEV_AGENT_INDEXER_CONCURRENCY=8
28+
29+
# Memory-conservative systems (<4GB RAM)
30+
export DEV_AGENT_CONCURRENCY=5
31+
export DEV_AGENT_INDEXER_CONCURRENCY=2
32+
33+
dev index . # Uses your custom settings
34+
```
35+
36+
**Auto-detection:** If no variables are set, dev-agent automatically detects optimal settings based on your CPU and memory.
37+
38+
**🛠️ Enhanced Reliability**
39+
40+
- **Runtime WASM validation** — Go scanner now validates tree-sitter WASM availability before parsing
41+
- **Build script improvements** — Prevent silent failures that could break Go support
42+
- **Better error handling** — Clearer error messages and proper logging levels across all scanners
43+
44+
**📚 Documentation Improvements**
45+
46+
- Document missing `dev update` command for fast incremental indexing
47+
- Add timing expectations: initial indexing takes 5-10 minutes for large codebases (4k+ files)
48+
- Enhanced troubleshooting guide with performance tuning recommendations
49+
- New performance tuning section in configuration docs
50+
51+
**🧪 Testing & Architecture**
52+
53+
- 57 new tests covering utility functions and edge cases
54+
- Dependency injection for testable file system operations
55+
- Extracted reusable modules for WASM resolution, concurrency calculation, and file validation
56+
57+
### Migration
58+
59+
No breaking changes. All improvements are backward compatible.
60+
61+
### Environment Variables
62+
63+
| Variable | Description | Auto-Detected Default |
64+
|----------|-------------|----------------------|
65+
| `DEV_AGENT_CONCURRENCY` | Global concurrency setting | Based on CPU/memory |
66+
| `DEV_AGENT_TYPESCRIPT_CONCURRENCY` | TypeScript file processing | Based on system resources |
67+
| `DEV_AGENT_INDEXER_CONCURRENCY` | Vector embedding batches | Based on available memory |
68+
| `DEV_AGENT_GO_CONCURRENCY` | Go file processing | Based on system resources |
69+
70+
---
71+
1272
## v0.6.0 — Go Language Support
1373

1474
*December 10, 2025*

0 commit comments

Comments
 (0)