You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
prompt: 'Triage the changed code for correctness, regressions, and weak assumptions. Prefer concrete bugs over style.',
30
34
useTools: true,
31
35
maxToolRounds: 6
36
+
},
37
+
{
38
+
name: 'Optional Brainstorm',
39
+
provider: 'openai',
40
+
endpoint: 'https://api.openai.com/v1',
41
+
apiKeyEnv: 'OPENAI_API_KEY',
42
+
timeoutMs: 120000,
43
+
mode: 'suggestions',
44
+
structuredOutputMode: 'json_schema',
45
+
allowSuggestions: true,
46
+
maxSuggestions: 2,
47
+
// If you do set maxOutputTokens on GPT-5, keep it well above the visible output target.
48
+
model: 'gpt-5-mini',
49
+
prompt: 'Suggest at most 2 grounded follow-up improvements. Do not repeat defects. Avoid rewrites and generic wishlist items. Every suggestion must have a clear benefit and tradeoff.',
50
+
useTools: true,
51
+
maxToolRounds: 4
52
+
}
53
+
],
54
+
brainstormers: [
55
+
{
56
+
name: 'Scope Challenger',
57
+
provider: 'nvidia-free',
58
+
endpoint: 'https://integrate.api.nvidia.com/v1',
59
+
apiKeyEnv: 'NVIDIA_API_KEY',
60
+
model: 'meta/llama-3.1-70b-instruct',
61
+
structuredOutputMode: 'json_object',
62
+
prompt: 'Challenge the concept scope and identify missing constraints, hidden assumptions, and where the MVP should be narrower.',
63
+
useTools: false
64
+
},
65
+
{
66
+
name: 'Validation Planner',
67
+
provider: 'nvidia-free',
68
+
endpoint: 'https://integrate.api.nvidia.com/v1',
69
+
apiKeyEnv: 'NVIDIA_API_KEY',
70
+
model: 'meta/llama-3.1-405b-instruct',
71
+
structuredOutputMode: 'json_object',
72
+
prompt: 'Turn the concept into validation work: experiments, acceptance checks, and success criteria before implementation starts.',
73
+
useTools: false
32
74
}
33
-
]
75
+
],
76
+
brainstormSynthesis: {
77
+
name: 'Core Coder',
78
+
provider: 'nvidia-free',
79
+
endpoint: 'https://integrate.api.nvidia.com/v1',
80
+
apiKeyEnv: 'NVIDIA_API_KEY',
81
+
model: 'meta/llama-3.1-405b-instruct',
82
+
structuredOutputMode: 'json_object',
83
+
prompt: 'Synthesize the brainstorm panel into a grounded recommendation, a short priority list, open questions, deferred ideas, and a revised next brief.',
`prr` is a Node.js CLI for running parallel code reviews against multiple NVIDIA-hosted language models.
3
+
`prr` is a Node.js CLI for running parallel code reviews and concept brainstorms against OpenAI-compatible language models, including NVIDIA-hosted and OpenAI backends.
4
+
5
+
## Why multiple reviewers?
6
+
7
+
Different models often produce code that looks similarly correct on the surface, but they reach that result through different internal strengths, assumptions, and blind spots.
8
+
9
+
`prr` uses that diversity on purpose: by collecting multiple independent reviews, you increase the chance that one model notices an edge case, risk, or alternative approach that another model would miss.
4
10
5
11
## Requirements
6
12
- Node.js 18+
@@ -17,6 +23,14 @@ For local development you can run the CLI without publishing:
17
23
node bin/prr.js review src/**/*.js
18
24
```
19
25
26
+
## Fastest way to try `prr`: NVIDIA quick start
27
+
28
+
Most examples in this repo work well with a free NVIDIA Build account.
29
+
30
+
1. Create a free account on NVIDIA: `https://build.nvidia.com/`
31
+
2. After login, go to `https://build.nvidia.com/settings/api-keys` and `Generate API Key`
32
+
3. Copy `.env.example` to `.env`, set `NVIDIA_API_KEY`, then run `node bin/prr.js doctor`
33
+
20
34
## API Keys
21
35
`prr` reads API keys from the configured environment variable name, defaulting to `NVIDIA_API_KEY`.
prompt:'Review this code for concrete bugs and risks with custom rules.',
108
126
maxOutputTokens:2048,
109
127
useTools:true,
110
128
maxToolRounds:6,
111
129
structuredOutputMode:'json_schema'
130
+
},
131
+
{
132
+
name:'Optional Brainstorm',
133
+
provider:'openai',
134
+
endpoint:'https://api.openai.com/v1',
135
+
apiKeyEnv:'OPENAI_API_KEY',
136
+
mode:'suggestions',
137
+
model:'some/model',
138
+
prompt:'Suggest at most 3 grounded follow-up improvements. Do not repeat defects. Include a clear benefit and tradeoff for each suggestion.',
139
+
useTools:true,
140
+
maxToolRounds:4,
141
+
allowSuggestions:true,
142
+
maxSuggestions:3,
143
+
structuredOutputMode:'json_schema'
144
+
}
145
+
],
146
+
brainstormers: [
147
+
{
148
+
name:'Scope Challenger',
149
+
model:'meta/llama-3.1-70b-instruct',
150
+
structuredOutputMode:'json_object',
151
+
prompt:'Challenge the concept scope and identify missing constraints.',
152
+
useTools:false
153
+
},
154
+
{
155
+
name:'Validation Planner',
156
+
model:'meta/llama-3.1-405b-instruct',
157
+
structuredOutputMode:'json_object',
158
+
prompt:'Turn the concept into validation work before implementation starts.',
159
+
useTools:false
112
160
}
113
-
]
161
+
],
162
+
brainstormSynthesis: {
163
+
name:'Core Coder',
164
+
model:'meta/llama-3.1-405b-instruct',
165
+
structuredOutputMode:'json_object',
166
+
prompt:'Synthesize the brainstorm panel into a grounded recommendation and revised next brief.',
167
+
useTools:false
168
+
}
114
169
};
115
170
```
116
171
@@ -127,31 +182,63 @@ Configuration notes:
127
182
-`timeoutMs`: provider request timeout in milliseconds
128
183
-`structuredOutput`: set to `false` to force plain-text parsing even when the provider preset supports structured responses
129
184
-`structuredOutputMode`: override with `json_schema` or `json_object` when the provider supports it
130
-
-`--context-file`: inject a coder-prepared markdown briefing into the review request
185
+
-`--context-file`: inject an optional markdown context file into the review request
131
186
-`maxOutputTokens`: optional per-reviewer cap; omit it to let the provider decide
132
187
-`useTools`: whether the reviewer may fetch file contents on demand
133
188
-`maxToolRounds`: optional per-reviewer cap for tool-call rounds
189
+
-`mode`: reviewer lane, either `defects` or `suggestions` (default `defects`)
190
+
-`allowSuggestions`: when `true`, structured reviewers may return a separate `suggestions` array in addition to defect `findings` (`mode: 'suggestions'` turns this on automatically)
191
+
-`maxSuggestions`: optional cap for reviewer suggestions when `allowSuggestions` is enabled
192
+
-`brainstormers`: separate model panel for `prr brainstorm`, useful for broader or cheaper no-tools model fans
193
+
-`brainstormSynthesis`: optional single-model synthesis stage that runs after `brainstormers` and produces a tighter recommendation plus a revised markdown next brief
134
194
135
195
`reviewSharing` is only effective in `sequential` mode. In `parallel` mode, reviewers run concurrently and do not have prior results to inspect.
136
196
When reviewers run with `useTools: false`, increase `contextMaxChars` if you need more inline source context for larger files.
197
+
`mode: 'suggestions'` makes the reviewer an explicit improvement lane with suggestion-focused prompting and metadata. Structured output still gives the cleanest separation because it can return a dedicated `suggestions` array.
198
+
Recommended pattern: keep normal reviewers in `mode: 'defects'`, then add a separate optional reviewer in `mode: 'suggestions'` with a low `maxSuggestions` cap.
199
+
For GPT-5-family models, `maxOutputTokens` maps to `max_completion_tokens`, which includes reasoning tokens as well as visible output. If you set it too low, the model can spend the whole budget reasoning and return an empty visible response. Prefer leaving it unset or keeping it comfortably above the expected final output size.
137
200
138
201
Reviewer transport overrides inherit from the global config by default, but each reviewer can override:
139
202
-`provider`
140
203
-`endpoint`
141
204
-`apiKeyEnv`
142
205
-`timeoutMs`
143
206
-`structuredOutputMode`
207
+
-`mode`
208
+
-`allowSuggestions`
209
+
-`maxSuggestions`
144
210
145
211
That means one review run can mix free NVIDIA-hosted models with paid OpenAI-compatible backends.
212
+
For brainstorming, it is often cleaner to keep a dedicated config such as `.prrrc.brainstorm.js` and pass it with `--config`.
213
+
214
+
## Brainstorm Mode
215
+
`prr brainstorm <brief.md>` runs a concept-review panel against a markdown brief instead of source files.
216
+
217
+
- input is a markdown concept note, design draft, or implementation plan
218
+
- models come from `brainstormers`, separate from normal `reviewers`
219
+
- an optional `brainstormSynthesis` reviewer can consolidate the panel into a prioritized recommendation and revised markdown next brief
220
+
- no file selection, diff parsing, or tool calls are required
221
+
- this makes it practical to fan out across many cheaper NVIDIA-hosted models
222
+
223
+
Typical flow:
224
+
- draft `concept-brief.md`
225
+
- run `prr brainstorm concept-brief.md`
226
+
- review the optional synthesis output and merge the concrete findings back into the brief
227
+
- re-run until the concept is tighter
228
+
- start implementation and switch to normal `prr review`
146
229
147
230
## Output
148
231
The default report is Markdown with:
149
232
- timestamp
150
-
- analyzed files and line counts
151
-
- one section per reviewer
152
-
- summary totals for findings, errors, and duration
233
+
- one section per brainstormer
234
+
- combined concept findings sections
235
+
- optional synthesis section with priorities and a draft next brief
236
+
- summary totals for brainstorm items, synthesis status, errors, and duration
153
237
154
238
Use `--json` for machine-readable output suitable for automation.
239
+
When synthesis is enabled, `nextBrief` is normalized as markdown text so it can be copied back into the source brief or saved directly for another brainstorm round.
240
+
241
+
Brainstorm runs are persisted under `.prr/brainstorms/<run-id>/` with the same `request.json`, `progress.json`, per-reviewer results, and final `result.json` pattern as review runs.
155
242
156
243
Each normal review run is persisted under `.prr/reviews/<run-id>/` with:
157
244
-`request.json`
@@ -164,7 +251,7 @@ Use `--resume` to continue the latest incomplete review run, or `--resume <run-i
164
251
Use `--dry-run` to inspect what would happen without calling the provider:
165
252
- selected files
166
253
- resolved config and execution mode
167
-
- optional coder-prepared context file
254
+
- optional markdown context file
168
255
- context snippets that would be sent
169
256
- reviewer prompts
170
257
- actual batching plan
@@ -182,36 +269,25 @@ When a provider preset supports structured output, `prr` asks for machine-readab
182
269
183
270
`prr` also builds a deterministic combined-findings view after each run. This deduplicates overlapping findings locally across reviewers before any optional checker/judge stage, so repeated reports do not need another model call just to collapse duplicates.
184
271
185
-
For programmer-agent handoff, the repository also includes [prrrc.coder.guide.md](prrrc.coder.guide.md). It is a repo-local briefing file for tools like Claude or Codex: summarize changed files, call out missing context that reviewers would otherwise guess, and prepare a focused review task list before running `prr`.
186
-
187
-
You can print it directly with:
272
+
Use `--context-file` when the reviewer needs short factual context that is not obvious from the code or diff alone, for example intended behavior, constraints, or already-known tradeoffs.
188
273
189
-
```bash
190
-
prr guide coder
191
-
```
192
-
193
-
The expected workflow is:
194
-
1. print the guide
195
-
2. create a new `review.instructions.md`
196
-
3. pass that file into `prr`
197
-
198
-
And inject a prepared briefing directly into review requests:
A good context file should stay concise and focus on:
282
+
- what changed or what to double-check
283
+
- expected behavior or constraints that are easy to miss
284
+
- intentional tradeoffs that should not be re-raised as defects
285
+
205
286
Stop rule for review loops:
206
287
- stop when no high-severity findings remain and two consecutive rounds add no materially new concrete defects
207
288
- treat repeated low/info or policy-only findings as closure signal, not as mandatory code churn
208
289
- record intentional tradeoffs and rejected findings before closing the loop
209
290
210
-
Decision authority:
211
-
- the core-programmer agent may declare a target "done" once the stop rule is satisfied
212
-
- that declaration should include fix/reject rationale and remaining tradeoffs
213
-
- the human operator can always request another round
214
-
215
291
## Context Strategy
216
292
The default review path is diff-first. If the target files are inside a Git work tree, `prr` sends diff context first and lets the model request file contents or line ranges only when needed. This reduces prompt size and helps avoid truncating large reviews on smaller context-window models.
217
293
@@ -231,22 +307,23 @@ The repository includes a first real benchmark target under [examples/yai-timeou
prr experiment run examples/yai-timeout/yai-timeout.js --preset smoke
239
314
```
240
315
241
-
The first live benchmark on `YaiTimeout` showed that a coder-prepared `review.instructions.md` improved review quality and reduced token waste compared with a plain run.
316
+
Early live benchmarks on `YaiTimeout` showed that a short, focused context note improved review quality and reduced token waste compared with a plain run.
242
317
243
318
The example is now considered closed for this repository cycle: no unresolved high-severity defects remain, and later rounds mostly repeated low-priority or speculative findings.
244
319
245
320
## Auto Mode
246
321
`prr auto run` is the first proof-of-concept orchestration step. It currently:
247
322
- resolves explicit files or Git-changed files
248
323
- prefers the latest experiment recommendation when available
324
+
- swaps tool-enabled recommendations to the verified elite list when the recommended model is not tool-verified
249
325
- otherwise runs the configured reviewers
326
+
- falls back to the top elite tool-capable reviewer when recommendation runs fail with connection-only errors
250
327
- applies the local cleanup and quality scoring pass
0 commit comments