Skip to content

feat: add forecast explain output#299

Merged
ndycode merged 4 commits intomainfrom
feat/pr6-forecast-explain-latest
Mar 23, 2026
Merged

feat: add forecast explain output#299
ndycode merged 4 commits intomainfrom
feat/pr6-forecast-explain-latest

Conversation

@ndycode
Copy link
Owner

@ndycode ndycode commented Mar 22, 2026

Summary

  • add --explain support to codex auth forecast
  • include structured recommendation reasoning in JSON output and an explain section in text output
  • cover the new behavior with the manager CLI test suite

Validation

  • npm run typecheck
  • npm run lint -- lib/forecast.ts lib/codex-manager.ts test/codex-manager-cli.test.ts
  • npm run test -- test/codex-manager-cli.test.ts
  • npm run build
  • node \"scripts/codex-multi-auth.js\" auth forecast --help

note: greptile review for oc-chatgpt-multi-auth. cite files like lib/foo.ts:123. confirm regression tests + windows concurrency/token redaction coverage.

Greptile Summary

this pr adds --explain support to codex auth forecast, surfacing the per-account reasoning behind the recommendation in both text and json output, independent of whether showRecommendations is enabled in dashboard settings. it also fixes a pre-existing bug where runForecast was hardcoded to DEFAULT_DASHBOARD_DISPLAY_SETTINGS instead of loading the real user settings.

  • lib/forecast.ts: adds ForecastExplanation / ForecastExplanationAccount interfaces and a pure buildForecastExplanation() function that maps all evaluated results into a flat snapshot, marking the recommended account as selected: true
  • lib/codex-manager.ts: parses --explain, wires the new explanation into json and text output paths; the explain block is correctly outside the showRecommendations gate (prior concern resolved); loadDashboardDisplaySettings() now called for real instead of defaults — good fix, but adds filesystem i/o even in --json mode where display is never consumed; buildForecastExplanation is also computed unconditionally regardless of --explain
  • test/codex-manager-cli.test.ts: four new vitest cases cover json explain, text explain, explain-with-recommendations-hidden, and concurrent isolation; the concurrent test correctly avoids index-dependent ordering assumptions
  • docs/reference/commands.md: flags table and example updated accurately

Confidence Score: 4/5

  • safe to merge; prior showRecommendations suppression concern is resolved and the two remaining issues are non-blocking inefficiencies
  • core explain logic is correct and well-tested. the DEFAULT_DASHBOARD_DISPLAY_SETTINGS fix is a solid improvement. two p2s remain (unconditional loadDashboardDisplaySettings i/o in json mode, unconditional buildForecastExplanation allocation) but neither causes incorrect behavior. concurrent test coverage is a nice addition.
  • lib/codex-manager.ts — the two inefficiencies around loadDashboardDisplaySettings and buildForecastExplanation being called unconditionally

Important Files Changed

Filename Overview
lib/forecast.ts adds ForecastExplanation / ForecastExplanationAccount interfaces and buildForecastExplanation(). logic is pure: maps all results into a flat snapshot and marks the recommended index. disabled/hardFailure accounts correctly appear as selected: false. no side effects, no shared state.
lib/codex-manager.ts adds --explain flag parsing, wires buildForecastExplanation, and fixes the long-standing bug where runForecast used DEFAULT_DASHBOARD_DISPLAY_SETTINGS instead of the real loaded settings. the explain section is correctly outside the showRecommendations gate (prior concern addressed). two minor inefficiencies: loadDashboardDisplaySettings is called even in --json mode where display is never used, and buildForecastExplanation is computed unconditionally.
test/codex-manager-cli.test.ts four new vitest cases cover: json explain output, text explain output, explain when showRecommendations=false, and concurrent isolation of plain vs explain runs. the concurrent test correctly uses outputs.find rather than index-based ordering, so it's not sensitive to promise resolution order.
docs/reference/commands.md adds --explain to the flags table and notes section; updates the example command. accurate and consistent with the implementation.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[runForecast] --> B[parseForecastArgs]
    B --> C{--help?}
    C -- yes --> Z[print usage, exit 0]
    C -- no --> D[loadDashboardDisplaySettings]
    D --> E[load accounts + quota cache]
    E --> F[evaluateForecastAccounts]
    F --> G[summarizeForecast]
    G --> H[recommendForecastAccount]
    H --> I[buildForecastExplanation\nalways computed]
    I --> J{--json?}
    J -- yes --> K{--explain?}
    K -- yes --> L[JSON output\nwith explanation field]
    K -- no --> M[JSON output\nexplanation: undefined]
    J -- no --> N[text output: per-account rows]
    N --> O{showRecommendations?}
    O -- yes --> P[print Best next account + Why]
    O -- no --> Q[skip recommendation block]
    P --> R{--explain?}
    Q --> R
    R -- yes --> S[print Explain: block\nfor all considered accounts]
    R -- no --> T[done]
    S --> T
Loading

Comments Outside Diff (1)

  1. lib/codex-manager.ts, line 341 (link)

    P1 stale global help text missing --explain

    the global command listing at line 341 still shows the old synopsis without --explain, but the per-command help at line 2327 was correctly updated. a user who runs codex auth --help will never see --explain advertised.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: lib/codex-manager.ts
    Line: 341
    
    Comment:
    **stale global help text missing `--explain`**
    
    the global command listing at line 341 still shows the old synopsis without `--explain`, but the per-command help at line 2327 was correctly updated. a user who runs `codex auth --help` will never see `--explain` advertised.
    
    
    
    How can I resolve this? If you propose a fix, please make it concise.

    Fix in Codex

Fix All in Codex

Prompt To Fix All With AI
This is a comment left during a code review.
Path: lib/codex-manager.ts
Line: 2763

Comment:
**`loadDashboardDisplaySettings` called unconditionally in JSON mode**

`display` is only consumed in the text output path (after the early `return 0` in `if (options.json)`), but the async I/O happens for every invocation — including `--json`. on windows, that's a filesystem read that can be slow or fail in locked/restricted environments, and it's wasted work in any automated pipeline that uses `--json` for machine-readable output.

consider guarding it:

```suggestion
	const display = options.json ? DEFAULT_DASHBOARD_DISPLAY_SETTINGS : await loadDashboardDisplaySettings();
```

or defer loading until the text path is actually reached.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: lib/codex-manager.ts
Line: 2842

Comment:
**`buildForecastExplanation` computed unconditionally**

`explanation` is built on every `forecast` run regardless of whether `--explain` was passed. for a pool with many accounts each carrying multiple `reasons` strings, this allocates and maps needlessly on every plain `forecast` or `forecast --json` invocation.

guard it so the cost is only paid when the output is actually consumed:

```suggestion
	const explanation = (options.explain)
		? buildForecastExplanation(forecastResults, recommendation)
		: null;
```

then update the downstream usages to reference `explanation` conditionally (already done with `options.explain ? explanation : undefined`).

How can I resolve this? If you propose a fix, please make it concise.

Last reviewed commit: "fix: keep forecast e..."

@chatgpt-codex-connector
Copy link

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 22, 2026

Important

Review skipped

Auto reviews are limited based on label configuration.

🚫 Review skipped — only excluded labels are configured. (1)
  • skip-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 05c1defe-388e-4c22-9c3e-90c748909b12

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

adds structured explanation support to the forecast command via a new --explain flag. when enabled, the cli computes and outputs per-account reasoning and recommendation metadata using new types and a transformer function (buildForecastExplanation) for both json and human-readable outputs (lib/forecast.ts:line, lib/codex-manager.ts:line).

Changes

Cohort / File(s) Summary
Forecast explanation infra
lib/forecast.ts
added ForecastExplanation and ForecastExplanationAccount interfaces and implemented buildForecastExplanation(results, recommendation) to convert ForecastAccountResult[] into a structured explanation payload. minor formatting-only signature adjustments.
cli integration
lib/codex-manager.ts
added explain: boolean to ForecastCliOptions, parsed --explain, and updated runForecast() to compute explanation and include it in json output or print an “Explain:” section for text output when enabled.
tests & docs
test/codex-manager-cli.test.ts, docs/reference/commands.md
added json explain test (runs forecast in json explain mode) and a text-mode explain test (prints explain details in text forecast mode). docs updated to document --explain flag. several test mocks and formatting refactors.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Review flags

  • missing regression tests: only json and one text-case added (test/codex-manager-cli.test.ts:line). need extra tests for: human-readable explain formatting variations, recommendation.recommendedIndex === null, and empty/single-account forecasts with --explain enabled (test/codex-manager-cli.test.ts:line).
  • explain output construction risk: human-readable explain rendering is built from explanation.considered in lib/codex-manager.ts:line. verify formatting across account counts and long reason strings and ensure no accidental sensitive data leakage.
  • windows line-ending validation: explain text output may be consumed by external tools; confirm CRLF handling in lib/codex-manager.ts:line and tests on windows CI.
  • concurrency / immutability concern: buildForecastExplanation() iterates and maps results to mutable objects in lib/forecast.ts:line. if forecast results are shared or processing is parallelized, ensure objects are created immutably or cloned to avoid cross-request contamination.
🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ❓ Inconclusive PR description covers summary, validation steps, and includes greptile review findings; however, missing explicit docs/governance checklist, risk level, and rollback plan sections from template. Add explicit checklist items for README/docs updates, risk assessment (low/medium/high), and rollback strategy to align with template structure.
✅ Passed checks (1 passed)
Check name Status Explanation
Title check ✅ Passed Title follows conventional commits format with 'feat:' prefix and clearly summarizes the main change: adding forecast explain output.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/pr6-forecast-explain-latest
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch feat/pr6-forecast-explain-latest

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
lib/codex-manager.ts (1)

2405-2419: ⚠️ Potential issue | 🟡 Minor

duplicate null check at lib/codex-manager.ts:2406-2407.

there are two identical if (!arg) continue; statements. one should be removed.

proposed fix
 	for (let i = 0; i < args.length; i += 1) {
 		const arg = args[i];
 		if (!arg) continue;
-		if (!arg) continue;
 		if (arg === "--live" || arg === "-l") {
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/codex-manager.ts` around lines 2405 - 2419, The loop that reads
command-line tokens creates const arg = args[i] and contains two identical guard
checks if (!arg) continue; — remove the duplicate so there is only a single
null/undefined guard for arg; update the block around const arg = args[i] (the
loop handling args and options.live/options.json/options.explain) to keep one if
(!arg) continue; and leave the subsequent option checks unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@test/codex-manager-cli.test.ts`:
- Around line 689-692: The test currently asserts
payload.explanation.recommendedIndex and that the first considered row is
selected, but doesn't ensure uniqueness; update the test around
payload.explanation.considered (and the recommendedIndex check) to assert that
exactly one entry in payload.explanation.considered has selected === true (e.g.,
count selected entries and expect the count to equal 1) so ambiguous
multiple-selection regressions are caught.
- Around line 643-692: The existing test "runs forecast in json explain mode"
only covers the --json --explain path; add a parallel deterministic regression
test for the non-JSON text explain path by creating a new test (e.g., "runs
forecast in text explain mode") that sets up the same mocked accounts via
loadAccountsMock, spies console.log and console.error, imports
runCodexMultiAuthCli, runs
runCodexMultiAuthCli(["auth","forecast","--explain"]), asserts exit code 0 and
no console.error, and then verifies the console.log output is plain-text
(parse/inspect the string) and contains the expected explanation pieces
(recommended index, two considered entries, first selected true) to exercise the
new text output code path in runCodexMultiAuthCli.

---

Outside diff comments:
In `@lib/codex-manager.ts`:
- Around line 2405-2419: The loop that reads command-line tokens creates const
arg = args[i] and contains two identical guard checks if (!arg) continue; —
remove the duplicate so there is only a single null/undefined guard for arg;
update the block around const arg = args[i] (the loop handling args and
options.live/options.json/options.explain) to keep one if (!arg) continue; and
leave the subsequent option checks unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 002b7936-07da-42d6-bc5c-c3c15c7f8d85

📥 Commits

Reviewing files that changed from the base of the PR and between 1be5e95 and b85ac45.

📒 Files selected for processing (3)
  • lib/codex-manager.ts
  • lib/forecast.ts
  • test/codex-manager-cli.test.ts
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Greptile Review
🧰 Additional context used
📓 Path-based instructions (2)
lib/**

⚙️ CodeRabbit configuration file

focus on auth rotation, windows filesystem IO, and concurrency. verify every change cites affected tests (vitest) and that new queues handle EBUSY/429 scenarios. check for logging that leaks tokens or emails.

Files:

  • lib/forecast.ts
  • lib/codex-manager.ts
test/**

⚙️ CodeRabbit configuration file

tests must stay deterministic and use vitest. demand regression cases that reproduce concurrency bugs, token refresh races, and windows filesystem behavior. reject changes that mock real secrets or skip assertions.

Files:

  • test/codex-manager-cli.test.ts
🪛 ast-grep (0.41.1)
lib/codex-manager.ts

[warning] 964-964: Regular expression constructed from variable input detected. This can lead to Regular Expression Denial of Service (ReDoS) attacks if the variable contains malicious patterns. Use libraries like 'recheck' to validate regex safety or use static patterns.
Context: new RegExp((?:^|\\|)\\s*${windowLabel}\\s+(\\d{1,3})%, "i")
Note: [CWE-1333] Inefficient Regular Expression Complexity [REFERENCES]
- https://owasp.org/www-community/attacks/Regular_expression_Denial_of_Service_-_ReDoS
- https://cwe.mitre.org/data/definitions/1333.html

(regexp-from-variable)

🔇 Additional comments (7)
lib/codex-manager.ts (5)

964-966: static analysis false positive: regexp from variable.

the static analysis tool flagged this as a potential redos risk, but windowLabel is typed as "5h" | "7d" literal union (see line 962). these are fixed strings with no user input, so no redos vulnerability exists here.


2286-2290: explain flag added to ForecastCliOptions.

the interface extension looks correct. default false is appropriate.


2842-2842: explanation built correctly.

the call to buildForecastExplanation is placed after evaluateForecastAccounts and recommendForecastAccount, which is the right ordering.


2854-2868: json output conditionally includes explanation.

the options.explain ? explanation : undefined pattern is correct. when --explain is not passed, the explanation field is omitted from json output entirely (undefined values are stripped by JSON.stringify).


2965-2975: text explain output implementation.

the explain section prints per-account details with selection markers. the labels come from explanation.considered which uses sanitized labels from lib/forecast.ts:288, so no pii leak.

the template literal at line 2972 is quite dense. consider extracting a helper function if this grows more complex, but acceptable for now.

lib/forecast.ts (2)

36-52: clean interface design for explanation payload.

the new ForecastExplanationAccount and ForecastExplanation interfaces are well-structured. omitting internal flags like hardFailure and disabled from the user-facing explanation output is appropriate.


389-408: pure transformation function is correct.

buildForecastExplanation at lib/forecast.ts:389-408 correctly maps results to explanation accounts and marks the selected one via recommendedIndex comparison. the label field at lib/forecast.ts:398 is already sanitized—redactEmails masks email addresses at lib/forecast.ts:288 before results reach this function, so no pii leak concern here.

test coverage exists at test/codex-manager-cli.test.ts:655-692 that exercises the function end-to-end via the "auth forecast --explain" cli flow, validating exit code, error-free execution, and correct payload structure with the considered array and selection logic. the function has no concurrency, auth rotation, or i/o concerns.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

♻️ Duplicate comments (1)
test/codex-manager-cli.test.ts (1)

689-692: ⚠️ Potential issue | 🟡 Minor

tighten selected-account assertions to enforce uniqueness.

test/codex-manager-cli.test.ts:689 (Line 689) still allows ambiguous multi-selection because only the first row is asserted as selected.

proposed assertion hardening
 		expect(payload.explanation.recommendedIndex).toBe(0);
 		expect(payload.explanation.considered).toHaveLength(2);
 		expect(payload.explanation.considered[0]?.selected).toBe(true);
+		expect(payload.explanation.considered[1]?.selected).toBe(false);
+		expect(
+			payload.explanation.considered.filter((entry) => entry.selected),
+		).toHaveLength(1);

As per coding guidelines, test/**: tests must stay deterministic and use vitest. demand regression cases that reproduce concurrency bugs, token refresh races, and windows filesystem behavior. reject changes that mock real secrets or skip assertions.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/codex-manager-cli.test.ts` around lines 689 - 692, The current
assertions only check the first considered row is selected which allows multiple
selected entries; update the test to assert selection is unique by verifying
payload.explanation.considered has exactly one item with selected===true (e.g.,
assert the filtered array length is 1) and that its index equals
payload.explanation.recommendedIndex, and also explicitly assert that all other
considered entries have selected===false; target the symbols
payload.explanation.considered and payload.explanation.recommendedIndex when
making these changes.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@docs/reference/commands.md`:
- Line 50: Add an upgrade-note entry describing the new forecast explain
behavior introduced by the `--explain` CLI flag: update the upgrade notes
section to mention that `--explain` now includes recommendation reasoning in
forecast output and specify any user-visible format changes, and also add a
separate bullet noting the new/updated npm script mentioned in this release;
edit the docs/reference/commands.md upgrade notes to include these two bullets
so README/SECURITY/docs stay consistent with the CLI flag change.

In `@test/codex-manager-cli.test.ts`:
- Around line 694-738: Add a parallel concurrency regression that runs two
simultaneous forecast --explain invocations to ensure output isolation: modify
the test using runCodexMultiAuthCli to invoke Promise.all on two separate calls
(e.g., runCodexMultiAuthCli([...,"forecast","--explain"]) twice) and assert both
return exitCode 0, no console.error calls, and that each console.log call set
contains the expected "Explain:" and account readiness/risk strings without
cross-contamination; ensure mocks (loadAccountsMock) are configured to provide
separate predictable account sets for each parallel run and keep using vitest
spies (vi.spyOn) so the test remains deterministic.
- Around line 643-692: Add a negative regression asserting that when
runCodexMultiAuthCli is invoked with "--json" but without "--explain" the
emitted JSON payload does not include the explanation field: update the test in
test/codex-manager-cli.test.ts to call
runCodexMultiAuthCli(["auth","forecast","--json"]) and parse console.log output,
then assert that payload.explanation is undefined (or not present) and ensure
console.error was not called; this guards against contract drift in
lib/codex-manager.ts where --json gating must omit explanation unless --explain
is passed.

---

Duplicate comments:
In `@test/codex-manager-cli.test.ts`:
- Around line 689-692: The current assertions only check the first considered
row is selected which allows multiple selected entries; update the test to
assert selection is unique by verifying payload.explanation.considered has
exactly one item with selected===true (e.g., assert the filtered array length is
1) and that its index equals payload.explanation.recommendedIndex, and also
explicitly assert that all other considered entries have selected===false;
target the symbols payload.explanation.considered and
payload.explanation.recommendedIndex when making these changes.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 3dec86fb-d67f-4eb5-a53d-92b0b13a1e02

📥 Commits

Reviewing files that changed from the base of the PR and between b85ac45 and 07222d8.

📒 Files selected for processing (2)
  • docs/reference/commands.md
  • test/codex-manager-cli.test.ts
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Greptile Review
🧰 Additional context used
📓 Path-based instructions (2)
docs/**

⚙️ CodeRabbit configuration file

keep README, SECURITY, and docs consistent with actual CLI flags and workflows. whenever behavior changes, require updated upgrade notes and mention new npm scripts.

Files:

  • docs/reference/commands.md
test/**

⚙️ CodeRabbit configuration file

tests must stay deterministic and use vitest. demand regression cases that reproduce concurrency bugs, token refresh races, and windows filesystem behavior. reject changes that mock real secrets or skip assertions.

Files:

  • test/codex-manager-cli.test.ts

@ndycode ndycode added the passed label Mar 22, 2026
@ndycode ndycode merged commit 4473542 into main Mar 23, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant