Skip to content

fix: mark partial archive scans inconclusive#907

Open
mldangelo-oai wants to merge 2 commits intomainfrom
mdangelo/codex/archive-routing-audit
Open

fix: mark partial archive scans inconclusive#907
mldangelo-oai wants to merge 2 commits intomainfrom
mdangelo/codex/archive-routing-audit

Conversation

@mldangelo-oai
Copy link
Copy Markdown
Contributor

@mldangelo-oai mldangelo-oai commented Apr 10, 2026

Summary

  • mark fail-closed ZIP, TAR, and 7z archive traversal paths with explicit inconclusive scan metadata
  • route TAR nested member scans through the archive callback boundary used by core scans
  • add regressions for direct scanner metadata, aggregate exit-code behavior, and ZIP cache metadata round-trips

Validation

  • uv run ruff format modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
  • uv run ruff check --fix modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
  • uv run ruff format --check modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
  • uv run ruff check modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
  • uv run mypy modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
  • uv run pytest tests/scanners/test_zip_scanner.py tests/scanners/test_tar_scanner.py tests/scanners/test_sevenzip_scanner.py
  • uv run pytest -n auto -m "not slow and not integration" --maxfail=1

Summary by CodeRabbit

  • Bug Fixes

    • Archive scans (ZIP, TAR, 7z) that cannot be fully traversed are now properly recorded as "inconclusive" in scan metadata instead of being treated as complete outcomes.
    • Scan metadata now includes reasons why archive analysis was incomplete.
  • Tests

    • Added test coverage for incomplete archive scan scenarios and metadata validation.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 10, 2026

Workflow run and artifacts

Performance Benchmarks

Compared 6 shared benchmarks with a regression threshold of 15%.
Status: 0 regressions, 0 improved, 6 stable, 0 new, 0 missing.
Aggregate shared-benchmark median: 692.76ms -> 688.71ms (-0.6%).

Benchmark Target Size Files Baseline Current Change Status
tests/benchmarks/test_scan_benchmarks.py::test_validate_file_type_pytorch_zip state_dict.pt 1.5 MiB 1 48.6us 47.5us -2.4% stable
tests/benchmarks/test_scan_benchmarks.py::test_scan_duplicate_directory duplicate-corpus 840.0 KiB 81 489.89ms 485.88ms -0.8% stable
tests/benchmarks/test_scan_benchmarks.py::test_detect_file_format_safe_pickle safe_model.pkl 49.4 KiB 1 167.8us 166.6us -0.7% stable
tests/benchmarks/test_scan_benchmarks.py::test_scan_pytorch_zip state_dict.pt 1.5 MiB 1 34.72ms 34.57ms -0.4% stable
tests/benchmarks/test_scan_benchmarks.py::test_scan_safe_pickle safe_model.pkl 49.4 KiB 1 28.76ms 28.71ms -0.2% stable
tests/benchmarks/test_scan_benchmarks.py::test_scan_mixed_directory mixed-corpus 1.7 MiB 54 139.17ms 139.33ms +0.1% stable

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Apr 10, 2026

Warning

Rate limit exceeded

@mldangelo-oai has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 6 minutes and 54 seconds before requesting another review.

Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 6 minutes and 54 seconds.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 24321ec7-31f5-4409-894e-a02c378823cf

📥 Commits

Reviewing files that changed from the base of the PR and between b5d9a60 and ba89f91.

📒 Files selected for processing (7)
  • modelaudit/scanners/_archive_outcomes.py
  • modelaudit/scanners/sevenzip_scanner.py
  • modelaudit/scanners/tar_scanner.py
  • modelaudit/scanners/zip_scanner.py
  • tests/scanners/test_sevenzip_scanner.py
  • tests/scanners/test_tar_scanner.py
  • tests/scanners/test_zip_scanner.py

Walkthrough

This PR introduces a mechanism to mark archive scans as inconclusive when analysis cannot be fully completed. Changes include a new utility function to update scan metadata, integration across ZIP/TAR/7z scanners to detect and flag incomplete traversals, a configurable nested scan callback for dispatch flexibility, and comprehensive test coverage validating inconclusive metadata propagation.

Changes

Cohort / File(s) Summary
Incomplete Scan Marking
modelaudit/scanners/_archive_outcomes.py
New function mark_archive_scan_incomplete() that sets analysis_incomplete=True, scan_outcome=INCONCLUSIVE_SCAN_OUTCOME, and maintains a scan_outcome_reasons list in scan metadata.
Archive Scanner Integration
modelaudit/scanners/sevenzip_scanner.py, modelaudit/scanners/tar_scanner.py, modelaudit/scanners/zip_scanner.py
Integrated incomplete marking across multiple early-exit and failure paths (missing dependencies, invalid archives, depth/budget/entry limits, constraint failures). Added _scan_nested_archive_entry() dispatch helper to sevenzip and tar scanners for configurable nested scanning. Updated completion logic to flag incomplete analysis when traversal stops prematurely.
Test Coverage
tests/scanners/test_sevenzip_scanner.py, tests/scanners/test_tar_scanner.py, tests/scanners/test_zip_scanner.py
Extended existing tests and added new ones to validate inconclusive metadata is correctly set when analysis cannot complete, including entry count limit scenarios, nested scan callbacks, and cache roundtrip behavior.
Documentation
CHANGELOG.md
Added entry documenting incomplete archive traversal now marked as "inconclusive" rather than complete.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 In burrows deep where archives dwell,
Incomplete scans now have a tale to tell,
No longer hiding half-dug holes,
We mark them "inconclusive" for cleaner roles,
Three archive friends sing in accord,
Honesty's the greatest reward! 🎉

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'fix: mark partial archive scans inconclusive' directly and clearly describes the main change: ensuring incomplete archive traversals are marked as inconclusive in scan metadata across ZIP, TAR, and 7z formats.
Docstring Coverage ✅ Passed Docstring coverage is 80.77% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch mdangelo/codex/archive-routing-audit

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@mldangelo-oai mldangelo-oai enabled auto-merge (squash) April 10, 2026 16:54
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@modelaudit/scanners/sevenzip_scanner.py`:
- Around line 475-477: The current logic treats any scan_complete=False (driven
by _scan_extracted_file) as a partial traversal and calls
mark_archive_scan_incomplete even when the outer scan failed because of genuine
nested findings; change the condition so mark_archive_scan_incomplete is only
called for true partial/aborted traversals (e.g., budget.should_stop() or
traversal failure) and not when the failure is due to nested errors: update the
if guarding mark_archive_scan_incomplete to also check result.has_errors (or use
a traversal_incomplete flag derived from scan_complete and _scan_extracted_file)
so you only mark incomplete when there are no result.has_errors, then leave
result.finish(...) as-is.

In `@modelaudit/scanners/zip_scanner.py`:
- Around line 471-473: The current logic marks the archive inconclusive whenever
scan_complete is false, but scan_complete is also set false when any nested
member returns success=False (including critical findings) — causing fully
traversed ZIPs with real errors to be labeled INCONCLUSIVE. Change the condition
so mark_archive_scan_incomplete(result, "zip_analysis_incomplete") is only
called when traversal was actually incomplete (e.g., an explicit
analysis_incomplete flag or a nested warning-only fail-closed condition), not
for every nested failure; update the code paths that set/clear scan_complete
(and any nested result flags) so that only a dedicated traversal-incomplete
indicator triggers mark_archive_scan_incomplete before calling
result.finish(success=scan_complete and not result.has_errors).

In `@tests/scanners/test_sevenzip_scanner.py`:
- Around line 115-117: The test asserts for the SevenZip scanner currently check
metadata but do not ensure the result's warning/error flags reflect a
WARNING-only fail-closed path; update the assertions around the failing-py7zr
cases (the block that checks result.metadata["scan_outcome"] ==
INCONCLUSIVE_SCAN_OUTCOME and result.metadata["analysis_incomplete"]) to also
assert result.has_warnings is True and result.has_errors is False (do the same
for the second occurrence of this case later in the file), ensuring ScanResult
uses has_warnings for WARNING-severity failures and has_errors remains reserved
for CRITICAL issues.

In `@tests/scanners/test_tar_scanner.py`:
- Around line 737-740: The test callback nested_scan should match the real
contract: change its signature from nested_scan(_path: str, _config: dict[str,
Any] | None) -> ScanResult to accept a non-optional config (e.g.,
nested_scan(_path: str, _config: dict[str, Any]) -> ScanResult) because
_scan_nested_archive_entry always passes nested_config: dict[str, Any]; update
the nested_scan definition accordingly (it still returns ScanResult and calls
nested_result.finish(...)) to remove the unnecessary Optional type.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 77a1a5c0-c665-4185-bed2-15d31f382515

📥 Commits

Reviewing files that changed from the base of the PR and between f285a05 and b5d9a60.

📒 Files selected for processing (8)
  • CHANGELOG.md
  • modelaudit/scanners/_archive_outcomes.py
  • modelaudit/scanners/sevenzip_scanner.py
  • modelaudit/scanners/tar_scanner.py
  • modelaudit/scanners/zip_scanner.py
  • tests/scanners/test_sevenzip_scanner.py
  • tests/scanners/test_tar_scanner.py
  • tests/scanners/test_zip_scanner.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant