fix: add standalone-primary pickle migration mode

[mldangelo-oai]

Summary

This PR continues the picklescan audit by tightening the root PickleScanner migration boundary.

The split-package docs said the root scanner should treat the standalone modelaudit-picklescan engine as primary and merge legacy-only compatibility checks into it. A direct default flip is not safe yet: the focused scanner suite produced 58 compatibility failures because callers and tests still depend on legacy-primary result shape. This change makes that future path explicit and testable without breaking current behavior.

Changes:

• Keep the default root pickle scan path legacy-primary for compatibility.
• Add opt-in use_standalone_pickle_primary=True scanner config to exercise the standalone-primary merge path.
• Add pickle_primary_engine metadata so results show whether the root scanner returned the legacy or standalone-primary result.
• Extend scripts/compare_pickle_scanners.py with --include-root and --root-standalone-primary to compare the actual root scanner alongside the pure legacy/package/adapter baselines.
• Add regression tests for default legacy-primary behavior, opt-in standalone-primary merging, and root parity harness reporting.
• Update migration docs, script docs, and the changelog.

Validation

• uv run ruff format --check modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
• uv run ruff check modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
• uv run mypy modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
• uv run pytest tests/scanners/test_pickle_scanner.py tests/scanners/test_picklescan_adapter.py tests/scripts/test_compare_pickle_scanners.py -q --tb=short
  • 410 passed, 14 subtests passed
• uv run pytest -n auto -m "not slow and not integration" --maxfail=1
  • 3336 passed, 75 skipped
• env PYTHONPATH=packages/modelaudit-picklescan/src uv run python scripts/compare_pickle_scanners.py --root-standalone-primary
  • compared 34 fixtures
  • package: 13 match, 21 rule_drift
  • adapter: 23 match, 11 rule_drift
  • root: 26 match, 8 rule_drift
  • safe fixtures: root 6 match, 0 drift

Notes

This is intentionally stacked on top of #901 / mdangelo/codex/picklescan-audit-hardening; it should be reviewed as the migration-boundary layer after the standalone picklescan hardening work.

The remaining root drift is rule-shape/accounting drift, not verdict or exit-status drift. The next audit should close those remaining standalone-primary rule drifts so the default can eventually flip safely.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 4f870e99-e5c2-4926-9e4c-983361af1b0b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch mdangelo/codex/pickle-migration-boundary-audit

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

Workflow run and artifacts

Performance Benchmarks

Compared 6 shared benchmarks with a regression threshold of 15%.
Status: 0 regressions, 0 improved, 6 stable, 0 new, 0 missing.
Aggregate shared-benchmark median: 611.05ms -> 609.91ms (-0.2%).

Benchmark	Target	Size	Files	Baseline	Current	Change	Status
tests/benchmarks/test_scan_benchmarks.py::test_validate_file_type_pytorch_zip	state_dict.pt	1.5 MiB	1	33.8us	35.5us	+5.1%	stable
tests/benchmarks/test_scan_benchmarks.py::test_scan_duplicate_directory	duplicate-corpus	840.0 KiB	81	433.16ms	432.02ms	-0.3%	stable
tests/benchmarks/test_scan_benchmarks.py::test_scan_pytorch_zip	state_dict.pt	1.5 MiB	1	31.04ms	30.96ms	-0.3%	stable
tests/benchmarks/test_scan_benchmarks.py::test_detect_file_format_safe_pickle	safe_model.pkl	49.4 KiB	1	118.1us	118.0us	-0.1%	stable
tests/benchmarks/test_scan_benchmarks.py::test_scan_mixed_directory	mixed-corpus	1.7 MiB	54	121.21ms	121.29ms	+0.1%	stable
tests/benchmarks/test_scan_benchmarks.py::test_scan_safe_pickle	safe_model.pkl	49.4 KiB	1	25.49ms	25.49ms	+0.0%	stable

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f26b1781aa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

modelaudit/modelaudit/scanners/pickle_scanner.py

Lines 4214 to 4219 in 8ac54be

	except Exception as error:
	if isinstance(error, RecursionError) or self._is_pickle_parse_failure(error):
	raise
	self._record_pickle_runtime_error(package_result, error, location=source)
	package_result.finish(success=False)
	return package_result

Ignore legacy fallback crashes in standalone-primary mode

Guard the legacy-exception path behind use_standalone_pickle_primary. In _scan_pickle_stream_with_package_engine, any non-parse exception from legacy fallback always records pickle_scan_runtime_failed and returns success=False before the standalone-primary branch runs. This lets a clean standalone scan fail solely due to legacy compatibility code, breaking standalone-primary ownership semantics.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

modelaudit/modelaudit/scanners/pickle_scanner.py

Lines 4214 to 4219 in bb028cd

	except Exception as error:
	if isinstance(error, RecursionError) or self._is_pickle_parse_failure(error):
	raise
	self._record_pickle_runtime_error(package_result, error, location=source)
	package_result.finish(success=False)
	return package_result

Ignore legacy fallback crashes in standalone-primary mode

In _scan_pickle_stream_with_package_engine, any non-parse exception from the legacy fallback unconditionally records an operational error on package_result and returns failure. That makes use_standalone_pickle_primary=True still fail because of legacy fallback runtime errors, contradicting standalone-primary ownership semantics and potentially forcing exit code 2 on otherwise complete package scans. Fresh evidence: this branch executes before _merge_missing_pickle_checks(..., propagate_fallback_state=False).

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".