fix: recover malformed Jinja template configs

[mldangelo-oai]

Summary

• mark malformed JSON/YAML template configs as scan_outcome=inconclusive when structured extraction fails
• add a bounded raw-content fallback so visible Jinja2 payloads in malformed tokenizer/YAML configs are still analyzed
• run Jinja2 scanner tests in reduced-version CI lanes and add JSON/YAML regression coverage

Validation

• uv run pytest tests/scanners/test_jinja2_template_scanner.py
• uv run ruff format modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
• uv run ruff check --fix modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
• uv run mypy modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
• uv run pytest -n auto -m "not slow and not integration" --maxfail=1
• uv run ruff format --check modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
• uv run ruff check modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
• git diff --check

Summary by CodeRabbit

Release Notes

Bug Fixes

• Improved Jinja2 template configuration handling: visible template payloads are now recovered when standard parsing fails, enabling more robust vulnerability detection even with malformed configurations.
• Enhanced extraction failure handling with secure fail-closed behavior: the scanner now treats extraction failures as inconclusive outcomes rather than failing silently.

[coderabbitai]

Warning

Rate limit exceeded

@mldangelo-oai has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 5 minutes and 2 seconds before requesting another review.

Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 5 minutes and 2 seconds.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 7dc032ac-0a18-4e3b-bbdf-e4b10948b412

📥 Commits

Reviewing files that changed from the base of the PR and between 578794a and 0e1e2a7.

📒 Files selected for processing (2)

• modelaudit/scanners/jinja2_template_scanner.py
• tests/scanners/test_jinja2_template_scanner.py

Walkthrough

The pull request enhances Jinja2 template extraction by treating failures as inconclusive outcomes rather than hard failures. When tokenizer configs are malformed, a raw-template fallback mechanism recovers visible templates from the original content. Extraction failures are logged with structured metadata, and results are finalized based on whether security findings are present.

Changes

Cohort / File(s)	Summary
Documentation CHANGELOG.md	Added bug-fix entry documenting inconclusive handling of malformed configs and raw-template fallback behavior.
Core Scanner Logic modelaudit/scanners/jinja2_template_scanner.py	Refactored extraction pipeline to return both templates and structured failures; introduced inconclusive outcome tracking, raw-text fallback for JSON/YAML parse errors, and conditional result finalization based on security finding presence.
Test Infrastructure tests/conftest.py	Allowlisted test_jinja2_template_scanner.py for execution on Python 3.10/3.12/3.13.
Test Coverage tests/scanners/test_jinja2_template_scanner.py	Updated assertions for malformed config handling to verify inconclusive outcomes, extraction failure metadata, and SSTI detection via raw-template fallback; added edge-case tests for JSON/YAML parse failures and aggregated scan exit codes.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 When configs break and templates fail,
A fallback path tells the tale,
Raw extracts save the scattered scraps,
And inconclusive fills the gaps,
Our scanner now won't let them slip away! ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 70.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix: recover malformed Jinja template configs' directly and specifically summarizes the main change: handling recovery of malformed Jinja2 template configurations through raw-content fallback extraction.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch mdangelo/codex/jinja-parse-fallback-audit

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

Workflow run and artifacts

Performance Benchmarks

Compared 6 shared benchmarks with a regression threshold of 15%.
Status: 0 regressions, 0 improved, 6 stable, 0 new, 0 missing.
Aggregate shared-benchmark median: 686.33ms -> 682.38ms (-0.6%).

Benchmark	Target	Size	Files	Baseline	Current	Change	Status
tests/benchmarks/test_scan_benchmarks.py::test_detect_file_format_safe_pickle	safe_model.pkl	49.4 KiB	1	164.8us	170.6us	+3.5%	stable
tests/benchmarks/test_scan_benchmarks.py::test_validate_file_type_pytorch_zip	state_dict.pt	1.5 MiB	1	49.7us	48.6us	-2.1%	stable
tests/benchmarks/test_scan_benchmarks.py::test_scan_safe_pickle	safe_model.pkl	49.4 KiB	1	28.19ms	27.82ms	-1.3%	stable
tests/benchmarks/test_scan_benchmarks.py::test_scan_mixed_directory	mixed-corpus	1.7 MiB	54	137.19ms	136.01ms	-0.9%	stable
tests/benchmarks/test_scan_benchmarks.py::test_scan_pytorch_zip	state_dict.pt	1.5 MiB	1	33.96ms	33.70ms	-0.8%	stable
tests/benchmarks/test_scan_benchmarks.py::test_scan_duplicate_directory	duplicate-corpus	840.0 KiB	81	486.77ms	484.64ms	-0.4%	stable

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 578794a40d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Analyze bounded prefix instead of skipping malformed large configs

_extract_raw_template_fallback bails out when the file is over 256 KiB and also when decoded fallback text exceeds max_template_size (50k by default). Malformed tokenizer*.json/YAML files above these thresholds never get fallback SSTI analysis, so visible payloads are missed and scans become inconclusive (exit 2) rather than reporting security findings.

Useful? React with 👍 / 👎.