Skip to content

Add failing tests for #533: Orchestrator doesn't validate duplicate issue resolution status#534

Draft
prompt-driven-github[bot] wants to merge 1 commit intomainfrom
fix/issue-533
Draft

Add failing tests for #533: Orchestrator doesn't validate duplicate issue resolution status#534
prompt-driven-github[bot] wants to merge 1 commit intomainfrom
fix/issue-533

Conversation

@prompt-driven-github
Copy link
Contributor

Summary

Adds failing tests that detect the bug reported in #533 where the orchestrator blindly trusts LLM duplicate detection without validating that the original issue is actually resolved.

Test Files

  • Unit test: tests/test_issue_533_duplicate_validation.py (10 tests)
  • E2E test: tests/test_e2e_issue_533_duplicate_validation.py (3 tests)

What This PR Contains

  • Failing unit tests that reproduce the reported bug in various scenarios
  • Failing E2E tests that verify the bug at integration level with real prompt loading
  • Tests are verified to fail on current code and will pass once the bug is fixed

Root Cause

The bug is at pdd/agentic_bug_orchestrator.py:441 where the orchestrator checks for the string "Duplicate of #" but doesn't validate whether the original issue is actually resolved. When the LLM fails to follow prompt instructions and outputs "Duplicate of #520" even though #520 is still OPEN, the orchestrator incorrectly triggers a hard stop and closes the duplicate issue.

Test Results on Current Code

  • Unit tests: 8 failures (detecting bug), 2 passes (regression tests)
  • E2E tests: 2 failures (detecting bug), 1 pass (regression test)

Next Steps

  1. Implement the fix at pdd/agentic_bug_orchestrator.py:441
  2. Add validation to check if the original issue is resolved before triggering hard stop
  3. Verify all unit tests pass
  4. Verify all E2E tests pass
  5. Run full test suite for regression checks
  6. Mark PR as ready for review

Fixes #533


Generated by PDD agentic bug workflow

This commit adds comprehensive test coverage to detect the bug where
the orchestrator blindly trusts LLM duplicate detection without
validating that the original issue is actually resolved.

Tests added:
- tests/test_issue_533_duplicate_validation.py: Unit tests (10 tests)
- tests/test_e2e_issue_533_duplicate_validation.py: E2E tests (3 tests)

The tests correctly fail on the current code, demonstrating the bug
at pdd/agentic_bug_orchestrator.py:441 where the hard stop is triggered
without checking if the original issue is resolved.

Related to #533

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pdd bug agents still closes duplicated issues that are not resolved

1 participant