FEAT Beam Search for OpenAIResponseTarget#1346
FEAT Beam Search for OpenAIResponseTarget#1346riedgar-ms wants to merge 150 commits intoAzure:mainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.
You can also share your feedback on Copilot code review. Take the survey.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.
You can also share your feedback on Copilot code review. Take the survey.
| Beam | ||
| BeamReviewer | ||
| BeamSearchAttack | ||
| TopKBeamReviewer |
There was a problem hiding this comment.
The four new entries (Beam, BeamReviewer, BeamSearchAttack, TopKBeamReviewer) are appended at the end of the alphabetically ordered list (after TreeOfAttacksWithPruningAttack), breaking the alphabetical ordering of the existing entries. They should be inserted at their alphabetically correct positions: Beam and BeamReviewer between AttackStrategy and ChunkedRequestAttack, and BeamSearchAttack and TopKBeamReviewer at their respective positions.
There was a problem hiding this comment.
Looks like there's an error in the output?
There was a problem hiding this comment.
It's a warning, but something went wrong on one of the API calls (most likely a grammar had an incorrectly escaped character, although I'm having a hard time running that down). Would you prefer something like that be logged as info rather than warning?
| # Log the attack configuration | ||
| self._logger.info(f"Starting {self.__class__.__name__} with objective: {context.objective}") | ||
|
|
||
| beams = [Beam(id=context.conversation_id, text="", score=0.0) for _ in range(self._num_beams)] |
There was a problem hiding this comment.
are we using the same conversation ID for all of them? Feels like we should create a fresh one per beam / per iteration.
There was a problem hiding this comment.
They don't share (I think that would be a key violation on the database) - it's actually the opposite extreme in that every time a beam updates, it gets a fresh conversation id (hence my concerns above about whether I'm using the database correctly). That said, how this happens (when _propagate_beam_async() re-calls _setup_async() is perhaps too well hidden. I can add comments, but would like to straighten out the higher level question of how I'm spawning new conversations with abandon virst.
| "tool_choice": "required", | ||
| } | ||
|
|
||
| return self._objective_target.fresh_instance(extra_body_parameters=ebp, grammar_name=str(grammar_tool["name"])) |
There was a problem hiding this comment.
The fact that our target is static in its configuration and doesn't allow for custom grammars to be fed in per request is rather annoying here...
The problem is that even if we managed to allow per-request overrides the target identifier wouldn't have those unless we managed to take care of that as well (which is doable). Perhaps the fresh instances are the only way as it stands. Getting a whole new level of appreciation for why the openai client separates such parameters out into the send call rather than the constructor.
CC @rlundeen2
There was a problem hiding this comment.
I'm somewhat frustrated that OpenAI made grammars a tool, rather than a response_format (like a JSON schema).
To the specific case here, there's certainly a reason for always keeping the same tools (not thrashing the KV cache), but it does complicate matters for this scenario
Description
Use the Lark grammar feature of the
OpenAIResponseTargetto create a beam search for PyRIT. This is a single turn attack, where a collection of candidate responses (the beams) are maintained. On each iteration, the model's response is allowed to extend a little for each beam. The beams are scored, with the worst performing ones discarded, and replaced with copies of higher scoring beams.Tests and Documentation
Have basic unit tests of the classes added, but since this requires features only currently in the
OpenAIResponseTargetthere didn't seem much point in mocking that. There is a notebook which runs everything E2E.