[bugfix]: Add compatibility handling for Qwen3.5 GatedDeltaNet padding-free training and fix create_causal_mask patch when cache_positions removed in transformers >5.3.0#202
Draft
meichangsu1 wants to merge 6 commits into
Conversation
added 3 commits
May 22, 2026 19:16
Add `_call_with_supported_kwargs` utility to filter out unsupported keyword arguments when calling forward methods, preventing errors from incompatible function signatures. This fixes issues where `origin_forward` methods may not accept all passed kwargs.
…handling - Add `_call_with_supported_kwargs` and `_call_create_causal_mask` helpers to filter unsupported kwargs - Rename `cache_position` parameter to `q_length` in flash_attention_mask and sdpa_mask for clarity - Fix device detection in sdpa_mask when `q_length` is not a tensor - Ensure compatibility with models that don't accept `cache_position` in causal mask functions
…ill path In sequence parallel training, when newer Transformers versions pass q_length/q_offset instead of cache_position, the causal mask creation may still see the local shard length. This change restores the global query length for the no-cache prefill path while keeping cache/sliding paths with their upstream offsets. Also refactor GDN padding-free detection to use transformers version check instead of source inspection, supporting transformers >= 5.9.0.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces compatibility helpers and updates masking and forward logic to support varying transformers library versions, specifically handling changes in argument signatures such as cache_position. The review feedback primarily focuses on optimizing performance by caching inspect.signature results and moving signature checks out of hot-path function definitions to avoid significant overhead during the model's forward pass.
Comment on lines
+181
to
+186
| if 'cache_position' in inspect.signature(masking_utils.origin_create_causal_mask).parameters: | ||
| cache_position_or_past_key_values = torch.arange( | ||
| 0, | ||
| input_embeds.shape[1], | ||
| device=input_embeds.device, | ||
| ) |
Contributor
There was a problem hiding this comment.
Similar to the optimization suggested for sdpa_mask, the signature of masking_utils.origin_create_causal_mask should be inspected once outside the create_causal_mask function to avoid overhead in the forward pass.
added 2 commits
May 24, 2026 15:32
The version check for transformers >= 5.9.0 was removed because it is no longer needed. The GDN padding-free patch should always be applied regardless of the transformers version, as the native support check is handled elsewhere or the patch is required for all versions.
Add version check to only apply the chunk_gated_delta_rule cu_seqlens patch when using transformers versions below 5.9.0, preventing compatibility issues with newer releases.
meichangsu1
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR type
PR information
This PR fixes padding-free and sequence-parallel compatibility issues with newer Transformers versions.
Main changes:
when transformers >5.3.0 ,cache_positoins is removed in this pr,so we :
cu_seq_lens_qin this prExperiment results
end to end test results as followed:
sp:
padding_free :