[tx] Implement Qwen 3.5 model architecture

[pcmoritz]

This PR implements the Qwen 3.5 model architecture, supporting mixed linear and full attention layers. For now we don't stack the layers yet to keep it simple. This PR also doesn't include MoE support yet.

Here are some examples you can run on 8xH100:

Qwen/Qwen3.5-27B Model

uv run --extra gpu --extra tinker -m skyrl.tinker.api     --base-model Qwen/Qwen3.5-27B     --backend-config '{"max_lora_adapters": 2, "max_lora_rank": 1, "tensor_parallel_size": 8, "train_micro_batch_size": 1, "shard_attention_heads": false}'

and then

export TINKER_API_KEY="tml-dummy"
uv run --with wandb --with tinker sl_loop.py     base_url=http://localhost:8000     model_name=Qwen/Qwen3.5-27B lora_rank=1 max_length=128 train_on_what=LAST_ASSISTANT_MESSAGE

Qwen/Qwen3.5-4B Model

uv run --extra gpu --extra tinker -m skyrl.tinker.api     --base-model Qwen/Qwen3.5-4B     --backend-config '{"max_lora_adapters": 2, "max_lora_rank": 1, "tensor_parallel_size": 8, "train_micro_batch_size": 1, "shard_attention_heads": false}'

and then

export TINKER_API_KEY="tml-dummy"
uv run --with wandb --with tinker sl_loop.py     base_url=http://localhost:8000     model_name=Qwen/Qwen3.5-4B lora_rank=1 train_on_what=LAST_ASSISTANT_MESSAGE max_length=512

RL example

uv run --extra gpu --extra tinker -m skyrl.tinker.api \
    --base-model Qwen/Qwen3.5-2B \
    --backend-config '{"max_lora_adapters": 3, "max_lora_rank": 1, "tensor_parallel_size": 8, "train_micro_batch_size": 1, "sample_max_num_sequences": 64, "shard_attention_heads": false}' > out.log

export TINKER_API_KEY="tml-dummy"
uv run --with wandb --with tinker rl_loop.py     base_url=http://localhost:8000     model_name="Qwen/Qwen3.5-2B"     lora_rank=1 max_tokens=1024

---

[pcmoritz]

/gemini review

[pcmoritz]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces the Qwen 3.5 model architecture, which supports a mix of full attention and linear attention layers. The implementation is comprehensive, including changes to the model configuration, KV caching for the new layer types, and extensive tests comparing against the reference Hugging Face implementation. My review focuses on improving the robustness of the configuration handling and fixing a potential bug in model initialization. Overall, this is a solid contribution.

[gemini-code-assist]

Modifying the config object in-place by setting config.layer_types can lead to unexpected side effects if the same config object is used elsewhere. A cleaner approach is to avoid mutating the config. You could compute layer_types and store it as a member of this class, then pass the specific layer_type to each Qwen3_5DecoderLayer during its initialization. This would require a small change to Qwen3_5DecoderLayer.__init__ to accept layer_type as an argument.