[BUG] Geneformer finetuning with peft does not work

[dorotat-nv]

BioNeMo Framework Version

67bc54f

Bug Description

It seems that geneformer finetuning with peft does not work according to the error msg

bionemo-framework/sub-packages/bionemo-geneformer/tests/bionemo/geneformer/test_model.py

Line 1021 in 67bc54f

@pytest.mark.skip(reason="PEFT currently broken with fusions activated.")

Steps to Reproduce

This test is skipped since peft is broken

Error Messages and Logs

Docker Image

No response

System Information

Environment Details:

• OS: [e.g., Ubuntu 20.04]
• CPU: [e.g., Intel i9-12900K]
• RAM: [e.g., 64GB]

GPU Details:

• GPU Model: [e.g., NVIDIA RTX 4090]
• GPU Memory: [e.g., 24GB]
• CUDA Version: [e.g., 12.1]
• CUDA Driver: [e.g., 525.85.05]
• cuDNN Version: [e.g., 8.9.0]

Additional Context

No response

[trvachov]

This issue reported a bug in the old bionemo-geneformer sub-package, which has since been fully removed in PR #1398. Geneformer has been migrated to bionemo-recipes/ with a new architecture.

The new Geneformer implementation (bionemo-recipes/models/geneformer/ and bionemo-recipes/recipes/geneformer_native_te_mfsdp_fp8/) does not currently include PEFT/LoRA support. The original code and test that this issue referenced no longer exist.

Closing this as the original bug is no longer applicable. If PEFT support for the new Geneformer recipe is desired, please open a new feature request targeting the bionemo-recipes codebase.