Skip to content

HG - Hardware-specific sharding test and instructions with torchrun#1479

Merged
cirquit merged 120 commits intomainfrom
rsy/hg-model-hardware-test
Mar 19, 2026
Merged

HG - Hardware-specific sharding test and instructions with torchrun#1479
cirquit merged 120 commits intomainfrom
rsy/hg-model-hardware-test

Conversation

@rsyue
Copy link
Copy Markdown
Contributor

@rsyue rsyue commented Dec 10, 2025

What does this PR do? Please describe:
When more than 1 GPU present, model sharding can be tested with torchrun --nproc-per-node 8 test_hg_factory.py

Does your PR introduce any breaking changes? If yes, please list them:
None aware of

Check list:

  • Was the content of this PR discussed and approved via a GitHub issue? (no need for typos or documentation improvements)
  • Did you read the contributor guideline?
  • Did you make sure that your PR does only one thing instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests?
  • Did you verify new and existing tests pass locally with your changes?
  • Did you update the CHANGELOG? (no need for typos, documentation, or minor internal changes)

zyaoj and others added 24 commits September 20, 2025 11:48
@zyaoj
draft hg model integration
240440c
@rsyue
Handled edge case (no safe serialization possible for Qwen special mo…
1b3848c 
…del) and changed allowed patterns to allow for the json index file
@rsyue
Updated docs prior to merging sharder deprecation
caf6519
@rsyue
added sharding capabilities and a test script
b6044af
@rsyue
v1 of sharding complete for Qwen2.5 Omni (MHA layers and FFNs, gated …
6a61d99 
…projs are sharded in fs2)
@rsyue
Fixed parenthesis issues, cpu edge case (FakeGang only)
a3f43e8
@rsyue
Working first implementation of hf model sharding - Qwen2.5-Omni mode…
b83c2b4 
…ls supported
@annasun28
wip with torch DataLoader
8a3dd41
@rsyue
Completed manual sharding strategy from GPU to GPU
e7cd277
@annasun28
enable fsdp for hg model
d9d1545
@annasun28
revert DataPipelineReader changes
da929c3
@annasun28
Merge remote-tracking branch 'origin/rsy/hg-model-v1' into ays/hg-mod…
7b61ccd 
…el-v2
@annasun28
fix fsdp issue
70ee448
@annasun28
update fsdp for v1+v2
c49d030
@annasun28
lint
1a9b1a3
@annasun28
lint/format
37266b4
@rsyue
Added gangs to the model loading args in test
71bb1b8
@rsyue
Cleaned up comment tags
d8d1958
@rsyue
Removed print debug statements
6fcddc7
@rsyue
Updated unit tests to include model sharding - runs if more than one …
c0a6f07 
…GPU present.
@rsyue
Added hardware-dependent sharding test and instructions
895bb79
@rsyue
Moved hg.py to utils to prevent conflicts with hg/__init__.py; update…
fd51539 
…d references; new hg doc added
@rsyue
Reworked testing and some gang logic for HG Model Factory
990cdae
@rsyue
Fixed LocalFileSystem bug in updated hg_model_hub loader
d26616b
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Dec 10, 2025
rsyue and others added 24 commits March 18, 2026 15:28
@rsyue @cirquit
Update src/fairseq2/models/hg/adapter.py
6b253b8 
Co-authored-by: Alexander Erben <aerben@meta.com>
@rsyue @cirquit
Update src/fairseq2/models/hg/adapter.py
e53e000 
Co-authored-by: Alexander Erben <aerben@meta.com>
@rsyue @cirquit
Update src/fairseq2/models/hg/adapter.py
1c9fab5 
Co-authored-by: Alexander Erben <aerben@meta.com>
@rsyue @cirquit
Update src/fairseq2/models/hg/adapter.py
f659874 
Co-authored-by: Alexander Erben <aerben@meta.com>
@rsyue @cirquit
Update src/fairseq2/models/hg/adapter.py
341c2bc 
Co-authored-by: Alexander Erben <aerben@meta.com>
@rsyue @cirquit
Update src/fairseq2/models/hg/adapter.py
9048cf4 
Co-authored-by: Alexander Erben <aerben@meta.com>
@rsyue @cirquit
Update src/fairseq2/models/hg/adapter.py
e7969a3 
Co-authored-by: Alexander Erben <aerben@meta.com>
@rsyue @cirquit
Update src/fairseq2/models/hg/adapter.py
905ccff 
Co-authored-by: Alexander Erben <aerben@meta.com>
@rsyue @cirquit
Update src/fairseq2/models/hg/adapter.py
5787190 
Co-authored-by: Alexander Erben <aerben@meta.com>
@rsyue @cirquit
Update src/fairseq2/models/hg/adapter.py
4f34539 
Co-authored-by: Alexander Erben <aerben@meta.com>
@rsyue @cirquit
Update src/fairseq2/models/hg/api.py
81bd30a 
Co-authored-by: Alexander Erben <aerben@meta.com>
@rsyue @cirquit
Update src/fairseq2/models/hg/config.py
131c94a 
Co-authored-by: Alexander Erben <aerben@meta.com>
@rsyue @cirquit
Update src/fairseq2/models/hg/factory.py
a45a978 
Co-authored-by: Alexander Erben <aerben@meta.com>
@rsyue @cirquit
Apply suggestions from code review
83d90ef 
Co-authored-by: Alexander Erben <alex.erben@tum.de>
@rsyue @cirquit
Update src/fairseq2/models/hg/factory.py
15c0364 
Co-authored-by: Alexander Erben <aerben@meta.com>
@rsyue @cirquit
Update src/fairseq2/models/hg/factory.py
a2b8c1b 
Co-authored-by: Alexander Erben <aerben@meta.com>
@rsyue @cirquit
Update src/fairseq2/models/hg/factory.py
134b7d3 
Co-authored-by: Alexander Erben <aerben@meta.com>
@rsyue @cirquit
Update src/fairseq2/models/hg/factory.py
0360e70 
Co-authored-by: Alexander Erben <aerben@meta.com>
@rsyue @cirquit
Update src/fairseq2/models/hg/factory.py
4fee46e 
Co-authored-by: Alexander Erben <aerben@meta.com>
@rsyue @cirquit
Update src/fairseq2/models/hg/factory.py
2b36b15 
Co-authored-by: Alexander Erben <aerben@meta.com>
@rsyue
Revert data test changes to match main
42e9cde
@rsyue
Revert formatting-only changes in non-hg files to match main
acfa8c4 
Restore column alignment in sharder files, structured.py, and
revert style-only change in basic.py and parquet test.
@rsyue
Merge branch 'main' into rsy/hg-model-hardware-test
3cce075
@rsyue
Fix isort import ordering in hg __init__.py
fc97f73
@cirquit cirquit self-requested a review March 19, 2026 00:14
cirquit
cirquit approved these changes Mar 19, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@cirquit cirquit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Nice work on addressing all of these.

@cirquit cirquit merged commit 91d24c6 into main Mar 19, 2026
19 checks passed
@rsyue rsyue mentioned this pull request Mar 19, 2026
Closed
cirquit added a commit that referenced this pull request Mar 25, 2026
@cirquit
v0.8 regressions, backward compat shims, and release prep (#1495)
d0ec690 
**Summary**
- Fix test and lint regressions introduced in v0.8 development
- Add backward compatibility shims for breaking API changes
- Update CHANGELOG and README for v0.8 release

**Regressions**
- Fix `test_get_shard_dims_work` device mismatch when running with `--device cuda`
- Replace deprecated `datetime.utcnow()` with `datetime.now(timezone.utc)`
- Bump `black` to `~=26.3` (CVE fix) and reformat lines that the new parser rejects
- Fix `Flash3SDPA` to support `flash-attn-3` v3.0.0 API (#1495)
- Pin `pandas~=2.2` for Python 3.12 compatibility

**Backward compatibility shims**
- Add re-export shims for `fairseq2.recipe.validator` and `fairseq2.recipe.task` (#1417)
- Add deprecated `resolve_optional()` on `DependencyResolver` (#1462)
- Add deprecated `ModelCheckpointError` alias for `CorruptModelCheckpointError` (#1475)

**Release prep**
- Update CHANGELOG with missing entries, PR references, and new features (#1479, #1496)
- Add v0.7 and v0.8 rows to README version matrix
YunchaoYang pushed a commit that referenced this pull request Mar 31, 2026
@cirquit @YunchaoYang
v0.8 regressions, backward compat shims, and release prep (#1495)
46bfb8a 
**Summary**
- Fix test and lint regressions introduced in v0.8 development
- Add backward compatibility shims for breaking API changes
- Update CHANGELOG and README for v0.8 release

**Regressions**
- Fix `test_get_shard_dims_work` device mismatch when running with `--device cuda`
- Replace deprecated `datetime.utcnow()` with `datetime.now(timezone.utc)`
- Bump `black` to `~=26.3` (CVE fix) and reformat lines that the new parser rejects
- Fix `Flash3SDPA` to support `flash-attn-3` v3.0.0 API (#1495)
- Pin `pandas~=2.2` for Python 3.12 compatibility

**Backward compatibility shims**
- Add re-export shims for `fairseq2.recipe.validator` and `fairseq2.recipe.task` (#1417)
- Add deprecated `resolve_optional()` on `DependencyResolver` (#1462)
- Add deprecated `ModelCheckpointError` alias for `CorruptModelCheckpointError` (#1475)

**Release prep**
- Update CHANGELOG with missing entries, PR references, and new features (#1479, #1496)
- Add v0.7 and v0.8 rows to README version matrix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@artemru artemru artemru left review comments

@cbalioglu cbalioglu Awaiting requested review from cbalioglu cbalioglu is a code owner

@MartinGleize MartinGleize Awaiting requested review from MartinGleize MartinGleize is a code owner

@zyaoj zyaoj Awaiting requested review from zyaoj zyaoj is a code owner

+1 more reviewer

@cirquit cirquit cirquit approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@rsyue @artemru @cirquit @zyaoj @annasun28