Skip to content

updated docker overrides#1381

Merged
kheiss-uwzoo merged 7 commits intoNVIDIA:mainfrom
kheiss-uwzoo:kheiss/docker-overwrite
Feb 23, 2026
Merged

updated docker overrides#1381
kheiss-uwzoo merged 7 commits intoNVIDIA:mainfrom
kheiss-uwzoo:kheiss/docker-overwrite

Conversation

@kheiss-uwzoo
Copy link
Copy Markdown
Collaborator

Summary

  1. Tip in Step 1f (Start core services)

Tip that explains overrides in one sentence and links to the new section:
"Use a docker compose override file to reduce VRAM usage. Override files typically lower per-service memory allocation, batch sizes, or concurrency..."

  1. New section: "Docker Compose override files"
  • Added a dedicated section (after Profile Information, before Specify MIG slices) that:
  • Explains overrides: When default compose exceeds VRAM, override files reduce memory/batch size/concurrency so the pipeline fits on one GPU.
  • Explains usage: Use a second -f with the override file; Docker Compose merges and the override wins.
  • Lists all current override files in a table:
Override file GPU target
docker-compose.a10g.yaml NVIDIA A10G
docker-compose.a100-40gb.yaml NVIDIA A100-SXM4-40GB
docker-compose.l40s.yaml NVIDIA L40S
  • Covers RTX Pro 6000 Server Edition: Notes that for this GPU and others with limited VRAM, users should pick the override that best matches GPU memory (e.g. docker-compose.l40s.yaml or docker-compose.a10g.yaml). (There is no docker-compose.rtx-pro-6000.yaml in the repo.)

Adds examples for A100 40GB, A10G, and L40S.

@kheiss-uwzoo kheiss-uwzoo requested a review from a team as a code owner February 5, 2026 22:21
@kheiss-uwzoo kheiss-uwzoo added the doc Improvements or additions to documentation label Feb 5, 2026
Comment thread docs/docs/extraction/quickstart-guide.md Outdated
Comment thread docs/docs/extraction/quickstart-guide.md Outdated
Copy link
Copy Markdown
Collaborator

@nkmcalli nkmcalli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few suggestions.

Comment thread docs/docs/extraction/quickstart-guide.md Outdated
Comment thread docs/docs/extraction/quickstart-guide.md Outdated
--profile retrieval up
```

**Example — A10G:**
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment here - please adjust and add an intro sentence

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated per comment

--profile retrieval up
```

**Example — L40S:**
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment here - please adjust and add an intro sentence

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated per review comment

kheiss-uwzoo and others added 3 commits February 11, 2026 17:41
Co-authored-by: nkmcalli <nkmcalli@yahoo.com>
Co-authored-by: nkmcalli <nkmcalli@yahoo.com>
Updated per review comments
Copy link
Copy Markdown
Collaborator Author

@kheiss-uwzoo kheiss-uwzoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated file per review comment

@kheiss-uwzoo kheiss-uwzoo merged commit d8e17ad into NVIDIA:main Feb 23, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants