Allow to force p2p

[Nero10578]

Mod to allow P2P for custom all-reduce on PCIe only GPUs. Not necessary to have NVLink to have P2P between GPUs.

[gemini-code-assist]

Summary of Changes

Hello @Nero10578, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the flexibility of GPU communication by introducing a mechanism to force Peer-to-Peer (P2P) functionality for custom all-reduce operations, specifically addressing scenarios with PCIe-connected GPUs that might not be recognized as fully connected. This change allows users to leverage P2P benefits even without NVLink, optimizing performance in diverse hardware configurations. Additionally, the PR includes a cleanup by removing several outdated Marlin MoE kernels.

Highlights

• New Environment Variable: Introduced APHRODITE_FORCE_P2P environment variable to explicitly control P2P capability for custom all-reduce kernels.
• Custom All-Reduce Logic Update: Modified the CustomAllReduce initialization to bypass the fully_connected check when APHRODITE_FORCE_P2P is enabled, allowing P2P on PCIe-only GPUs.
• Kernel Removals: Removed several Marlin MoE kernels (kernel_bf16_ite::ku4.cu, kernel_bf16_ite::ku4b8.cu, kernel_bf16_ite::ku8b128.cu, kernel_fp16_ite::ku4.cu, kernel_fp16_ite::ku4b8.cu, kernel_fp16_ite::ku8b128.cu) from the kernels/moe/marlin_moe_wna16/ directory.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a new environment variable APHRODITE_FORCE_P2P to allow bypassing the P2P connectivity check for custom all-reduce kernels, which is particularly useful for setups with PCIe-only GPUs. The implementation is sound, but I've identified a minor issue in custom_all_reduce.py where a check for self.rank == 0 was unintentionally removed. This would cause all ranks to log the same warning, leading to noisy logs. I've provided a suggestion to restore the original logging behavior while also improving the boolean check to be more idiomatic.

[gemini-code-assist]

This change introduces two small issues:

1. The check envs.APHRODITE_FORCE_P2P == 0 is not idiomatic for a boolean value. It's better to use if not envs.APHRODITE_FORCE_P2P: for clarity and standard Python style.
2. The if self.rank == 0: check around logger.warning was removed. This will cause all ranks to log the same warning message, which can be noisy. It's best to restore this check so only rank 0 logs the warning, while all ranks still execute the return statement to disable custom all-reduce when appropriate.

My suggestion below addresses both points.

Suggested change

	if envs.APHRODITE_FORCE_P2P == 0:
	if world_size > 2 and not fully_connected:
	logger.warning(
	"Custom allreduce is disabled because it's not supported on"
	" more than two PCIe-only GPUs. To silence this warning, "
	"specify disable_custom_all_reduce=True explicitly.")
	return
	return
	if not envs.APHRODITE_FORCE_P2P:
	if world_size > 2 and not fully_connected:
	if self.rank == 0:
	logger.warning(
	"Custom allreduce is disabled because it's not supported on"
	" more than two PCIe-only GPUs. To silence this warning, "
	"specify disable_custom_all_reduce=True explicitly.")
	return