Challenges integrating with rules_buf

[michaelloehnig]

My team and I are evaluating rules_buf for our project and have run into several integration challenges. Would love to get some clarity on the current capabilities and recommendation.

Workspace setup

• Multiple proto trees with different import paths
  • example: directory: /service/common/common.proto import path: company.com/sub//service/common
• Existing Bazel build setup with gazelle directives
• A preference to avoid publishing to BSR for dependency management

Questions/Issues

1. Dependency Management Without BSR

The Buf documentation states that dependencies must be published to BSR:

The value must be a valid path to a BSR module (either the public BSR at buf.build or a private BSR instance). It can't be a local Git reference to a buf.yaml file or a URL path to a Git repo.

Question: Is there any support for local dependencies or workspace-style setups? We have multiple proto modules in the same repo that need to reference each other, but we'd prefer not to publish internal modules to BSR.

2. Gazelle Integration and Ignore Patterns

Questions:

• Does rules_buf respect gazelle:resolve directives for controlling dependency resolution?
• How does it interact with .bazelignore files? We need certain directories to be excluded from processing.

3. Multi-Module Repo Support

Since we have multiple proto trees with different import paths:

• Can rules_buf handle multiple buf modules in a single repository?
• What's the recommended approach for this kind of setup?

Example: We have a repository structure like:

/core      # Contains foundational proto definitions (separate bzlmod module)
/service   # Contains service-specific protos that import from /core

Where the /service protos have imports like:

import "core/types/common.proto";
import "core/config/settings.proto";

Currently, each directory only contains Bazel BUILD files with proto_library definitions - there are no buf.yaml files in these directories yet. The /core directory is configured as a separate bzlmod module, and the /service protos depend on definitions from /core, but we need both to be built together in the same repository without publishing /core to BSR as an external dependency.

4. Well-Known Types Import Resolution

We're encountering errors for standard protobuf types when running bazel run //:buf_breaking:

system error: proto: could not resolve import "google/protobuf/timestamp.proto": not found

Questions:

• How should well-known types (like google/protobuf/*) be handled in rules_buf? From our understanding this should work out the box.
• Are there dependencies or buf modules that need to be explicitly declared for well-known types?

Any guidance, examples, or documentation pointers would be greatly appreciated! Happy to provide more specific details about our setup if that would help.

Thanks!

[srikrsna-buf]

Question: Is there any support for local dependencies or workspace-style setups? We have multiple proto modules in the same repo that need to reference each other, but we'd prefer not to publish internal modules to BSR.

For local, we don't need a special rule. Regular bazel repo rules will work.

• Does rules_buf respect gazelle:resolve directives for controlling dependency resolution?

rules_buf doesn't interfere with proto_library imports unless the gazelle proto plugin is unable to figure it out. If everything is local it should be able to figure it out. You can read more on this here.

• How does it interact with .bazelignore files? We need certain directories to be excluded from processing.

The gazelle:exclude directive can be used, more on this here.

• Can rules_buf handle multiple buf modules in a single repository?

Yes, you can look at this example.

Questions:

• How should well-known types (like google/protobuf/*) be handled in rules_buf? From our understanding this should work out the box.
• Are there dependencies or buf modules that need to be explicitly declared for well-known types?

Dependencies need to be declared but they are handled by gazelle. Otherwise they need to be added from the protobuf repo.

[michaelloehnig]

Thanks for the helpful response!

The reason that I ask about gazelle:resolve is because
We have a local_path_override in our setup:

local_path_override(
    module_name = "org_example_core",
    path = "core",
)

When we run bazel run //:gazelle, it always adds out "core" modules as "@buf_deps//core/...:core_proto", instead of using the reference org_example_core which is local. This causes an issue when running the buf_breaking_test since it attempts to find the modules externally. We need gazelle to use the module_name rather than the path because our .bazelignore filters out the core directory.

Is there a way to configure gazelle to respect the module_name from the local_path_override instead of using the path?

After manually editing the buf_breaking_test definition to use the appropriate module name, we're still getting:

system error: proto: could not resolve import "google/protobuf/duration.proto": not found
--buf-plugin_out: protoc-gen-buf-plugin: Plugin failed with status code 1.

We've tried adding these dependencies to our buf.yaml:

deps:
  - buf.build/googleapis/googleapis
  - buf.build/protocolbuffers/wellknowntypes

But this didn't resolve the import issues. It is mentioned here that these types should work out of the box, is there a way to debug why they are not being found?

For further context here is a slimmed down version of out tree structure:

core
├── BUILD
├── v1
│   └── proto
│       ├── BUILD // proto_library definitions can be found here
│       ├── core.proto // imports and uses all of types
│       └── types
│           ├── BUILD // proto_library definitions can be found here
│           ├── geophys
│           │   ├── BUILD // proto_library definitions can be found here
│           │   ├── coordinates.proto
│           └── physical
│               ├── BUILD // proto_library definitions can be found here
│               └── physical.proto
service
├── common
│   ├── BUILD // proto_library definitions can be found here
│   ├── common.proto // imports core/v1/prot/types/physical

Any suggestions for troubleshooting this would be really helpful!

[ciarand]

Just chiming in here with a few more details:

Our setup:

• We have a bzlmod monorepo that contains a Bazel submodule ("core", imported via a local_path_override) with its own .proto files.
• We have a set of gazelle:resolve directives that make the core Gazelle experience work relatively seamlessly across module boundaries (running Gazelle generates buildable libraries, libraries that depend on targets from both the root module and the core submodule work correctly).
• The "core" submodule directory is listed in the root module's .bazelignore file to work around Targets in nested modules are visible from top module bazelbuild/bazel#21515

What we expected:

• Running Gazelle from the root workspace will curate the targets provided to the buf_breaking_test rule. Targets within the core/ directory should be generated as @org_example_core//target/name

What we're getting:

• The Buf Gazelle plugin seems to generate targets for the root module's buf_breaking_test rule that look like @buf_deps//core/v1/proto:proto_proto

What we've tried:

• Exhaustive list of gazelle:resolve proto directives (seem to be ignored by the Buf Gazelle plugin)

I recognize that this is probably a bit of an unusual case. Our team would be happy to help contribute fixes to make this work, but probably need some guidance on how you'd like cases like this to be handled in the future