feat: per-module library dependency filter via ocamldep BFS#14516
Draft
robinbb wants to merge 1 commit into
Draft
feat: per-module library dependency filter via ocamldep BFS#14516robinbb wants to merge 1 commit into
robinbb wants to merge 1 commit into
Conversation
This was referenced May 13, 2026
robinbb
force-pushed
the
robinbb-14492-l3-includes-refactor
branch
3 times, most recently
from
d3418da to
246a914
Compare
robinbb
force-pushed
the
robinbb-14492-l4-tight-branch
branch
from
da056f0 to
a0893d7
Compare
robinbb
force-pushed
the
robinbb-14492-l3-includes-refactor
branch
from
246a914 to
d88da28
Compare
robinbb
force-pushed
the
robinbb-14492-l4-tight-branch
branch
2 times, most recently
from
e38c928 to
4e63363
Compare
Activates the tight branch in [lib_deps_for_module]: a per-module BFS over the cross-library dependency graph (built from each lib's [ocamldep -modules] output, normalised through [build_lib_index]'s post-pp module map) produces the set of dep-lib modules actually referenced by the consumer module. The compile rule sees only those [.cmi]/[.cmx] files; sibling-module recompilations on unreferenced dep-lib cmi changes drop out. Include flags are still the cctx-wide [-I]/[-H] in this layer; the filtered include flags ship separately. Wrapped-lib soundness recovery, virtual-impl gating on the deps side, ppx-runtime force-glob, and the new soundness test fixtures ship in a follow-up — this commit leaves five existing cram tests broken ([auto-wrapped-child-reexport.t], [ppx-runtime-libraries.t], [virtual-library.t], [wrapped-closure-precision.t], [wrapped-reexport-via-open-flag.t]) that the soundness recovery restores. [Module_compilation]: - [union_module_name_sets_mapped]: parallel fold over a list of [Module_name.Set.t] producers. - [module_kind_is_filterable]: predicate excluding kinds whose dep story is handled outside the BFS ([Root], [Wrapped_compat], [Impl_vmodule], [Virtual], [Parameter]). - [cross_lib_tight_set]: BFS expanding through the lib_index's [(lib, entry)] pairs, reading each entry's impl + intf [ocamldep] output. Non-tight-eligible libs terminate chains. - [lib_deps_for_module]: replaces the scaffold body. A [can_filter] guard (consumer-side virtual / parameter, dummy dep graph, module kind, [Module.has m ~ml_kind]) falls back to glob; otherwise runs the BFS, classifies libs via [Lib_file_deps.Lib_index.filter_libs_with_modules], and emits specific-file deps for tight libs + glob deps for non-tight / unreached-non-eligible libs. Returns the cctx-wide [Includes]; filtered include flags follow in a later layer. [Compilation_context.create]: peek [direct_requires] / [hidden_requires] and pass [has_library_deps] to [Dep_rules.rules]. Single-module stanzas with library deps now produce real dep graphs (the filter needs them). [Dep_rules.rules]: gate the singleton short-circuit on [(not has_library_deps) || for_ = Melange]. Other singletons fall through to the full dep-graph build. [Ocaml_flags]: [extract_open_module_names] surfaces [-open Foo] references that ocamldep doesn't see; they join [BFS]'s initial frontier. [Virtual_rules]: [is_virtual_or_parameter] — true for virtual impls and parameter cctxs; used by [can_filter] to suppress per-module filtering on consumer cctxs whose dep story [Dep_rules] handles specially. [Parameterised_rules]: pass [~has_library_deps:true] to the [Dep_rules.rules] call; conservative — the dep-rules path here serves external parameterised libs whose dep set is built from generated sources. Tests: rebuild-precision promotions for the existing modified-test set in #14492 — cram outputs reflect the tighter dep / rebuild behavior that L4 already produces. New soundness test fixtures (and the two tests gated on filtered include flags, [per-module-include-flags.t] / [add-unreferenced-sibling-lib.t]) are deferred to their respective follow-ups. Signed-off-by: Robin Bate Boerop <me@robinbb.com>
robinbb
force-pushed
the
robinbb-14492-l4-tight-branch
branch
from
4e63363 to
4c95b95
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Layer 4 of 9 of #14492. The core BFS. NOTE: this is the only layer that is expected to fail tests. This cannot be merged without layer 5.
Module_compilation.lib_deps_for_moduleactivates the tight branch: a per-module BFS over the cross-library dep graph (cross_lib_tight_set) computes the dep-lib modules the consumer actually references. The compile rule sees only those cmi/cmx files.can_filterfalls back to glob for Melange, dummy dep graphs, synthesised modules, non-filterable kinds, and consumer-side virtual / parameter cctxs.Dep_rules.rulesgates its singleton short-circuit on~has_library_deps || for_ = Melange.Compilation_context.createcomputes and passes the flag.Ocaml_flags.extract_open_module_namessurfaces-open Fooreferences for the BFS frontier.Virtual_rules.is_virtual_or_parameterpowers the consumer-sidecan_filtercheck.Parameterised_rulespasses~has_library_deps:trueto itsDep_rules.rulescall.Includes are still the cctx-wide
-I/-H; filtered include flags ship in layer 6. Soundness recovery for wrapped libs / ppx-runtime / virtual-impl deps ships in layer 5.Five existing cram tests fail on this PR; layer 5's soundness recovery restores them without test file changes:
test/blackbox-tests/test-cases/per-module-lib-deps/auto-wrapped-child-reexport.ttest/blackbox-tests/test-cases/per-module-lib-deps/ppx-runtime-libraries.ttest/blackbox-tests/test-cases/per-module-lib-deps/virtual-library.ttest/blackbox-tests/test-cases/per-module-lib-deps/wrapped-closure-precision.ttest/blackbox-tests/test-cases/per-module-lib-deps/wrapped-reexport-via-open-flag.tStack: rebases on #14515. Next: #14517.
Part of #14492. Related to #4572.