[performance] Import the large project #17

snjeza · 2025-11-05T18:59:38Z

Fixes #16

Steps to test:

clone https://github.com/snjeza/bazel-ls-demo-project/
cd bazel-ls-demo-project/large
build the large project:

bazel build //...

add

directories:
#  .  # import everything (remove the dot if this is too much)
  module1496_public
  module0472_public
  module0910
  module0910_public
  module1805_public
  module1805
  
derive_targets_from_directories: true

target_provisioning_strategy: project-per-package

to .eclipse/.bazelproject

add the JVM argumente -Xmx10G -Xms4g -Declipse.bazel.model.cache.expireAfterAccessSeconds=86400 -Djdk.xml.elementAttributeLimit=0 to Eclipse
import the project in Eclipse without the PR
The first time it takes about 25+ minutes to import the project on my machine.
call Sync Bazel Projects View
The action now takes 20+ minutes.
delete all projects
import the project in Eclipse with the PR
The first time it takes about 20 seconds to import on my machine.
call Sync Bazel Projects View
The action now takes 10 seconds.

github-actions · 2025-11-05T19:10:54Z

Test Results

42 files ±0 42 suites ±0 1d 11h 21m 18s ⏱️ + 7h 35m 41s
67 tests ±0 66 ✅ ±0 1 💤 ±0 0 ❌ ±0
134 runs ±0 123 ✅ ±0 11 💤 ±0 0 ❌ ±0

Results for commit babbf0b. ± Comparison against base commit 9542833.

♻️ This comment has been updated with latest results.

snjeza · 2025-12-03T18:24:28Z

The PR includes the folowing changes:

minimized the number of 'bazel query...'
minimized the number of calls of BazelPackageInfo.findProject(BazelPackage)
minimized the number of calls of BazelTargetInfo.findProject(BazelTarget)
minimized the number of calls of IResource.refreshLocal(int, IProgressMonitor)
fixes JvmConfigurator; VM won't be reconfigured whenever the Sync Bazel Projects View action is called
replaces BazelElementOpenJob; with BazelElementInfoCache.putOrGetCached(BazelElement<I, ?>, Supplier)

cc @guw

guw

This PR mixes programming changes with performance fixes. It needs to be separated. So the performance improvements can be inspected separate from job/thread/scheduling changes.

The use of jobs framework is preferred. Thus, any removal of this needs really good data on why this is needed.

guw · 2025-12-08T12:56:37Z

...se.core/src/com/salesforce/bazel/eclipse/core/model/execution/JobsBasedExecutionService.java

                } finally {
                    try {
-                        refreshResources(resourcesToRefresh, monitor);
+                        //refreshResources(resourcesToRefresh, monitor);


Why is this disabled?

You may want to take a look at #16 , issue 4.

guw · 2025-12-08T13:00:20Z

....salesforce.bazel.eclipse.core/src/com/salesforce/bazel/eclipse/core/model/BazelElement.java

-
-        // loads can be potentially expensive; we synchronize on the location
-        var location = getLocation();
-        var openJob = new BazelElementOpenJob<>(location != null ? location : IPath.ROOT, this, infoCache);


Eliminating the job removes transparency and visibility into the package. The job framework provides valuable insights in the Eclipse UI. I would rather like to see JDTLS improve here.

The second important thing provided by jobs framework is synchronization using ISchedulingRule. It is important that at most ONE Bazel command is executed per workspace in general. The Bazel client prevents concurrent executions usually, which can lead to unexpected pauses/waits when multiple Bazel commands are executed by different threads. The job framework (again) helps visualizing this in the UI.

guw · 2025-12-08T13:01:54Z

...el.eclipse.core/src/com/salesforce/bazel/eclipse/core/model/cache/BazelElementInfoCache.java

+     * @return
+     */
+    public abstract <I extends BazelElementInfo> I putOrGetCached(BazelElement<I, ?> bazelElement,
+            Supplier<I> supplier);


The use of supplier adds unwanted complexity. I experimented with suppliers here in the past. However, it was discarded because we do not want to allow computation by direction. We rather like to be in control of when Bazel commands are executed and how.

guw · 2025-12-08T13:06:51Z

...force.bazel.eclipse.core/src/com/salesforce/bazel/eclipse/core/model/BazelWorkspaceInfo.java


    private volatile Map<String, BazelRuleAttributes> externalRepositoryRuleByName;

+    private volatile Map<BazelLabel, IProject> projectByLabel = new HashMap<>();


If we start caching these here we need some logic in the manager to invalidate when users open/close projects.

guw · 2025-12-08T13:08:06Z

...force.bazel.eclipse.core/src/com/salesforce/bazel/eclipse/core/model/BazelWorkspaceInfo.java

                .collect(toMap(BazelRuleAttributes::getName, Function.identity())); // index by the "name" attribute
    }
+
+    public void put(BazelLabel label, IProject project) {


A public put is problematic. What is the reason for this?

guw · 2025-12-08T13:08:24Z

...azel.eclipse.core/src/com/salesforce/bazel/eclipse/core/model/SynchronizeProjectViewJob.java

            // during synchronization resource changes may occur; however, they are triggered by the synchronization activities
            // therefore we suspend cache invalidation of the model due to resource changes
-            workspace.getModelManager().getResourceChangeProcessor().suspendInvalidationFor(workspace);
+            workspace.getModelManager().getResourceChangeProcessor().suspendInvalidationFor(workspace.getModel());


Is this a bug fix?

Yes, it is. I have created #44

guw · 2025-12-08T13:10:02Z

...alesforce.bazel.eclipse.core/src/com/salesforce/bazel/eclipse/core/model/BazelWorkspace.java

+    public Map<BazelPackage, Map<String, Target>> queryForTargetsWithDependencies(BazelWorkspace bazelWorkspace,
+            Collection<BazelPackage> bazelPackages, BazelElementCommandExecutor bazelElementCommandExecutor)
+            throws CoreException {
+        // bazel query 'kind(rule, deps(//foo:all + //bar:all))"'


This looks like an interesting optimization for removing bazel query calls. Please submit as separate PR/commit

I have created #43

snjeza · 2025-12-09T19:45:57Z

@guw I have created separate PRs:

snjeza force-pushed the issue-16 branch 4 times, most recently from e42b1ae to 9ae8b3d Compare November 27, 2025 15:53

guw force-pushed the main branch 5 times, most recently from 6d3a529 to 87cc953 Compare November 28, 2025 15:32

snjeza force-pushed the issue-16 branch 5 times, most recently from 1c87a87 to f4f63c7 Compare December 3, 2025 18:16

snjeza changed the title ~~Import https://github.com/snjeza/bazel-ls-demo-project/tree/master/large~~ [performance] Import the large project Dec 3, 2025

[performance] Import the large project

babbf0b

snjeza force-pushed the issue-16 branch from f4f63c7 to babbf0b Compare December 5, 2025 18:54

guw requested changes Dec 8, 2025

View reviewed changes

This was referenced Dec 8, 2025

Minimize the number of 'bazel-query...' #43

Open

Fix bug in SynchronizeProjectViewJob #44

Open

Fix JvmConfigurator #45

Open

snjeza closed this Dec 9, 2025


		private volatile Map<String, BazelRuleAttributes> externalRepositoryRuleByName;

		private volatile Map<BazelLabel, IProject> projectByLabel = new HashMap<>();

[performance] Import the large project #17

[performance] Import the large project #17

Uh oh!

Conversation

snjeza commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

snjeza commented Dec 3, 2025

Uh oh!

guw left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

snjeza Dec 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

snjeza Dec 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

snjeza commented Dec 9, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

snjeza commented Nov 5, 2025 •

edited

Loading

github-actions bot commented Nov 5, 2025 •

edited

Loading

snjeza Dec 8, 2025 •

edited

Loading

snjeza Dec 8, 2025 •

edited

Loading