Potential image highlighting memory leak in Firefox

[MADiep]

Expected Behavior

After generating several images then clearing the batch and allowing (or forcing) the browser to garbage collect, the memory footprint of the page should return close to baseline.

Actual Behavior

In Firefox, the page consumes progressively more memory as you generate images but seems to be unable to release it even when performing actions which should theoretically help (such as clearing the batch). I compared the behavior (briefly) with Chrome, but Chrome does not seem to have this issue. This might indicate some weird underlying Firefox bug, but I don't have the expertise to dig any further; however, the problematic code seems to have its roots in the image highlighting logic (see other section).

Steps to Reproduce

1. Using Firefox, have the following debugging pages ready:
   • about:processes
     • For tracking near real-time memory usage
   • about:memory
     • To force memory usage reduction such as garbage collection
2. In SwarmUI, generate a single image. Prompt and parameters likely don't matter, but higher resolution (to exacerbate the issue) and lower steps (the output image has no significance, so might as well make it fast) may be useful.
3. Clear the batch then observe the memory usage in about:processes. In theory, this should settle around the baseline. Note: Since the main window does not clear with the batch, the first image generation is to make sure it doesn't throw off the numbers (i.e. recording baseline with an image instead of the welcome page).
4. Generate several images one at a time. In my experience, about 5 should be good enough to see the trend. Then clear the batch.
5. Using about:memory, force a garbage collection (you can try all of the other ones too if you want; they don't seem to help either way). Then review the memory usage again in about:processes. At this point, memory usage should have went back to baseline but in my observation, it ends up several dozen MB higher.
6. If desired, repeat steps 4 and 5 any number of times and you should observe memory usage get increasingly higher.

Debug Logs

This seems to be a client-side bug.

Other

After fiddling around with the code, I managed to trace the leak to this line of code in the highlightSelectedImage function:

SwarmUI/src/wwwroot/js/genpage/gentab/currentimagehandler.js

Line 1052 in 1fe2f7c

let batchImg = batchContainer.querySelector(`[data-src="${src}"]`);

Honestly, I don't understand why this querySelector alone is causing the leak since I'm not seeing any problems with the variable scoping around it. This leads me to believe Firefox itself is doing something weird, but I can't quite pinpoint the deeper root cause. The only thing I can see that might be interesting about it is that it's probably an abnormally large query compared to typical use cases since it includes an entire image's data: contents.

In any case, I tried refactoring the code to something like this:

    if (batchContainer) {
        let highlighted = false; // optimization to skip full comparison after finding first match
        for (let i of batchContainer.getElementsByClassName('image-block')) {
            if (!highlighted && src == i.dataset.src) {
                i.classList.add('image-block-current');
                highlighted = true;
            }
            else {
                i.classList.remove('image-block-current');
            }
        }
    }

Since the code is already iterating the elements anyway, it just does the selected image check directly inside the loop instead of doing a lookup first and comparing against that. Theoretically, it might even be slightly more optimal since it cuts out the additional traversal done by the querySelector on top of the loop. I'm not a performance expert, nor have I done any benchmarks, so... feel free to take that with a grain of salt. Anyway, this change seems to have resolved (or at least greatly reduced) the memory issues while, as far as I can tell, retaining the original functionality.

[mcmonkey4eva]

Did a quick test, saw a biiitt of memory stacking up (spamming very large images adding like 1 megabyte per 5 of 2048x2048), but the "Minimize memory usage" button on the firefox about:memory menu clears it away back to the starting point consistently.

I tested enabled Auto Swap To Previews since that actually makes the long urls exist (normally urls are quite short, only preview urls need the giant data: blocks), same result, cleans up fine.

That said, not sure why that code even used a query then a loop like that, so I went ahead and swapped the code anyway