Skip to content

[WIP] feat(ci): add asan memory leak test and suppression list#772

Open
johnnyshields wants to merge 4 commits intocompio-rs:masterfrom
johnnyshields:asan-global-memory-leak-test
Open

[WIP] feat(ci): add asan memory leak test and suppression list#772
johnnyshields wants to merge 4 commits intocompio-rs:masterfrom
johnnyshields:asan-global-memory-leak-test

Conversation

@johnnyshields
Copy link
Contributor

@johnnyshields johnnyshields commented Mar 16, 2026

Follow-up from #769

There are some legitimate leaks to fix here, particularly in the compio-quic lib. I've added everything existing to the suppression list for now.

@johnnyshields johnnyshields changed the title feat(ci): add asan memory leak test plus suppression list feat(ci): add asan memory leak test and suppression list Mar 16, 2026
- '**/Cargo.toml'
- 'scripts/asan.sh'
- 'scripts/asan.supp'
- '.github/workflows/ci_test_asan.yml'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only run when .rs or these scripts are touched

@Berrysoft
Copy link
Member

Why did asan test pass before #769 being merged?

@johnnyshields
Copy link
Contributor Author

@johnnyshields
Copy link
Contributor Author

I'll remove it now and we can see it fail.

@johnnyshields
Copy link
Contributor Author

I'll investigate what's going on here later.

@johnnyshields johnnyshields changed the title feat(ci): add asan memory leak test and suppression list [WIP] feat(ci): add asan memory leak test and suppression list Mar 16, 2026
@johnnyshields johnnyshields force-pushed the asan-global-memory-leak-test branch from f87e167 to aa78366 Compare March 17, 2026 06:23
@johnnyshields johnnyshields changed the title [WIP] feat(ci): add asan memory leak test and suppression list feat(ci): add asan memory leak test and suppression list Mar 17, 2026
@johnnyshields
Copy link
Contributor Author

@Berrysoft this is ready for review. You can see the compio driver leaks clearly in the CI run:


Unsuppressed leaks:
  compio-driver/op: AddressSanitizer: 12952 byte(s) leaked in 18 allocation(s).
    compio-driver/src/fd.rs:45:18: >::new
    compio-driver/src/fd.rs:55:14: >::new_unchecked
    compio-driver/src/key.rs:139:22: >>>::new::
    compio-driver/src/key.rs:210:30: ::new::>>
    compio-driver/src/lib.rs:200:14: ::push::>>
  compio-driver/asyncify: AddressSanitizer: 64 byte(s) leaked in 1 allocation(s).
    compio-driver/src/key.rs:139:22: >>::new::
    compio-driver/src/key.rs:210:30: ::new::>
    compio-driver/src/lib.rs:200:14: ::push::>
    compio-driver/tests/asyncify.rs:25:13: asyncify::disable
  compio-ws/websocket: AddressSanitizer: 1192222 byte(s) leaked in 60 allocation(s).
    compio-driver/src/asyncify.rs:98:34: ::new
    compio-driver/src/fd.rs:55:14: >::new_unchecked
    compio-driver/src/key.rs:139:22: >, compio_driver::fd::SharedFd>>>::new::
    compio-driver/src/key.rs:210:30: ::new::>, compio_driver::fd::SharedFd>>
    compio-driver/src/lib.rs:118:21: ::with_builder

Copy link
Member

@Berrysoft Berrysoft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have merged #769 . Could you rebase this PR so that we can see if it passes now?

extract_patterns_for_crate() {
local crate="$1"
[[ -f "$TOML_FILE" ]] || return 0
python3 -c "
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is if possible to exact the python code to separate files?

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an AddressSanitizer/LeakSanitizer test runner for the Rust workspace and wires it into CI, with a suppression config to keep known leaks from failing runs.

Changes:

  • Introduces scripts/asan.sh to build all test binaries with ASan and run them individually, optionally collecting suppression patterns.
  • Adds scripts/asan.toml for per-crate (intended per-test) LSan suppression patterns.
  • Adds a GitHub Actions workflow to run the ASan script on PRs/pushes to master.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File Description
scripts/asan.toml Adds LSan suppression patterns keyed by crate and test file.
scripts/asan.sh New ASan/LSan runner that builds with -Zsanitizer=address and executes each test binary with configured suppressions.
.github/workflows/ci_test_asan.yml New CI workflow invoking ./scripts/asan.sh --ci x86_64-unknown-linux-gnu.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines +133 to +136
if '/' in pkg_id:
pkg = pkg_id.split('#')[0].split('/')[-1]
else:
pkg = pkg_id.split(' ')[0]
Comment on lines +61 to +92
# --- Helper: extract suppression patterns for a crate from asan.toml ---
# Writes leak:pattern lines to stdout.
extract_patterns_for_crate() {
local crate="$1"
[[ -f "$TOML_FILE" ]] || return 0
python3 -c "
import sys, re
crate, toml_path = sys.argv[1], sys.argv[2]
with open(toml_path) as f:
content = f.read()
in_crate = in_patterns = False
for line in content.splitlines():
s = line.strip()
if s.startswith('#'):
continue
m = re.match(r'^\[(.+)\]$', s)
if m:
parts = m.group(1).split('.', 1)
in_crate = (parts[0].strip().strip('\"') == crate)
in_patterns = False
continue
if not in_crate:
continue
if 'patterns' in s and '=' in s:
in_patterns = True
if in_patterns:
for p in re.findall(r'\"([^\"]+)\"', s):
if not p.startswith('tests/'):
print(f'leak:{p}')
if ']' in s:
in_patterns = False
" "$crate" "$TOML_FILE"
Comment on lines +202 to +223
# Run the test binary.
if [[ "${ASAN_VERBOSE:-0}" == "1" ]]; then
"$EXE" --test-threads=1 "${TEST_ARGS[@]}" 2>&1 | tee "$TMPFILE"
else
"$EXE" --test-threads=1 "${TEST_ARGS[@]}" > "$TMPFILE" 2>&1 || true

# Parse results.
RESULT_LINE=$(grep '^test result:' "$TMPFILE" | tail -1 || true)
if [[ -z "$RESULT_LINE" ]]; then
TESTS_RAN=0
else
P=$(echo "$RESULT_LINE" | grep -oP '\d+(?= passed)' || echo 0)
F=$(echo "$RESULT_LINE" | grep -oP '\d+(?= failed)' || echo 0)
I=$(echo "$RESULT_LINE" | grep -oP '\d+(?= ignored)' || echo 0)
TESTS_RAN=$((P + F))
fi

HAS_LEAK=0
if grep -q 'LeakSanitizer: detected memory leaks' "$TMPFILE"; then
HAS_LEAK=1
fi

Comment on lines +145 to +161
BINARY_COUNT=$(echo "$BINARIES" | wc -l)
echo "Built $BINARY_COUNT test binaries."
echo ""

# --- Phase 2: Run each binary with per-crate suppressions ---
LEAK_DETAILS=()
FAIL_DETAILS=()
TMPFILES=()
trap 'rm -f "${TMPFILES[@]}"' EXIT

# Group binaries by crate, generate one supp file per crate.
PREV_CRATE=""
CRATE_SUPP=""
BIN_INDEX=0

while IFS=$'\t' read -r CRATE NAME EXE; do
BIN_INDEX=$((BIN_INDEX + 1))
export ASAN_SYMBOLIZER_PATH="$(command -v "$candidate")"
break
fi
done
@johnnyshields
Copy link
Contributor Author

johnnyshields commented Mar 17, 2026

OK so compio-driver has multiple leaks, and the item I removed on the suppression list seems to have exposed the leaks not fixed by my previous PR 🤦‍♂️, so it's still failing.

I will do a third pass at this whole thing, probably just re-write the whole script in python given the complexity it is evolving into, and address the Copilot comments.

This was easier when I was doing it in a single crate 😅

@johnnyshields johnnyshields changed the title feat(ci): add asan memory leak test and suppression list [WIP] feat(ci): add asan memory leak test and suppression list Mar 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants