LOADTEST: Shared desired state CTE for Apple profile reconciler#45572
Draft
MagnusHJensen wants to merge 4 commits into
Draft
LOADTEST: Shared desired state CTE for Apple profile reconciler#45572MagnusHJensen wants to merge 4 commits into
MagnusHJensen wants to merge 4 commits into
Conversation
… reconciler ListMDMAppleProfilesToInstallAndRemove used to run the install and remove queries back-to-back inside a single read transaction. The desired-state subquery (a 4-way UNION over mdm_apple_configuration_profiles × hosts × nano_enrollments × nano_devices, plus label-membership joins) is the dominant cost on every tick of the mdm_apple_profile_manager cron, and was being evaluated twice per pass. Fold both halves into one statement that lifts the desired state into a CTE referenced by an install LEFT JOIN and a remove RIGHT JOIN with their respective WHERE shapes preserved, discriminated by a literal `op` column. Add a NO_MERGE optimizer hint on each SELECT in the UNION so MySQL 8 materializes the CTE once instead of inlining it per reference. No behavior change: the per-half SELECT lists and WHERE clauses match the originals, and the existing matchCombinedProfiles assertion in testMDMAppleProfileManagement compares the combined result against the legacy two-query path for every covered case.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #45572 +/- ##
==========================================
+ Coverage 66.71% 66.75% +0.04%
==========================================
Files 2734 2740 +6
Lines 218824 219464 +640
Branches 10947 10947
==========================================
+ Hits 145979 146508 +529
- Misses 59626 59743 +117
+ Partials 13219 13213 -6
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Test renders the combined Apple profile reconciler SQL with literals substituted for placeholders and writes it to disk, so it can be pasted into a MySQL shell after EXPLAIN ANALYZE without needing to hand-assemble the query from generateDesiredStateQuery's template. Runs without a database (string formatting only). Output path defaults to /tmp/reconciler_query.sql; override with DUMP_RECONCILER_QUERY_PATH.
The label-based branches of generateDesiredStateQuery (include-all,
exclude-any, include-any) used to express their predicates as
LEFT JOIN ... GROUP BY ... HAVING count_*. On reconciler-scale fleets
that built large aggregation temp tables — branches 3 and 4 alone
accounted for ~17 of the ~22 seconds spent materializing the
desired-state CTE per cron tick.
Each predicate is straightforwardly expressible with EXISTS / NOT
EXISTS:
- include-all: NOT EXISTS a label the host misses (covers both
"broken label" and "host not a member" via the LEFT JOIN's
NULL semantics).
- exclude-any: NOT EXISTS a label that's broken, that the host
is in, or whose results the host hasn't reported yet.
- include-any: EXISTS a label_membership row joining one of the
profile's include-any labels to the host.
EXPLAIN ANALYZE on a representative dataset (288K desired-state
rows, 343K host_mdm_apple_profiles rows) drops the combined
reconciler query from ~32s to ~21.7s — a ~32% reduction, driven
mostly by branches 3 (7.6s → 4.6s) and 4 (9.6s → 7.4s) no longer
building Aggregate-using-temporary-table nodes. The CTE's
column shape is preserved; the count_* fields are now constant
placeholders since no caller of generateDesiredStateQuery ever
reads them.
Declarations share the same template via dynamicNames, so they
get the same rewrite for free.
Adds a second variant of the desired-state UNION
(generateReconcilerDesiredStateQuery) used only by the cron's combined
install+remove query, with two changes to branches 3 and 4:
- Branch 4 (include-any) uses SELECT STRAIGHT_JOIN to invert the join
order. Today's plan starts from the include-any mel rows (~42) but
fans hosts out per profile via fk_hosts_team_id (319K rows on the
test dataset) and probes label_membership last with ~3.6% selectivity.
Driving instead from mdm_configuration_profile_labels through
label_membership via idx_lm_label_id (~21K rows) and then resolving
mae / hosts / nano_enrollments / nano_devices via PK lookups cut
branch 4 from ~7.4s to ~390ms on the load-test dataset.
- Branch 3 (exclude-any) splits the original single NOT EXISTS
antijoin into two pieces: a hash antijoin against a new
excluded_in_label_pairs CTE (materialized once at the top of the
cron statement, ~20K rows on the test dataset) that captures the
dominant "host is in some label" disqualifier in O(1) hash probes,
plus a smaller per-row NOT EXISTS that only inspects mel/lbl for
the rarer broken-label and stale-label-results cases. Branch 3
drops from ~5.1s to ~4.8s; total query 21.7s → ~13.5s end-to-end.
Per-host callers of generateDesiredStateQuery (worker, orbit, bulkSet
writer path) operate on bounded inputs where the CTE materialization
overhead would not pay back, so they continue to use the unchanged
generateDesiredStateQuery. Only the cron's full-scan path routes
through the new variant.
Removes the throwaway TestDumpReconcilerQuery helper that was added to
serialize the cron query for EXPLAIN ANALYZE iteration. It was useful
during the optimization passes; not needed in the committed tree.
CI Feedback 🧐A test triggered by this PR failed. Here is an AI-generated analysis of the failure:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ListMDMAppleProfilesToInstallAndRemove used to run the install and remove queries back-to-back inside a single read transaction. The desired-state subquery (a 4-way UNION over mdm_apple_configuration_profiles × hosts × nano_enrollments × nano_devices, plus label-membership joins) is the dominant cost on every tick of the mdm_apple_profile_manager cron, and was being evaluated twice per pass.
Fold both halves into one statement that lifts the desired state into a CTE referenced by an install LEFT JOIN and a remove RIGHT JOIN with their respective WHERE shapes preserved, discriminated by a literal
opcolumn. Add a NO_MERGE optimizer hint on each SELECT in the UNION so MySQL 8 materializes the CTE once instead of inlining it per reference.No behavior change: the per-half SELECT lists and WHERE clauses match the originals, and the existing matchCombinedProfiles assertion in testMDMAppleProfileManagement compares the combined result against the legacy two-query path for every covered case.