Fix stack-dependent related_integrations.version export by Mikaayenson · Pull Request #6208 · elastic/detection-rules

Mikaayenson · 2026-05-27T23:06:59Z

Issue link(s): Resolves #5601

Summary - What I changed

find_least_compatible_version picked a single ^X.Y.Z anchor from the build stack, so the same rule id shipped different related_integrations.version strings on 8.19 vs 9.x backports while the prebuilt rule version integer stayed the same.

This replaces that with find_compatible_version_range, which emits a stack-invariant OR range from integration manifests (for example ^8.2.0 || ^9.0.0 || ^10.0.0 for endpoint). policy_templates are unioned across manifest anchors instead of parsing the OR string.

Performance: the new path resolves more stack lines per package. Informal local timing was roughly 2 to 9x median vs the old function on endpoint / aws / windows manifests, still sub-10ms per call and once per rule at package build. Not expected to move build-release wall time in a meaningful way.

How To Test

make deps
make pytest
python -m detection_rules view-rule rules_building_block/defense_evasion_download_susp_extension.toml

Spot check export: rule 8d366588-cbd6-43ba-95b4-0971c3f906e5 should show ^8.2.0 || ^9.0.0 || ^10.0.0 for endpoint on every release line.

  "related_integrations": [
    {
      "package": "endpoint",
      "version": "^8.2.0 || ^9.0.0 || ^10.0.0"
    }
  ],

Export toml and import into kibana to ensure it applies successfully.

uv run python -m detection_rules export-rules-from-repo -f rules_building_block/defense_evasion_download_susp_extension.toml

20260528T154018L.ndjson.txt

Backport validation: cherry-pick this PR onto each release branch (or apply the four changed files from 73c07800d), then run the same pytest commands. No version.lock changes required for that pass.

Backport `8.19` (origin/8.19 + PR code)

$ pytest -q
FAILED tests/test_all_rules.py::TestRuleMetadata::test_rule_change_has_updated_date
1 failed, 221 passed, 19 skipped, 2 subtests passed in 69.53s (0:01:09)

# same failure on vanilla origin/8.19 (diffs rules/ vs origin/main, not introduced here)

$ pytest -q -k "not test_rule_change_has_updated_date"
221 passed, 19 skipped, 1 deselected, 2 subtests passed in 65.20s (0:01:05)

$ pytest tests/test_integrations.py::TestFindCompatibleVersionRange -q
.......                                                                  [100%]
7 passed in 0.51s

$ spot-check 8d366588-cbd6-43ba-95b4-0971c3f906e5
[{'package': 'endpoint', 'version': '^8.2.0 || ^9.0.0 || ^10.0.0'}]

Backport `9.2` (origin/9.2 + PR code)

$ pytest -q
222 passed, 19 skipped, 2 subtests passed in 82.76s (0:01:22)

$ pytest tests/test_integrations.py::TestFindCompatibleVersionRange -q
.......                                                                  [100%]
7 passed in 0.54s

$ spot-check 8d366588-cbd6-43ba-95b4-0971c3f906e5
[{'package': 'endpoint', 'version': '^8.2.0 || ^9.0.0 || ^10.0.0'}]

Backport `9.3` (origin/9.3 + PR code)

$ pytest -q
222 passed, 19 skipped, 2 subtests passed in 96.19s (0:01:36)

$ pytest tests/test_integrations.py::TestFindCompatibleVersionRange -q
.......                                                                  [100%]
7 passed in 0.52s

$ spot-check 8d366588-cbd6-43ba-95b4-0971c3f906e5
[{'package': 'endpoint', 'version': '^8.2.0 || ^9.0.0 || ^10.0.0'}]

Backport `9.4` (origin/9.4 + PR code)

$ pytest -q
222 passed, 19 skipped, 2 subtests passed in 106.83s (0:01:46)

$ pytest tests/test_integrations.py::TestFindCompatibleVersionRange -q
.......                                                                  [100%]
7 passed in 0.50s

$ spot-check 8d366588-cbd6-43ba-95b4-0971c3f906e5
[{'package': 'endpoint', 'version': '^8.2.0 || ^9.0.0 || ^10.0.0'}]

Checklist

Added a label for the type of pr: bug
Added the meta:rapid-merge label if planning to merge within 24 hours
Secret and sensitive material has been managed correctly
Automated testing was updated or added to match the most common scenarios
Documentation and comments were added for features that require explanation

…tions Replace find_least_compatible_version with find_compatible_version_range so prebuilt rules export the same related_integrations.version across stack backports. Bump version.lock for rules whose export changes.

github-actions · 2026-05-27T23:07:12Z

…tions Replace find_least_compatible_version with find_compatible_version_range so prebuilt rules export the same related_integrations.version across stack backports. Bump pyproject.toml patch version. Resolves #5601

Copilot

Pull request overview

This PR makes related_integrations.version exports stack-invariant by replacing single stack-dependent compatibility anchors with an OR’d compatibility range derived from integration manifests.

Changes:

Adds find_compatible_version_range and supporting helpers for manifest-derived compatibility ranges.
Updates rule export conversion to use the new range and union policy templates across selected anchors.
Replaces related integration tests and bumps the package version to 1.6.44.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
`detection_rules/integrations.py`	Adds the new compatibility range computation and removes the old least-compatible helper.
`detection_rules/rule.py`	Uses the new range API when populating `related_integrations`.
`tests/test_integrations.py`	Updates unit tests for the new range behavior.
`pyproject.toml`	Bumps the project version.

Iterate all majors overlapped by parsed Kibana bounds, derive legacy stack walk versions from manifest range floors instead of 8.19.0, remove a dead aligned-major branch, and drop RST-style double backticks in new docstrings.

…e-in-security_detection_engine-package-versions

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

Walk every stack major whose band intersects a bounded Kibana clause (e.g. >=8.12.0 <9.1.0 includes major 9) and pick the earliest compatible stack point within a major for the legacy least-compatible walk.

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

+        effective_stack_majors = sorted(
+            stack_major for stack_major in stack_majors if stack_major >= max(stack_majors) - 1
+        )


eric-forte-elastic · 2026-05-29T15:33:26Z

+        for lo, hi in _parse_kibana_range(version_requirement):
+            majors_to_check: list[int]
+            if hi is None:
+                majors_to_check = [lo.major]


For this if check:

fix(integrations): tighten stack-major overlap and anchor resolution
Walk every stack major whose band intersects a bounded Kibana clause
(e.g. >=8.12.0 <9.1.0 includes major 9) and pick the earliest compatible
stack point within a major for the legacy least-compatible walk.

I don't think any of our manifests include a situation where hi is None unless one were to have a manifest with an unbounded range (e.g. ">=8.12.0"). Fine to have this check, but given this I am not sure that this functions as intended.

For the case with an unbounded range majors_to_check only will have [8] and not any other stack majors. Not sure if this accomplishes the goal.

Whereas for the case in the note, it correctly has multiple stack versions to check.

eric-forte-elastic · 2026-05-29T15:50:02Z

Validated expected results from testing/debugging. 🟢 base flows function as designed.

Details

eric-forte-elastic · 2026-05-29T15:51:46Z

+    package: str,
+    packages_manifest: dict[str, Any],
+) -> CompatibleVersionRange:
+    """Return a stack-invariant OR'd caret range for related_integrations.version.


Nit. Should this doc string be updated to fit the existing format? Short one sentence doc string with more detail in separate comment?

eric-forte-elastic · 2026-05-29T15:54:07Z

+        second = find_compatible_version_range("pkg", manifests)
+        self.assertEqual(first, second)
+
+    def test_single_major_appends_forward_anchor(self):


Nit. Akin to https://github.com/elastic/detection-rules/pull/6208/changes#r3325352838, if the goal is to support unbounded upper ranges like ">=8.12.0" should we include a test for that? The hi is none branch?

eric-forte-elastic

One main comment and a few nits. Otherwise LGTM 👍

Generally this PR is low risk, as the updated version computation logic in effect is not used in other workflows (like schema validation, etc.). Additionally, Kibana validates this to be a NonEmpty string just like we do in our dataclasses. (ref), so we have considerable freedom in what we populate it with.

Mikaayenson added the bug Something isn't working label May 27, 2026

Mikaayenson linked an issue May 27, 2026 that may be closed by this pull request

[Bug] Improper prebuilt rule version usage in security_detection_engine package versions #5601

Open

Mikaayenson added the backport: auto label May 27, 2026

Mikaayenson self-assigned this May 28, 2026

Mikaayenson added the patch label May 28, 2026

fix(integrations): satisfy ruff SIM110 in _major_has_compatible_stack

73c0780

Mikaayenson requested a review from Copilot May 28, 2026 20:49

Copilot started reviewing on behalf of Mikaayenson May 28, 2026 20:50 View session

Copilot AI reviewed May 28, 2026

View reviewed changes

Comment thread detection_rules/integrations.py Outdated

Comment thread detection_rules/integrations.py

Mikaayenson and others added 2 commits May 28, 2026 16:02

Merge branch 'main' into 5601-bug-improper-prebuilt-rule-version-usag…

2ac6605

…e-in-security_detection_engine-package-versions

Mikaayenson marked this pull request as ready for review May 29, 2026 12:48

Mikaayenson requested a review from eric-forte-elastic as a code owner May 29, 2026 12:48

Mikaayenson requested a review from Copilot May 29, 2026 12:49

Copilot started reviewing on behalf of Mikaayenson May 29, 2026 12:49 View session

Copilot AI reviewed May 29, 2026

View reviewed changes

Comment thread detection_rules/integrations.py Outdated

Comment thread detection_rules/integrations.py Outdated

botelastic Bot added the python Internal python for the repository label May 29, 2026

Mikaayenson added 2 commits May 29, 2026 09:19

fix(integrations): tighten stack-major overlap and anchor resolution

1d9fae1

Walk every stack major whose band intersects a bounded Kibana clause (e.g. >=8.12.0 <9.1.0 includes major 9) and pick the earliest compatible stack point within a major for the legacy least-compatible walk.

fix(integrations): annotate majors_to_check for pyright

2302006

Mikaayenson requested a review from Copilot May 29, 2026 14:29

Copilot started reviewing on behalf of Mikaayenson May 29, 2026 14:30 View session

Copilot AI reviewed May 29, 2026

View reviewed changes

Comment thread detection_rules/integrations.py

Comment on lines +380 to +382

effective_stack_majors = sorted(

stack_major for stack_major in stack_majors if stack_major >= max(stack_majors) - 1

)

eric-forte-elastic reviewed May 29, 2026

View reviewed changes

eric-forte-elastic approved these changes May 29, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix stack-dependent related_integrations.version export#6208

Fix stack-dependent related_integrations.version export#6208
Mikaayenson wants to merge 7 commits into
mainfrom
5601-bug-improper-prebuilt-rule-version-usage-in-security_detection_engine-package-versions

Mikaayenson commented May 27, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 27, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

eric-forte-elastic May 29, 2026 •

edited

Loading

Uh oh!

eric-forte-elastic commented May 29, 2026 •

edited

Loading

Uh oh!

eric-forte-elastic May 29, 2026

Uh oh!

eric-forte-elastic May 29, 2026

Uh oh!

eric-forte-elastic left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Mikaayenson commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary - What I changed

How To Test

Checklist

Uh oh!

github-actions Bot commented May 27, 2026

Bug - Guidelines

Documentation and Context

Code Standards and Practices

Testing

Additional Checks

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

eric-forte-elastic May 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eric-forte-elastic commented May 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eric-forte-elastic May 29, 2026

Choose a reason for hiding this comment

Uh oh!

eric-forte-elastic May 29, 2026

Choose a reason for hiding this comment

Uh oh!

eric-forte-elastic left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Mikaayenson commented May 27, 2026 •

edited

Loading

eric-forte-elastic May 29, 2026 •

edited

Loading

eric-forte-elastic commented May 29, 2026 •

edited

Loading