feat: implement prefix-based metadata filtering for StatefulSet and P…

[ductrung-nguyen]

Description

This PR implements prefix-based metadata filtering to give users control over how labels and annotations propagate from the Custom Resource (CR) to the underlying StatefulSet and Pods.
Note: This is a part of #1644.
It contains the first commit + a bug fix which we discovered after a long validation.

Previously, almost all metadata was copied everywhere. Now you can use specific prefixes to target where you want your labels to go:

• sts-only.*: Keys starting with this are added only to the StatefulSet metadata (and the prefix is stripped!). They won't appear on the Pod Template, so adding them won't trigger a pod rollout.
• pod-only.*: Keys are added only to the Pod Template (and excluded from the StatefulSet metadata).
• System prefixes: We now automatically exclude kubectl.kubernetes.io/* and operator.splunk.com/* prefixes to prevent clutter and conflicts.

Regular labels without these prefixes work exactly as before—propagatating to both.

Key Changes

• Added the core filtering logic. It checks for sts-only. and pod-only. prefixes.
• Implemented SyncParentMetaToStatefulSet which handles the transformation (stripping sts-only.) and syncing logic.
• Updated the main reconciliation loop to use these new syncing functions instead of the old blind copy.
• Added logic to detect if only StatefulSet metadata changed (so we can update the STS object without touching the Pod specs).

Testing and Verification

Many unit tests are implemented.

For E2E validation:

1. Update the existing Splunk CR.
2. Add a label sts-only.my-tag: true.
   • Verifed: The StatefulSet has my-tag: true.
   • Verifed: The Pods do not have my-tag (and thus no rollout if added later).
3. Add a label pod-only.scrapper: true.
   • Verified: The Pods have pod-only.scrapper: true.
   • Verified: The StatefulSet does not have this label.
4. Add a generic label environment: prod.
   • Verified: Both STS and Pods have environment: prod.

Related Issues

#1652

PR Checklist

• Code changes adhere to the project's coding standards.
• Relevant unit and integration tests are included.
• Documentation has been updated accordingly.
• All tests pass locally.
• The PR description follows the project's guidelines.

[github-actions]

CLA Assistant Lite bot CLA Assistant Lite bot All contributors have signed the COC ✍️ ✅

[ductrung-nguyen]

I have read the CLA Document and I hereby sign the CLA

[ductrung-nguyen]

I have read the Code of Conduct and I hereby accept the Terms

[Copilot]

Pull request overview

This pull request implements prefix-based metadata filtering to give users control over how labels and annotations propagate from Custom Resources (CR) to StatefulSets and Pods, addressing issue #1652 where CR metadata changes triggered unnecessary pod rolling restarts.

Purpose: The PR solves a significant operational problem where adding simple organizational labels (e.g., team: platform) to a CR would cause disruptive rolling restarts of all pods. The new filtering mechanism allows users to selectively control metadata propagation.

Changes:

• Implements sts-only.* and pod-only.* prefix-based filtering with automatic prefix stripping during propagation
• Adds full sync semantics for metadata (including removal of keys no longer in CR) via new SyncParentMetaToPodTemplate and SyncParentMetaToStatefulSet functions
• Fixes a critical memory-sharing bug where StatefulSet and Pod Template label maps shared the same underlying memory reference
• Adds metadata-only change detection in ApplyStatefulSet via MergeStatefulSetMetaUpdates to avoid unnecessary pod updates

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
pkg/splunk/common/util.go	Core filtering logic with prefix constants, exclusion rules, managed key tracking (Get/Set functions), and sync functions for both Pod Template and StatefulSet
pkg/splunk/common/util_test.go	Comprehensive unit tests (1800+ lines) covering all filtering scenarios, prefix stripping, managed key tracking, and sync behavior
pkg/splunk/common/statefulset_util.go	New MergeStatefulSetMetaUpdates function to detect StatefulSet metadata-only changes without triggering pod restarts
pkg/splunk/common/statefulset_util_test.go	Unit tests for metadata merge logic covering various edge cases including nil handling
pkg/splunk/enterprise/configuration.go	Creates separate label maps for StatefulSet and Pod Template (bug fix), replaces AppendParentMeta with new SyncParentMeta* functions
pkg/splunk/enterprise/configuration_test.go	Test verifying label maps don't share memory (regression test for bug fix)
pkg/splunk/splkcontroller/statefulset.go	Integrates MergeStatefulSetMetaUpdates into ApplyStatefulSet to detect metadata-only changes
pkg/splunk/splkcontroller/statefulset_metadata_test.go	Tests for annotation persistence and metadata-only change detection
pkg/splunk/test/controller.go	Adds DeepCopy() calls to prevent annotation pollution between test cases
pkg/splunk/enterprise/testdata/fixtures/*.json	Updates test fixtures with new managed-cr tracking annotations

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[ductrung-nguyen]

Hi @vivekr-splunk
I extracted the first commit in the "big" PR to open another smaller PR here.
I also include a small change to fix a bug which we discover using our long-run validation (so it is slightly different from the original commit).