Skip to content

CSPL-4513 Events for Index & Ingestion separation#1706

Open
kasiakoziol wants to merge 191 commits intofeature/slog-as-default-loggerfrom
feature/cspl-4513-events-for-index-ingest-sep
Open

CSPL-4513 Events for Index & Ingestion separation#1706
kasiakoziol wants to merge 191 commits intofeature/slog-as-default-loggerfrom
feature/cspl-4513-events-for-index-ingest-sep

Conversation

@kasiakoziol
Copy link
Collaborator

@kasiakoziol kasiakoziol commented Feb 18, 2026 •
edited
Loading

Description

It adds events and enhances logs for IngestorCluster and enhances events for IndexerCluster.

Key Changes

  • Refactoring existing event messages for IngestorCluster and IndexerCluster
  • Introducing new events for IngestorCluster: ScaledUp, ScaledDown, QueueConfigUpdated and
    IngestorsRestarted
  • Introducing new events for IndexerCluster: QueueConfigUpdated and
    IngestorsRestarted
  • Introducing slog
  • Automated tests for the above

Testing and Verification

Automated and manual tests

Related Issues

Jira: https://splunk.atlassian.net/browse/CSPL-4513

PR Checklist

  • Code changes adhere to the project's coding standards.
  • Relevant unit and integration tests are included.
  • Documentation has been updated accordingly.
  • All tests pass locally.
  • The PR description follows the project's guidelines.

kasiakoziol and others added 30 commits July 24, 2025 13:49
@kasiakoziol
CSPL-3551 Init IngestorCluster CR implementation
0e36568
@kasiakoziol
CSPL-3551 Enhancing Ingestor inputs
2cca0d7
@kasiakoziol
Merge branch 'develop' into CSPL-3551-ingestion-cr
f747ab4
@kasiakoziol
fix
9a7503c
@kasiakoziol
fix
7134350
@kasiakoziol
CSPL-3551 Adding tests
44349fa
@kasiakoziol
CSPL-3551 Mound defaults and update them with no Splunk restart
e71ea9c
@kasiakoziol
Merge branch 'develop' into CSPL-3551-ingestion-cr
14ccef4
@kasiakoziol
CSPL-3551 Fixing code after tests
8876d0e
@Igor-splunk @kasiakoziol
CSPL-3895-indexercluster (#1562)
5c31cab 
* test

* test

* test

* test

* Use endpoint to update conf file

* CSPL-3895 Matching changes from Ingestor to Indexer

---------

Co-authored-by: Kasia Koziol <kkoziol@splunk.com>
Co-authored-by: igor.grzankowski <@splunk.com>
@kasiakoziol
CSPL-3551 Applying fixes do Indexer integration
0c9e908
@kasiakoziol
Merge branch 'develop' into CSPL-3551-ingestion-cr
89f9eb4
@kasiakoziol
CSPL-3551 Fixes
d885a73
@kasiakoziol
Merge branch 'develop' into CSPL-3551-ingestion-cr
56aa780
@kasiakoziol
CSPL-3551 Fixes
1c77634
@kasiakoziol
CSPL-3560 Initial docs for I&I separation
fb4e87f
@kasiakoziol
Merge branch 'develop' into CSPL-3551-ingestion-cr
3e376d8
@kasiakoziol
CSPL-3551 Update documentation to reflect on Grafana
e62f381
@kasiakoziol
CSPL-3556 Unit tests
84d8491
@kasiakoziol
CSPL-3972 Addressing TODOs
c3b3aa6
@kasiakoziol
CSPL-3972 Addressing TODOs
2af9061
@kasiakoziol
CSPL-3551 Moving default from types to controller code
053352d
@kasiakoziol
CSPL-3551 Applying previous changes to IndexerCluster
e844286
@kasiakoziol
CSPL-3551 Moving validations to separate function and adding validati…
500cdfa 
…on for empty values
@kasiakoziol
CSPL-3551 Making sure all inputs are put into status
4b064a6
@kasiakoziol
CSPL-3551 Update of conf files when queue name or type change
9c6cd69
@kasiakoziol
CSPL-3551 Unit tests
759b09b
@kasiakoziol
CSPL-4003 Helm charts support for bus and pipeline configs
20d9d6c
@kasiakoziol
CSPL-4003 Docs update
f9fb45a
@kasiakoziol
CSPL-4003 Addressing comments
c5e4c2b
@kasiakoziol
CSPL-4002 Update Ansible role to ingestor (from standalone) for Inges…
7bef9e3 
…torCluster (#1606)

* CSPL-4002 Update role for ingestor to ingestor from standalone

* CSPL-4212 Address changes from unit tests from develop branch

* Addressing comments

* Run tests against Splunk 10.2

* Fix

* Address changes from develop

* CSPL-4002 Revert 10.2 changes
@kasiakoziol
Merge branch 'CSPL-3551-ingestion-cr' into feature/cspl-4513-events-f…
7b5154f 
…or-index-ingest-sep
@kasiakoziol @Igor-splunk
CSPL-3549 Splunk Operator Enhancement – Ingestion and Indexing Separa…
8f41f63 
…tion (#1550)

* CSPL-3551 Init IngestorCluster CR implementation

* CSPL-3551 Enhancing Ingestor inputs

* fix

* fix

* CSPL-3551 Adding tests

* CSPL-3551 Mound defaults and update them with no Splunk restart

* CSPL-3551 Fixing code after tests

* CSPL-3895-indexercluster (#1562)

* test

* test

* test

* test

* Use endpoint to update conf file

* CSPL-3895 Matching changes from Ingestor to Indexer

---------

Co-authored-by: Kasia Koziol <kkoziol@splunk.com>
Co-authored-by: igor.grzankowski <@splunk.com>

* CSPL-3551 Applying fixes do Indexer integration

* CSPL-3551 Fixes

* CSPL-3551 Fixes

* CSPL-3560 Initial docs for I&I separation

* CSPL-3551 Update documentation to reflect on Grafana

* CSPL-3556 Unit tests

* CSPL-3972 Addressing TODOs

* CSPL-3972 Addressing TODOs

* CSPL-3551 Moving default from types to controller code

* CSPL-3551 Applying previous changes to IndexerCluster

* CSPL-3551 Moving validations to separate function and adding validation for empty values

* CSPL-3551 Making sure all inputs are put into status

* CSPL-3551 Update of conf files when queue name or type change

* CSPL-3551 Unit tests

* CSPL-4003 Helm charts support for bus and pipeline configs

* CSPL-4003 Docs update

* CSPL-4003 Addressing comments

* CSPL-3558 Integ tests init

* CSPL-3558 Adding extra validation for integraion test

* CSPL-3558 Refactoring

* CSPL-3558 Adding scenario for update and delete

* CSPL-3558 Adding helm test

* CSPL-3558 Update of k8s version

* CSPL-3558 Fix metrics-server installation issues after k8s version upgrade

* CSPL-3558 Adding additional tests

* CSPL-3558 Fixing helm tests

* CSPL-3558 Fix tests after merge

* CSPL-3558 Fix helm tests

* CSPL-4022 Remove pipeline config from inputs

* CSPL-4022 Remove bus inputs

* CSPL-4022 Introduce BusConfiguration CR

* CSPL-4022 Update docs and tests

* CSPL-4022 Update ns reference for BusConfiguration

* CSPL-4022 Fixing tests and adding bus config to ingestor controller

* CSPL-4022 Fix update behaviour

* CSPL-4022 Docs update

* CSPL-4022 Fix failing tests

* CSPL-4022 Fix tests

* CSPL-4022 Addressing PR comments

* CSPL-4022 Address comments

* CSPL-4022 Fix helm tests

* CSPL-4358 Splitting BusConfiguration into Bus and LargeMessageStore

* CSPL-4358 Update docs

* CSPL-4358 Addressing comments

* CSPL-4358 Adding more validations

* CSPL-4360 Secret reference added for Bus CR

* CSPL-4360 Fix failing tests

* CSPL-4360 Add Splunk restart

* CSPL-4360 Fix failing tests

* CSPL-4360 Fix failing tests

* CSPL-4360 Fix errors with failing validation on status

* CSPL-4358 Rename Bus to Queue

* CSPL-4358 Rename LargeMessageStore to ObjectStorage

* CSPL-4358 Making region authRegion and optional, simplifying endpoint

* CSPL-4360 Fixing tests after merge

* CSPL-4360 Fix validation that fails for status

* CSPL-4360 Fix failing to get k8s secret

* CSPL-4360 Fix failing integ and helm tests

* CSPL-4360 Fixing failing tests due to incorrect secret ref

* CSPL-4360 Addressing comments

* CSPL-4360 Addressing secret value change and removing redundant controllers

* CSPL-4360 Update of docs, helm tests and validations

* CSPL-4360 Add secret watch and fix controller tests

* CSPL-4360 Update docs

* CSPL-4360 Restart Splunk when SA changed

* Addressing comments

* Addressing changes from recent PRs

* Formatting fix

* Addressing PR comments

* Address comments

* Addressing comments

* Fix pipeline issues

* Reverting

* Refactoring according to comments

* Fmt fix

* CSPL-4002 Update Ansible role to ingestor (from standalone) for IngestorCluster (#1606)

* CSPL-4002 Update role for ingestor to ingestor from standalone

* CSPL-4212 Address changes from unit tests from develop branch

* Addressing comments

* Run tests against Splunk 10.2

* Fix

* Address changes from develop

* CSPL-4002 Revert 10.2 changes

---------

Co-authored-by: Igor Grzankowski <igorg@splunk.com>
Base automatically changed from CSPL-3551-ingestion-cr to develop February 20, 2026 15:29
vivekr-splunk
vivekr-splunk reviewed Feb 23, 2026
View reviewed changes
vivekr-splunk
vivekr-splunk reviewed Feb 23, 2026
View reviewed changes
vivekr-splunk
vivekr-splunk reviewed Feb 23, 2026
View reviewed changes
pkg/splunk/enterprise/indexercluster.go Outdated
Copy link
Collaborator

@vivekr-splunk vivekr-splunk Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we pass new parameter here for logger ?

pkg/splunk/enterprise/indexercluster.go
return result, err
}

eventPublisher.Normal(ctx, "QueueConfigUpdated",
Copy link
Collaborator

@vivekr-splunk vivekr-splunk Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

queue configured message is updated before restart

Copy link
Collaborator Author

@kasiakoziol kasiakoziol Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it is updated in the .conf file before the restart. Restart just makes it work.

@kasiakoziol kasiakoziol force-pushed the feature/cspl-4513-events-for-index-ingest-sep branch from d540e31 to 14d99b5 Compare February 23, 2026 15:00
kasiakoziol and others added 10 commits February 24, 2026 11:55
@kasiakoziol
Add sqs_smartubus_cp to providers (#1716)
528ac31 
* Add sqs_smartubus_cp to providers

* Addressing comments
@kasiakoziol
Merge branch 'develop' into feature/cspl-4513-events-for-index-ingest…
2fa9090 
…-sep
@vivekr-splunk
Fix bundle target after Helm CRD removal
e4cd5b7
@vivekr-splunk
ci: make Coveralls upload non-blocking during outages
d9eaecb
@vivekr-splunk
adding parallel work for formatting and unit test
2e65b5d
@gabrielm-splunk
Merge pull request #1722 from splunk/fix/remove-stale-crd-copy-step
76f6658 
Fix pre-release bundle target after Helm CRD removal
@mmarod @gabrielm-splunk
Fix app_tel_for_sok app.conf install stanza header (#1724)
7fa8698 
* Fixes the app.conf within the app_tel_for_sok app which has -e [install] on its first line instead of just [install]

* skipping code coverage for PRs from forks (as the secrets for COVERALLS_TOKEN is not available in these workflow runs for GitHub security reasons)

* adding similar fix due to secret for FOSSA_API_TOKEN not being available

---------

Co-authored-by: Gabriel J Mendoza <gabrielm@splunk.com>
@kasiakoziol
Merge branch 'develop' into feature/cspl-4513-events-for-index-ingest…
cb7d7f7 
…-sep
@kasiakoziol
Merge branch 'feature/slog-as-default-logger' into feature/cspl-4513-…
f807ffc 
…events-for-index-ingest-sep
@kasiakoziol
CSPL-4513 Addressing Igor's changes
10d7853
@kasiakoziol kasiakoziol changed the base branch from develop to feature/slog-as-default-logger February 26, 2026 07:07
@github-actions
Copy link
Contributor

github-actions bot commented Feb 26, 2026

CLA Assistant Lite bot: All contributors have NOT signed the COC Document

I have read the Code of Conduct and I hereby accept the Terms

5 out of 6 committers have signed the CLA.
✅ @kasiakoziol
✅ @Igor-splunk
✅ @gabrielm-splunk
✅ @kubabuczak
✅ @minjieqiu
❌ @Nguyen Duc Trung
NGUYEN Duc Trung seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You can retrigger this bot by commenting recheck in this Pull Request

@github-actions
Copy link
Contributor

github-actions bot commented Feb 26, 2026

CLA Assistant Lite bot:
Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.

I have read the CLA Document and I hereby sign the CLA

5 out of 6 committers have signed the CLA.
✅ @kasiakoziol
✅ @Igor-splunk
✅ @gabrielm-splunk
✅ @kubabuczak
✅ @minjieqiu
❌ @Nguyen Duc Trung
NGUYEN Duc Trung seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You can retrigger this bot by commenting recheck in this Pull Request

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rlieberman-splunk rlieberman-splunk rlieberman-splunk approved these changes

@Igor-splunk Igor-splunk Igor-splunk approved these changes

@patrykw-splunk patrykw-splunk Awaiting requested review from patrykw-splunk

@kubabuczak kubabuczak Awaiting requested review from kubabuczak

@gabrielm-splunk gabrielm-splunk Awaiting requested review from gabrielm-splunk

@minjieqiu minjieqiu Awaiting requested review from minjieqiu

@qingw-splunk qingw-splunk Awaiting requested review from qingw-splunk

@vivekr-splunk vivekr-splunk Awaiting requested review from vivekr-splunk

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

10 participants

@kasiakoziol @coveralls @vivekr-splunk @rlieberman-splunk @Igor-splunk @gabrielm-splunk @kubabuczak @minjieqiu @patrykw-splunk @mmarod