Skip to content

Add fine-grained stream error logging control#2805

Merged
He-Pin merged 1 commit intoapache:mainfrom
He-Pin:port-stream-error-logging
Mar 30, 2026
Merged

Add fine-grained stream error logging control#2805
He-Pin merged 1 commit intoapache:mainfrom
He-Pin:port-stream-error-logging

Conversation

@He-Pin
Copy link
Copy Markdown
Member

@He-Pin He-Pin commented Mar 28, 2026

Motivation

Stream error logging was all-or-nothing with no way to control logging for specific stream stages.

Modification

Added configuration support for fine-grained control of stream error logging, allowing users to suppress expected errors while still logging unexpected ones.

Result

Users can configure stream error logging per-stage, reducing log noise from expected transient failures.

References

@He-Pin He-Pin marked this pull request as ready for review March 28, 2026 07:12
@He-Pin He-Pin added this to the 2.0.0-M2 milestone Mar 28, 2026
He-Pin added a commit to He-Pin/incubator-pekko that referenced this pull request Mar 28, 2026
Adds configurable throttling for stream stage error log messages to
prevent log spam from rapidly failing stages. Controlled via:

  pekko.stream.materializer.stage-errors-log-throttle-period = off

When enabled (e.g., = 1s), only the first error within each period
is logged. Suppressed errors are counted and reported when the
period expires.

This complements the fine-grained log level control added in apache#2805.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@He-Pin He-Pin force-pushed the port-stream-error-logging branch from 4ad8c8a to e397b16 Compare March 28, 2026 09:11
@He-Pin He-Pin requested a review from pjfanning March 28, 2026 09:50
Add configurable stream stage error log level via
'pekko.stream.materializer.stage-errors-default-log-level' config key.
Supports 'error' (default), 'warning', 'info', 'debug', or 'off'.

When a stage-specific LogLevels attribute is present, it takes
precedence. Otherwise the system-wide default is used, enabling
operators to reduce noise from expected stream errors without
per-stage configuration.

Changes:
- Add stage-errors-default-log-level to reference.conf
- Add LogLevels.defaultErrorLevel and LogLevels.fromString helpers
- Update GraphInterpreter.reportStageError to use per-level logging
- Update RestartFlow.loggingEnabled to respect system-wide setting
- Change NoMaterializer from object to case class accepting ActorSystem
  (required because GraphInterpreter now accesses materializer.system)

Upstream: akka/akka-core@519d33d897
Cherry-picked from akka/akka-core v2.8.0, which is now Apache licensed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@He-Pin He-Pin force-pushed the port-stream-error-logging branch from e397b16 to 8d74ef4 Compare March 28, 2026 12:47
Copy link
Copy Markdown
Member

@pjfanning pjfanning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@He-Pin He-Pin merged commit d548313 into apache:main Mar 30, 2026
9 checks passed
@He-Pin He-Pin deleted the port-stream-error-logging branch March 30, 2026 12:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants