Skip to content

pkg/beholder: add batch emitter service with service-engine lifecycle#2059

Draft
pkcll wants to merge 2 commits into
mainfrom
infoplat-3436-chipingress-batching-part-2
Draft

pkg/beholder: add batch emitter service with service-engine lifecycle#2059
pkcll wants to merge 2 commits into
mainfrom
infoplat-3436-chipingress-batching-part-2

Conversation

@pkcll
Copy link
Copy Markdown
Contributor

@pkcll pkcll commented May 14, 2026

Summary

  • Introduce ChipIngressBatchEmitterService backed by chipingress batch client, managed as a sub-service of the beholder Client via services.Engine
  • Refactor DualSourceEmitter to delegate fire-and-forget to ChipIngressEmitter
  • Pass explicit loggers throughout instead of creating new ones internally
  • Add batch emitter config fields, feature flag (ChipIngressBatchEmitterEnabled), tests, and benchmarks
  • Update pkg/loop server and config to propagate batch emitter settings
  • Use test loggers in all beholder tests so service-engine diagnostics are visible

Dependencies

Depends on #2058 (pkg/chipingress batch client metrics). After #2058 merges, chipingress dependency will be bumped from main.

Split from #1862 (part 2 of 2 — beholder + loop changes).

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 14, 2026

⚠️ API Diff Results - github.com/smartcontractkit/chainlink-common

⚠️ Breaking Changes (3)

pkg/beholder (3)
  • NewChipIngressEmitter — Type changed:
func(
  github.com/smartcontractkit/chainlink-common/pkg/chipingress.Client, 
  + github.com/smartcontractkit/chainlink-common/pkg/logger.Logger
)
(Emitter, error)
  • NewDualSourceEmitter — Type changed:
func(
  Emitter, 
  Emitter, 
  + github.com/smartcontractkit/chainlink-common/pkg/logger.Logger
)
(Emitter, error)
  • NewNoopClient — Type changed:
func(
  - 
  + ...github.com/smartcontractkit/chainlink-common/pkg/logger.Logger
)
*Client

✅ Compatible Changes (25)

pkg/beholder (2)
  • ChipIngressBatchEmitterService — ➕ Added

  • NewChipIngressBatchEmitterService — ➕ Added

pkg/beholder.BeholderClient (1)
  • Service — ➕ Added
pkg/beholder.Client (1)
  • Service — ➕ Added
pkg/beholder.Config (8)
  • ChipIngressBatchEmitterEnabled — ➕ Added

  • ChipIngressBufferSize — ➕ Added

  • ChipIngressDrainTimeout — ➕ Added

  • ChipIngressLogger — ➕ Added

  • ChipIngressMaxBatchSize — ➕ Added

  • ChipIngressMaxConcurrentSends — ➕ Added

  • ChipIngressSendInterval — ➕ Added

  • ChipIngressSendTimeout — ➕ Added

pkg/beholder.writerClientConfig (8)
  • ChipIngressBatchEmitterEnabled — ➕ Added

  • ChipIngressBufferSize — ➕ Added

  • ChipIngressDrainTimeout — ➕ Added

  • ChipIngressLogger — ➕ Added

  • ChipIngressMaxBatchSize — ➕ Added

  • ChipIngressMaxConcurrentSends — ➕ Added

  • ChipIngressSendInterval — ➕ Added

  • ChipIngressSendTimeout — ➕ Added

pkg/loop.EnvConfig (1)
  • ChipIngressBatchEmitterEnabled — ➕ Added
pkg/services.HealthReporter (3)
  • HealthReport — ➕ Added

  • Name — ➕ Added

  • Ready — ➕ Added

pkg/services.Service (1)
  • Start — ➕ Added

📄 View full apidiff report

@pkcll pkcll force-pushed the infoplat-3436-chipingress-batching-part-2 branch from 730b714 to fa7e617 Compare May 14, 2026 17:41
@pkcll pkcll marked this pull request as ready for review May 14, 2026 17:49
@pkcll pkcll requested a review from a team as a code owner May 14, 2026 17:49
Copilot AI review requested due to automatic review settings May 14, 2026 17:49
@pkcll pkcll requested a review from a team as a code owner May 14, 2026 17:49
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds lifecycle-managed Chip Ingress batch emission to Beholder and wires it into LOOP configuration/startup, while refactoring emitter ownership so the Beholder client manages service start/stop.

Changes:

  • Adds ChipIngressBatchEmitterService with batching, lifecycle integration, metrics, tests, and benchmarks.
  • Refactors beholder.Client, noop/writer/http clients, and emitters to use services.Engine lifecycle and injected loggers.
  • Adds LOOP env/config propagation for the batch emitter feature flag and starts/stops the Beholder client from the server.

Reviewed changes

Copilot reviewed 21 out of 22 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
pkg/services/service.go Removes obsolete lint suppression in service startup tracing.
pkg/loop/server.go Starts/stops the lifecycle-managed Beholder client and passes batch-emitter config/logger.
pkg/loop/config.go Adds LOOP batch-emitter env flag parsing and command env propagation.
pkg/loop/config_test.go Updates config parsing/export tests for the new flag.
pkg/loop/plugin_relayer_emitter_test.go Updates noop Beholder client construction with test logger.
pkg/beholder/config.go Adds batch-emitter configuration defaults and clarifies emitter batching docs.
pkg/beholder/config_test.go Updates config example output for new fields.
pkg/beholder/client.go Embeds service lifecycle, creates optional batch sub-service, and refactors close behavior.
pkg/beholder/client_test.go Adds lifecycle coverage for client close and batch sub-service behavior.
pkg/beholder/batch_emitter_service.go Adds the new Chip Ingress batch emitter service implementation.
pkg/beholder/batch_emitter_service_test.go Adds tests and benchmark coverage for batch emitting, callbacks, defaults, and metrics.
pkg/beholder/chip_ingress_emitter.go Moves legacy fire-and-forget behavior into the Chip Ingress emitter.
pkg/beholder/chip_ingress_emitter_test.go Updates legacy emitter tests for async publish and injected logger.
pkg/beholder/dual_source_emitter.go Simplifies dual-source emission to delegate Chip Ingress async behavior.
pkg/beholder/dual_source_emitter_test.go Updates constructor usage and blocking behavior coverage.
pkg/beholder/noop.go Initializes noop/writer clients with service lifecycle and optional logger.
pkg/beholder/noop_test.go Updates noop client tests for lifecycle startup.
pkg/beholder/httpclient.go Initializes HTTP client with service lifecycle and avoids duplicate provider shutdown.
pkg/beholder/global_test.go Updates global Beholder tests for lifecycle-aware clients.
pkg/beholder/beholdertest/beholder.go Uses lifecycle-aware noop client in test observer setup.
go.mod Bumps pkg/chipingress dependency for batch client support.
go.sum Updates checksums for the bumped pkg/chipingress dependency.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pkg/beholder/client.go
// teardown after parent close hook completes.
chipIngressEmitter = noCloseEmitter{Emitter: batchEmitterService}
} else {
chipIngressEmitter, err = NewChipIngressEmitter(chipIngressClient, cfg.ChipIngressLogger)
Comment thread pkg/loop/server.go
Comment on lines +352 to +363
s.beholderClient = beholderClient
beholder.SetClient(beholderClient)
beholder.SetGlobalOtelProviders()

if beholderCfg.LogStreamingEnabled {
otelLogger, err := NewOtelLogger(beholderClient.Logger, beholderCfg.LogLevel)
if err != nil {
return fmt.Errorf("failed to enable log streaming: %w", err)
}
s.Logger = logger.Sugared(logger.Named(otelLogger, s.Logger.Name()))
}

}

func (e *ChipIngressBatchEmitterService) start(ctx context.Context) error {
e.batchClient.Start(ctx)
@pkcll pkcll force-pushed the infoplat-3436-chipingress-batching-part-2 branch from e8165ba to d3e2a0e Compare May 14, 2026 21:55
@pkcll pkcll marked this pull request as draft May 14, 2026 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants