Skip to content

Coalesce function unsubscribes only after receiving 3 samples#1374

Open
simonvoelcker wants to merge 3 commits intofrequenz-floss:v1.x.xfrom
simonvoelcker:coalesce_cautious_unsubscribe
Open

Coalesce function unsubscribes only after receiving 3 samples#1374
simonvoelcker wants to merge 3 commits intofrequenz-floss:v1.x.xfrom
simonvoelcker:coalesce_cautious_unsubscribe

Conversation

@simonvoelcker
Copy link
Contributor

@simonvoelcker simonvoelcker commented Mar 13, 2026

Addresses #1332.

Background

The Coalesce function subscribes and unsubscribes from components based on data availability. If it receives None-values, it will subscribe to the next component, if any, from its parameter list. Once the other components start sending data again, we may unsubscribe from "later" components. We currently unsubscribe from all components "after" a given one once that given one sends us one non-None sample. This is undesired because we may receive intermittent None values for various reasons and would then unsubscribe and resubscribe excessively.

Change

The solution is to count non-None samples from earlier streams and unsubscribe only once a threshold (3) is reached.

Discussion

If there are only two components (main, backup), it is quite straightforward. With more components, although probably rarely seen in practice, one could come up with a more sophisticated solution that tracks each components samples and marks them as stable when 3 samples are reached. Happy to discuss, but I think it's overkill.

@github-actions github-actions bot added part:tests Affects the unit, integration and performance (benchmarks) tests part:data-pipeline Affects the data pipeline labels Mar 13, 2026
@simonvoelcker simonvoelcker force-pushed the coalesce_cautious_unsubscribe branch 2 times, most recently from dae4a2a to c69f4a4 Compare March 13, 2026 14:24
@github-actions github-actions bot added the part:docs Affects the documentation label Mar 13, 2026
@simonvoelcker simonvoelcker force-pushed the coalesce_cautious_unsubscribe branch from c69f4a4 to 89eb72f Compare March 13, 2026 14:26
Signed-off-by: Simon Völcker <simon.voelcker@frequenz.com>
Signed-off-by: Simon Völcker <simon.voelcker@frequenz.com>
Signed-off-by: Simon Völcker <simon.voelcker@frequenz.com>
@simonvoelcker simonvoelcker force-pushed the coalesce_cautious_unsubscribe branch from 89eb72f to 6a483f0 Compare March 13, 2026 14:32
@simonvoelcker simonvoelcker marked this pull request as ready for review March 13, 2026 14:52
@simonvoelcker simonvoelcker requested a review from a team as a code owner March 13, 2026 14:52
@simonvoelcker simonvoelcker requested review from shsms and removed request for a team March 13, 2026 14:52
@@ -0,0 +1,186 @@
# License: MIT
# Copyright © 2025 Frequenz Energy-as-a-Service GmbH
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

part:data-pipeline Affects the data pipeline part:docs Affects the documentation part:tests Affects the unit, integration and performance (benchmarks) tests

Projects

Status: To do

Development

Successfully merging this pull request may close these issues.

1 participant