Skip to content

[Labelling Health] Labelling Health Report – 2026-04-04 #35

@mnkiefer

Description

@mnkiefer

Summary

Status: Mixed (early-stage — insufficient baseline for trend judgment)

The labelling system became operational on 2026-03-31 and has run consistently since, but only ~4 days of history exist. Automation is working and the correction backlog is currently empty, but multiple summary issues are being created per run day and one run on March 31 shows anomalous change counts (likely a batch initialization), making trend comparison unreliable until a stable baseline forms.


Key Metrics

Metric Value Notes
Discussions reviewed (last 7 days) ~120–130 (estimated) 5 Label Discussions runs across 2 active days; deduplication needed (see detail below)
Label changes applied (last 7 days) ~52–173 (wide range) Inflated by a likely batch-init run (#25: 121 changes for 30 discussions); conservative estimate excludes it
Change rate (most recent canonical run) 50% (20 / 40) Run #27, 2026-04-03; prior run on 2026-03-31 was ~65% (26 / 40)
Label Discussions runs (last 7 days) 5 (all ✅ success) runs #21#23 on 2026-03-31; #26#27 on 2026-04-03
Labelling Correction Collector runs (last 7 days) 3 (all ✅ success) All on 2026-03-31
Labelling Correction Feedback runs (last 7 days) 9 (1 success, 8 skipped) Mostly skipped — no open signals to process
Open correction signals 0 Both signals from the first batch are closed
Signals created (last 7 days) 2 #27, #29 — both created 2026-03-31, closed 2026-04-03
Signals created (last 30 days) 2 Same two
Oldest open correction signal age N/A No open signals

⚠️ Precision caveat: The reviewed and changed fields did not parse from several summary issues (#25, #28, #33, #34) through the structured parser — values above were extracted from issue body text. Multiple summary issues per run day (3 on 2026-03-31, 2 on 2026-04-03) complicate aggregation; canonical per-day runs are used where identifiable.


Correction Pressure

Correction pressure is minimal at this stage — only 2 signals were generated in the entire observed window.

  • Discussion #98 (category: Other Feature Feedback, Questions & Ideas): received 2 correction events, with Actions as the latest label applied. This indicates the Actions label may have been applied in a context where the category or label mapping was ambiguous.
  • Discussion #118 (category: Enterprise): received 1 correction event, with bug as the latest label applied.

Neither signal is currently open. Pressure is concentrated in a single parent intake issue (Batch 01, #26), now closed. There is no multi-parent spread yet — the system is too new to show clustering.

No recurring label or category cluster is identifiable from two data points, but Actions in the Q&A category is worth monitoring in future batches.


Open Instruction Debt

The correction backlog is clean. Both signals from the first batch (Batch 01 / #26) were resolved within 3 days of creation. There are no open parent intake issues and no stale signals.

However, this does not yet reflect a stable operational state — the system was bootstrapped on 2026-03-31 and has not yet accumulated enough cycles to reveal recurring instruction gaps. The next 2–4 weeks will be the true indicator of whether instruction debt is accumulating or remaining low.


Recommendations

  1. Resolve the duplicate summary issue problem. Two summary issues were created on 2026-04-03 (#33 immediately closed, #34 open) and three on 2026-03-31 (#25, #28, #31). This makes metric aggregation unreliable. Investigate whether the Label Discussions workflow is triggering multiple times per schedule interval and, if so, add a deduplication or idempotency guard.

  2. Audit the 121-change batch-init run (#25). Issue #25 (2026-03-31) recorded 121 label changes across only 30 discussions — far above any other run. This is consistent with a bulk initialization of a previously-unlabeled backlog, but it should be confirmed. If it was a one-time batch-init, it should be tagged/excluded from ongoing rate baselines to avoid distorting future trend comparisons.

  3. Monitor the Actions label in Q&A-category discussions. Signal #29 (Discussion #98) received 2 correction events on the Actions label in the Other Feature Feedback, Questions & Ideas category. Review the relevant rules in .github/instructions/community-discussion-labeling.md to confirm the criteria for applying Actions are unambiguous, especially for discussions that mention GitHub Actions but are primarily support questions.

  4. Establish a stable weekly baseline before drawing trend conclusions. With only 4 days of data and a noisy first day, the current change rate (50%–65%) cannot yet be classified as improving or regressing. Allow 2–3 more weeks of consistent runs before interpreting directional movement.


Recent daily summary issue breakdown
Issue Date State Reviewed Changed Notes
#34 2026-04-03 open 40 20 Canonical April 3 run (run #27)
#33 2026-04-03 closed same day 40 20 Duplicate of #34 (run #26); identical counts
#31 2026-03-31 closed 2026-04-03 10 6 Partial run (run #23)
#28 2026-03-31 closed same day 40 26 Normal run (run #22); 14 skipped
#25 2026-03-31 closed same day 30 121 Anomalous run (run #21); likely batch init
Open correction signal breakdown

No open correction signals at time of report.

Closed signals (last 7 days):

Issue Category Latest Label Corrections Created Closed
#29 Other Feature Feedback, Questions & Ideas Actions 2 2026-03-31 2026-04-03
#27 Enterprise bug 1 2026-03-31 2026-04-03

Parent intake issues:

Issue State Created Closed
#26 — Batch 01 closed 2026-03-31 2026-04-03
Recent workflow run references
Workflow Run Date Result
Label Discussions §27 2026-04-03 ✅ success
Label Discussions §22 2026-03-31 ✅ success
Labelling Correction Collector §10 2026-03-31 ✅ success
Labelling Correction Feedback §17 2026-03-31 ✅ success
Label Discussions §21 2026-03-31 ✅ success

References

§27 · §10 · §17

Generated by Labelling Health Report · ● 438.1K ·

  • expires on May 4, 2026, 3:25 AM UTC

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions