Skip to content

Missing alert's tenant validation #9

@bsalsingh

Description

@bsalsingh

Summary

The message gateway does not validate the tenant field in incoming alerts against registered tenant/team records, causing alerts to be routed to incorrect L1 agents based solely on the tenant value in the alert payload.


Steps to Reproduce

  1. External monitoring system sends an alert to the incoming-alerts Kafka topic with payload:
{
  "status": "firing",
  "tenants": "tenant-foo-bar",
  "alerts": [{
    "alertname": "PodCrashLoopBackOff",
    "namespace": "workloads",
    "pod": "crashloop-demo-679f49fb9b-f8lvj",
    "severity": "critical"
  }]
}
  1. streams-router (k8s-agent/streams-router/router.py:59) extracts tenant="tenant-foo-bar" from payload.get("tenants")
  2. streams-router routes alert to the tenant-foo-bar Kafka topic without validating the tenant belongs to the submitting cluster
  3. L1 agent subscribed to tenant-foo-bar receives and processes the alert

Expected Behavior

The message gateway/streams-router should validate that the tenant field in the alert payload matches a registered tenant for the originating team/cluster before routing. If validation fails, the alert should be rejected and logged.


Actual Behavior

No tenant-to-team validation is performed. The streams-router trusts the tenant field in the alert payload unconditionally and routes to the corresponding topic. This allows:

  • Alerts with spoofed tenant IDs to be routed to wrong L1 agents
  • Cross-tenant alert forwarding when tenant in payload doesn't match the submitting team's registered tenants

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions