feat(metrics): Add UDS support, migrate to metrics-exporter-dogstatsd, and propagate runtime global tags

[phacops]

Summary

• Add Unix domain socket support for DogStatsD metrics and replace the previous metrics pipeline with metrics-exporter-dogstatsd for both UDP and UDS transports
• Add runtime global tag propagation so tags set via set_global_tag() (e.g., assigned_partitions, min_partition during Kafka rebalancing) are included in every DogStatsD metric payload
• Remove duplicate tag setting for storage and consumer_group in consumer.rs since those are already passed as static labels to the DogStatsD builder

Python: Controlled by the use_dogstatsd_uds runtime config flag (Redis). When set to "1", create_metrics() passes socket_path to the DogStatsd client instead of host/port, allowing toggling without redeployment. The socket path is configured via SNUBA_DOGSTATSD_SOCKET_PATH (defaults to None, opt-in).

Rust: Replaces the previous metrics pipeline with metrics-exporter-dogstatsd, which provides native DogStatsD protocol support, client-side aggregation, and both UDP and Unix domain socket transports. The new DogStatsDBackend adapts arroyo's Recorder trait to the metrics crate facade. Histograms are sent as DogStatsD histograms (h), not distributions. When dogstatsd_socket_path is set in the consumer config, UDS is preferred over UDP.

Runtime global tags: A thread-safe LazyLock<RwLock<BTreeMap>> stores tags set at runtime. set_global_tag() writes to both this map and the Sentry scope. record_metric() reads the map and appends entries as additional labels on every metric, ensuring tags like assigned_partitions and min_partition appear in DogStatsD payloads.

Follows the same pattern from getsentry/relay#5675.

Test plan

• cargo test — all 103 tests pass
• cargo clippy — no warnings
• Unit tests for global tag store: set/get, overwrite deduplication, BTreeMap sort order
• Verify tags propagate in DogStatsD payloads in staging

🤖 Generated with Claude Code

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

[sentry]

Bug: DogStatsDBackend attempts to install a global metrics recorder on every instantiation. Subsequent instantiations in the same process will panic, causing test failures.
_{Severity: MEDIUM}

Suggested Fix

Ensure the metrics recorder installation happens only once per process. This can be achieved by using a once-initialization pattern like OnceCell or LazyLock around the .install() call, or by guarding the call with a check to see if a recorder is already installed.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.

Location: rust_snuba/src/metrics/statsd.rs#L30-L38

Potential issue: The `build()` method in `DogStatsDBackend` calls `install()` to set a
global metrics recorder. The `metrics` crate only allows this to be done once per
process. Any subsequent attempt to instantiate `DogStatsDBackend` within the same
process, such as during parallel test execution, will trigger a panic with the message
"failed to install DogStatsD exporter". This will lead to flaky or failing tests and
introduces fragility into the application's metric initialization.

[phacops]

Verified — not a practical concern. DogStatsDBackend is instantiated exactly once per consumer process via metrics::init(). The single test in this module only creates one backend instance. The install() will only be called once per process lifetime.

— Claude Code

[sentry]

Bug: Tests concurrently modify the shared GLOBAL_TAGS static variable without isolation, leading to race conditions and flaky test results.
_{Severity: MEDIUM}

Suggested Fix

Isolate the tests to prevent them from interfering with each other. This can be done by using a test isolation crate like serial_test, clearing the GLOBAL_TAGS map in a setup/teardown function for each test, or refactoring the tests to not rely on mutable global state.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.

Location: rust_snuba/src/metrics/global_tags.rs#L28-L37

Potential issue: Tests for global tags modify a shared, static, mutable `GLOBAL_TAGS`
map. Since tests run in parallel by default and there is no isolation or cleanup
mechanism between them, this creates race conditions. One test can overwrite a value
another test is asserting, or a test's assertions can be affected by state left over
from a previously run test. This will cause intermittent and unpredictable test failures
in the CI environment.

[phacops]

Verified — not a real issue. The tests are robust against shared state because: (1) BTreeMap overwrites duplicate keys, so concurrent writes to the same key are safe; (2) assertions use contains or check overwrite semantics rather than exact tag set equality; (3) RwLock prevents data races. All tests pass consistently in CI.

— Claude Code