Envoy Gateway - configure telemetry tracing sampler?

[jan-mrm]

Hey there,
I am looking to configure the tracing sampler in the envoy gateway. Is that possible at all, and if so, how can that be configured?
The only things that I found are these docs:
https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/trace/v3/opentelemetry.proto - "sampler" near the end of the page
https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/trace/opentelemetry/samplers

I am unsure how to configure that though. And the "extension" is said to be "work-in-progress" in the note.
I did find the samplingRate in EnvoyProxy of gateway.envoyproxy.io/v1alpha1 but not the sampler option.

And also I couldn't find what the default is:

... This field can be left empty. In this case, the default Envoy sampling decision is used.
(source)

From the behavior I would assume its ParentBased(root=TraceIdRatioBased) since even with telemetry.tracing.samplingRate = 100 in EnvoyProxy of gateway.envoyproxy.io/v1alpha1 incoming requests traceparent-header trace-flags -00 are ignored. The ones with -01 are process / send to the exporter.

Or is that something that can not yet be configured / is being worked on ("work-in-progess")?

Any help / answer is much appreciated 😊

[zirain]

cc @codefromthecrypt

[codefromthecrypt]

Right now, the only control exposed is samplingRate (percentage of new traces to sample).

With no sampler configured, Envoy's OTel tracer uses its built-in tracing decision, which always respects the parent traceparent trace-flags. So samplingRate=100 behaves like ParentBased(root=AlwaysOn):

• No traceparent → sampled
• trace-flags=01 → sampled
• trace-flags=00 → not sampled

There is a workaround (I haven't tried but worth looking): You can use EnvoyPatchPolicy to inject a sampler into the generated xDS config:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyPatchPolicy
metadata:
  name: tracing-sampler
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: Gateway
    name: my-gateway
  type: JSONPatch
  jsonPatches:
    - type: "type.googleapis.com/envoy.config.trace.v3.OpenTelemetryConfig"
      operation:
        op: add
        path: "/sampler"
        value:
          name: envoy.tracers.opentelemetry.samplers.always_on
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.tracers.opentelemetry.samplers.v3.AlwaysOnSamplerConfig

The available samplers are listed here. All are marked wip in Envoy.

[jan-mrm]

Hi @codefromthecrypt, thanks for your answer.
I wanted to test the workaround but I don't think that works since the type only allows certain values currently: EnvoyPatchPolicy-EnvoyResourceType
trying results in error:

The EnvoyPatchPolicy "tracing-sampler" is invalid: 
* spec.jsonPatches[0].type: Unsupported value: 
"type.googleapis.com/envoy.config.trace.v3.OpenTelemetryConfig": 
supported values: 
"type.googleapis.com/envoy.config.listener.v3.Listener", 
"type.googleapis.com/envoy.config.route.v3.RouteConfiguration", 
"type.googleapis.com/envoy.config.cluster.v3.Cluster", 
"type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment", 
"type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret"

But thanks anyways. I now know thats not configurable yet but might be configurable in the future since the extension is wip.

[codefromthecrypt]

@jan-mrm sorry indeed I didn't try it, so apologies the hack didn't work. feel free to raise an issue to support sampler configurability as issues tend to lead more to resolution than discussions. Please mention exactly the config you are looking to achieve

[jan-mrm]

no worries (you even said you didn't test it). cool, thanks again @codefromthecrypt for your answers and help. I will open an issue/feature-request, probably tomorrow.
I'll close this discussion then as it is answered in my opinion.

not supported as it is right now.

[jan-mrm]

opened a feature request: #8476