From 78b82d06557cfdcd729087721ef9a33142b0ea0c Mon Sep 17 00:00:00 2001 From: Raphael Allier Date: Tue, 14 Apr 2026 11:46:50 +0200 Subject: [PATCH 1/5] [CLOUDS-7750] feat(docs): update AWS kinesis log documentation --- ...he-datadog-kinesis-firehose-destination.md | 40 +++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md b/content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md index f61f37aeb11..b46e762e6be 100644 --- a/content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md +++ b/content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md @@ -155,6 +155,46 @@ To populate all logs by ARN: **Note**: A single Kinesis payload must not be more than 65,000 log messages. Log messages after that limit are dropped. +## How source, service, and tags are set + +When logs arrive through Amazon Data Firehose, Datadog automatically determines the `source`, `service`, and tags for each log based on AWS metadata: + +1. **Source**: Detected from the CloudWatch Log Group name. For example, `/aws/lambda/my-function` sets `source:lambda`, `/aws/rds/...` sets `source:rds`. If the log group does not match a known AWS service pattern, the log group name itself is used as the source. +2. **Service**: Defaults to the same value as the detected source. For Lambda logs, the service is set to the function name. +3. **Tags**: AWS metadata such as `region`, `aws_account`, and `sourcecategory:aws` are added automatically. + +### Overriding source, service, and tags + +There are two ways to override these auto-detected values: + +#### Firehose Common Attributes + +You can set `source`, `service`, and arbitrary tags through the **Parameters** section (Common Attributes) of your Amazon Data Firehose delivery stream configuration. These values apply to all logs flowing through that delivery stream and take priority over the auto-detected values. + +For example, setting `source: nodejs` and `env: production` in the Common Attributes adds those as the log source and as a tag, respectively. + +#### Reserved attributes in the log body + +If your application writes structured JSON logs that include `ddsource`, `service`, or `ddtags` at the top level of the JSON body, Datadog can use those values instead of the auto-detected ones. For example: + +```json +{ + "message": "request completed", + "ddsource": "nodejs", + "service": "my-app", + "ddtags": "env:production,version:1.2.3" +} +``` + +When enabled, these fields take the highest priority, overriding both the auto-detected values and the Common Attributes: +- `ddsource` overrides the log source. +- `service` overrides the log service. +- `ddtags` values are merged into the log's tags (for example, `env` and `version` become searchable tags). + +These fields are consumed during intake and do not appear as log attributes. + +
This behavior is not enabled by default. To enable it for your organization, contact Datadog Support.
+ ## Further Reading {{< partial name="whats-next/whats-next.html" >}} From 5699cba1c0ca2f570d3e80c4b0f349543d862eda Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Rapha=C3=ABl=20Allier?= <118757729+RaphaelAllier@users.noreply.github.com> Date: Wed, 15 Apr 2026 11:36:29 +0200 Subject: [PATCH 2/5] [CLOUDS-7750] chore(kinesis): PR feedback Co-authored-by: Ursula Chen <58821586+urseberry@users.noreply.github.com> --- ...rvices-logs-with-the-datadog-kinesis-firehose-destination.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md b/content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md index b46e762e6be..4f1f02aae98 100644 --- a/content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md +++ b/content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md @@ -159,7 +159,7 @@ To populate all logs by ARN: When logs arrive through Amazon Data Firehose, Datadog automatically determines the `source`, `service`, and tags for each log based on AWS metadata: -1. **Source**: Detected from the CloudWatch Log Group name. For example, `/aws/lambda/my-function` sets `source:lambda`, `/aws/rds/...` sets `source:rds`. If the log group does not match a known AWS service pattern, the log group name itself is used as the source. +1. **Source**: Detected from the CloudWatch log group name. For example, `/aws/lambda/my-function` sets `source:lambda`, and `/aws/rds/...` sets `source:rds`. If the log group does not match a known AWS service pattern, the log group name itself is used as the source. 2. **Service**: Defaults to the same value as the detected source. For Lambda logs, the service is set to the function name. 3. **Tags**: AWS metadata such as `region`, `aws_account`, and `sourcecategory:aws` are added automatically. From 1e4228b58406342475eba264a9b8c5c0fe3413d3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Rapha=C3=ABl=20Allier?= <118757729+RaphaelAllier@users.noreply.github.com> Date: Wed, 15 Apr 2026 11:36:39 +0200 Subject: [PATCH 3/5] [CLOUDS-7750] chore(kinesis): PR feedback Co-authored-by: Ursula Chen <58821586+urseberry@users.noreply.github.com> --- ...rvices-logs-with-the-datadog-kinesis-firehose-destination.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md b/content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md index 4f1f02aae98..f795e7e2dc7 100644 --- a/content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md +++ b/content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md @@ -167,7 +167,7 @@ When logs arrive through Amazon Data Firehose, Datadog automatically determines There are two ways to override these auto-detected values: -#### Firehose Common Attributes +#### Firehose common attributes You can set `source`, `service`, and arbitrary tags through the **Parameters** section (Common Attributes) of your Amazon Data Firehose delivery stream configuration. These values apply to all logs flowing through that delivery stream and take priority over the auto-detected values. From c000205b74406fef9af4daf7d97e2a4aab4a1c6f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Rapha=C3=ABl=20Allier?= <118757729+RaphaelAllier@users.noreply.github.com> Date: Wed, 15 Apr 2026 11:36:47 +0200 Subject: [PATCH 4/5] [CLOUDS-7750] chore(kinesis): PR feedback Co-authored-by: Ursula Chen <58821586+urseberry@users.noreply.github.com> --- ...rvices-logs-with-the-datadog-kinesis-firehose-destination.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md b/content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md index f795e7e2dc7..94c3cdf930e 100644 --- a/content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md +++ b/content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md @@ -155,7 +155,7 @@ To populate all logs by ARN: **Note**: A single Kinesis payload must not be more than 65,000 log messages. Log messages after that limit are dropped. -## How source, service, and tags are set +## Set the source, service, and tags When logs arrive through Amazon Data Firehose, Datadog automatically determines the `source`, `service`, and tags for each log based on AWS metadata: From cb30e8090bd57c3ce36d1dc81de3592621a22696 Mon Sep 17 00:00:00 2001 From: Raphael Allier Date: Wed, 15 Apr 2026 11:44:42 +0200 Subject: [PATCH 5/5] [CLOUDS-7750] chore(kinesis): clarify reserved attributes alert --- ...ices-logs-with-the-datadog-kinesis-firehose-destination.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md b/content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md index 94c3cdf930e..388978124f2 100644 --- a/content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md +++ b/content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md @@ -175,6 +175,8 @@ For example, setting `source: nodejs` and `env: production` in the Common Attrib #### Reserved attributes in the log body +
Parsing reserved attributes (ddsource, service, ddtags) from the log body is not enabled by default. To enable it for your organization, contact Datadog Support.
+ If your application writes structured JSON logs that include `ddsource`, `service`, or `ddtags` at the top level of the JSON body, Datadog can use those values instead of the auto-detected ones. For example: ```json @@ -193,8 +195,6 @@ When enabled, these fields take the highest priority, overriding both the auto-d These fields are consumed during intake and do not appear as log attributes. -
This behavior is not enabled by default. To enable it for your organization, contact Datadog Support.
- ## Further Reading {{< partial name="whats-next/whats-next.html" >}}