Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,46 @@

**Note**: A single Kinesis payload must not be more than 65,000 log messages. Log messages after that limit are dropped.

## Set the source, service, and tags

When logs arrive through Amazon Data Firehose, Datadog automatically determines the `source`, `service`, and tags for each log based on AWS metadata:

1. **Source**: Detected from the CloudWatch log group name. For example, `/aws/lambda/my-function` sets `source:lambda`, and `/aws/rds/...` sets `source:rds`. If the log group does not match a known AWS service pattern, the log group name itself is used as the source.
2. **Service**: Defaults to the same value as the detected source. For Lambda logs, the service is set to the function name.
3. **Tags**: AWS metadata such as `region`, `aws_account`, and `sourcecategory:aws` are added automatically.

### Overriding source, service, and tags

There are two ways to override these auto-detected values:

#### Firehose common attributes

You can set `source`, `service`, and arbitrary tags through the **Parameters** section (Common Attributes) of your Amazon Data Firehose delivery stream configuration. These values apply to all logs flowing through that delivery stream and take priority over the auto-detected values.

For example, setting `source: nodejs` and `env: production` in the Common Attributes adds those as the log source and as a tag, respectively.

#### Reserved attributes in the log body

<div class="alert alert-info">Parsing reserved attributes (<code>ddsource</code>, <code>service</code>, <code>ddtags</code>) from the log body is not enabled by default. To enable it for your organization, <a href="/help/">contact Datadog Support</a>.</div>

If your application writes structured JSON logs that include `ddsource`, `service`, or `ddtags` at the top level of the JSON body, Datadog can use those values instead of the auto-detected ones. For example:

Check notice on line 180 in content/en/logs/guide/send-aws-services-logs-with-the-datadog-kinesis-firehose-destination.md

View workflow job for this annotation

GitHub Actions / vale

Datadog.sentencelength

Suggestion: Try to keep your sentence length to 25 words or fewer.

```json
{
"message": "request completed",
"ddsource": "nodejs",
"service": "my-app",
"ddtags": "env:production,version:1.2.3"
}
```

When enabled, these fields take the highest priority, overriding both the auto-detected values and the Common Attributes:
- `ddsource` overrides the log source.
- `service` overrides the log service.
- `ddtags` values are merged into the log's tags (for example, `env` and `version` become searchable tags).

These fields are consumed during intake and do not appear as log attributes.

## Further Reading

{{< partial name="whats-next/whats-next.html" >}}
Expand Down
Loading