[CLOUDS-7750] feat(docs): update AWS kinesis log documentation#35959
Open
RaphaelAllier wants to merge 1 commit intomasterfrom
Open
[CLOUDS-7750] feat(docs): update AWS kinesis log documentation#35959RaphaelAllier wants to merge 1 commit intomasterfrom
RaphaelAllier wants to merge 1 commit intomasterfrom
Conversation
Contributor
Preview links (active after the
|
urseberry
requested changes
Apr 14, 2026
|
|
||
| When logs arrive through Amazon Data Firehose, Datadog automatically determines the `source`, `service`, and tags for each log based on AWS metadata: | ||
|
|
||
| 1. **Source**: Detected from the CloudWatch Log Group name. For example, `/aws/lambda/my-function` sets `source:lambda`, `/aws/rds/...` sets `source:rds`. If the log group does not match a known AWS service pattern, the log group name itself is used as the source. |
Contributor
There was a problem hiding this comment.
Suggested change
| 1. **Source**: Detected from the CloudWatch Log Group name. For example, `/aws/lambda/my-function` sets `source:lambda`, `/aws/rds/...` sets `source:rds`. If the log group does not match a known AWS service pattern, the log group name itself is used as the source. | |
| 1. **Source**: Detected from the CloudWatch log group name. For example, `/aws/lambda/my-function` sets `source:lambda`, and `/aws/rds/...` sets `source:rds`. If the log group does not match a known AWS service pattern, the log group name itself is used as the source. |
|
|
||
| There are two ways to override these auto-detected values: | ||
|
|
||
| #### Firehose Common Attributes |
Contributor
There was a problem hiding this comment.
Suggested change
| #### Firehose Common Attributes | |
| #### Firehose common attributes |
|
|
||
| These fields are consumed during intake and do not appear as log attributes. | ||
|
|
||
| <div class="alert alert-info">This behavior is not enabled by default. To enable it for your organization, <a href="/help/">contact Datadog Support</a>.</div> |
Contributor
There was a problem hiding this comment.
Can you specifically describe the behavior that is not enabled by default? I lack context here, but it's not clear to me what feature I need to turn on.
Also, this information might be important for the user to know at the beginning of the section. Perhaps instead of this note, you could add a sentence at the beginning: " is turned off by default. To enable for your organization, contact Datadog Support."
|
|
||
| **Note**: A single Kinesis payload must not be more than 65,000 log messages. Log messages after that limit are dropped. | ||
|
|
||
| ## How source, service, and tags are set |
Contributor
There was a problem hiding this comment.
Suggested change
| ## How source, service, and tags are set | |
| ## Set the source, service, and tags |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do? What is the motivation?
Following https://datadoghq.atlassian.net/browse/CLOUDS-7750 I'm updating the docs on logs from Kinesis so customers have better expectations with regards to how the logs are parsed on the Datadog side
Merge instructions
N/A
Merge readiness:
AI assistance
Wrote the docs with a Claude prompt and did some iterations