Skip to content

[DOCS-13041] Add dedupe cache info #34723

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
estherk15 merged 3 commits into from
Feb 20, 2026
+6 −1
Merged

[DOCS-13041] Add dedupe cache info #34723

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1e46334
add info
maycmlee Feb 19, 2026
a1d647b
update
maycmlee Feb 19, 2026
43c4d7b
small edits
maycmlee Feb 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion content/en/observability_pipelines/processors/dedupe.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,12 @@ To set up the Deduplicate processor:

#### Cache size

The default cache size is 5,000 messages (recommended). The cached messages are kept in memory to determine if the incoming messages are duplicates. You can increase the cache size to fit your needs. **Note**: Increasing the cache size increases memory usage.
The default cache size is 5,000 messages (recommended). The cached messages are kept in memory to determine if the incoming messages are duplicates. You can increase the cache size to fit your needs.

**Notes**:
- Increasing the cache size increases memory usage.
- The cache is backed by a LRU cache, where the LRU cache size is the same as the configured cache size.
- Since the cache is not shared between Workers, only duplicated events processed by the same Worker are dropped.

### Path notation example

Expand Down
Loading