11# Record Modifier
22
3- The _ Record Modifier Filter_ plugin allows to append fields or to exclude specific fields.
3+ The _ Record Modifier_ [ filter] ( pipeline/filters.md ) plugin lets you append
4+ fields to a record, or exclude specific fields.
45
5- ## Configuration Parameters
6+ ## Configuration parameters
67
7- The plugin supports the following configuration parameters: _ Remove \_ key _ and _ Allowlist \_ key _ are exclusive.
8+ The plugin supports the following configuration parameters:
89
910| Key | Description |
1011| :--- | :--- |
11- | Record | Append fields. This parameter needs key and value pair. |
12- | Remove \_ key | If the key is matched, that field is removed. |
13- | Allowlist \_ key | If the key is ** not ** matched, that field is removed. |
14- | Whitelist \_ key | An alias of ` Allowlist_key ` for backwards compatibility. |
15- | Uuid \_ key | If set, the plugin appends uuid to each record. The value assigned becomes the key in the map.|
12+ | ` Record ` | Append fields. This parameter needs a key/ value pair. |
13+ | ` Remove_key ` | If the key is matched, that field is removed. You can this or ` Allowlist_key ` . |
14+ | ` Allowlist_key ` | If the key isn't matched, that field is removed. You can this or ` Remove_key ` . |
15+ | ` Whitelist_key ` | An alias of ` Allowlist_key ` for backwards compatibility. |
16+ | ` Uuid_key ` | If set, the plugin appends Uuid to each record. The value assigned becomes the key in the map. |
1617
17- ## Getting Started
18+ ## Get started
1819
19- In order to start filtering records, you can run the filter from the command line or through the configuration file.
20+ To start filtering records, run the filter from the command line or through a
21+ configuration file.
2022
21- This is a sample in \_ mem record to filter.
23+ This is a sample ` in_mem ` record to filter.
2224
2325``` text
2426{"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>299352, "Swap.total"=>2064380, "Swap.used"=>32656, "Swap.free"=>2031724}
2527```
2628
2729### Append fields
2830
29- The following configuration file is to append product name and hostname \( via environment variable\) to record.
31+ The following configuration file appends a product name and hostname to a record
32+ using an environment variable:
3033
3134{% tabs %}
3235{% tab title="fluent-bit.conf" %}
33- ``` python
36+
37+ ``` python copy
3438[INPUT ]
3539 Name mem
3640 Tag mem.local
@@ -45,48 +49,50 @@ The following configuration file is to append product name and hostname \(via en
4549 Record hostname $ {HOSTNAME }
4650 Record product Awesome_Tool
4751```
52+
4853{% endtab %}
4954
5055{% tab title="fluent-bit.yaml" %}
51- ``` yaml
56+
57+ ``` yaml copy
5258pipeline :
5359 inputs :
5460 - name : mem
5561 tag : mem.local
5662 filters :
5763 - name : record_modifier
5864 match : ' *'
59- record :
65+ record :
6066 - hostname ${HOSTNAME}
6167 - product Awesome_Tool
6268 outputs :
6369 - name : stdout
6470 match : ' *'
6571` ` `
72+
6673{% endtab %}
6774{% endtabs %}
6875
76+ You can run the filter from command line:
6977
70- You can also run the filter from command line.
71-
72- ` ` ` text
73- $ fluent-bit -i mem -o stdout -F record_modifier -p 'Record=hostname ${HOSTNAME}' -p 'Record=product Awesome_Tool' -m '*'
78+ ` ` ` shell copy
79+ fluent-bit -i mem -o stdout -F record_modifier -p 'Record=hostname ${HOSTNAME}' -p 'Record=product Awesome_Tool' -m '*'
7480```
7581
76- The output will be
82+ The output looks something like:
7783
78- ``` python
84+ ``` python copy
7985[0 ] mem.local: [1492436882.000000000 , {" Mem.total" => 1016024 , " Mem.used" => 716672 , " Mem.free" => 299352 , " Swap.total" => 2064380 , " Swap.used" => 32656 , " Swap.free" => 2031724 , " hostname" => " localhost.localdomain" , " product" => " Awesome_Tool" }]
8086```
8187
82- ### Remove fields with Remove\_ key
83-
84- The following configuration file is to remove 'Swap.\* ' fields.
88+ ### Remove fields with ` Remove_key `
8589
90+ The following configuration file removes ` Swap.* ` fields:
8691
8792{% tabs %}
8893{% tab title="fluent-bit.conf" %}
89- ``` python
94+
95+ ``` python copy
9096[INPUT ]
9197 Name mem
9298 Tag mem.local
@@ -102,47 +108,51 @@ The following configuration file is to remove 'Swap.\*' fields.
102108 Remove_key Swap.used
103109 Remove_key Swap.free
104110```
111+
105112{% endtab %}
106113
107114{% tab title="fluent-bit.yaml" %}
108- ``` yaml
115+
116+ ``` yaml copy
109117pipeline :
110118 inputs :
111119 - name : mem
112120 tag : mem.local
113121 filters :
114122 - name : record_modifier
115123 match : ' *'
116- remove_key :
124+ remove_key :
117125 - Swap.total
118126 - Swap.used
119127 - Swap.free
120128 outputs :
121129 - name : stdout
122130 match : ' *'
123131` ` `
132+
124133{% endtab %}
125134{% endtabs %}
126135
127136You can also run the filter from command line.
128137
129- ` ` ` text
130- $ fluent-bit -i mem -o stdout -F record_modifier -p 'Remove_key=Swap.total' -p 'Remove_key=Swap.free' -p 'Remove_key=Swap.used' -m '*'
138+ ` ` ` shell copy
139+ fluent-bit -i mem -o stdout -F record_modifier -p 'Remove_key=Swap.total' -p 'Remove_key=Swap.free' -p 'Remove_key=Swap.used' -m '*'
131140```
132141
133- The output will be
142+ The output looks something like:
134143
135144``` python
136145[0 ] mem.local: [1492436998.000000000 , {" Mem.total" => 1016024 , " Mem.used" => 716672 , " Mem.free" => 295332 }]
137146```
138147
139- ### Remove fields with Allowlist \_ key
148+ ### Retain fields with ` Allowlist_key `
140149
141- The following configuration file is to remain ' Mem.\* ' fields.
150+ The following configuration file retains ` Mem.* ` fields.
142151
143152{% tabs %}
144153{% tab title="fluent-bit.conf" %}
145- ``` python
154+
155+ ``` python copy
146156[INPUT ]
147157 Name mem
148158 Tag mem.local
@@ -158,37 +168,39 @@ The following configuration file is to remain 'Mem.\*' fields.
158168 Allowlist_key Mem.used
159169 Allowlist_key Mem.free
160170```
171+
161172{% endtab %}
162173
163174{% tab title="fluent-bit.yaml" %}
164- ``` yaml
175+
176+ ``` yaml copy
165177pipeline :
166178 inputs :
167179 - name : mem
168180 tag : mem.local
169181 filters :
170182 - name : record_modifier
171183 match : ' *'
172- Allowlist_key :
184+ Allowlist_key :
173185 - Mem.total
174186 - Mem.used
175187 - Mem.free
176188 outputs :
177189 - name : stdout
178190 match : ' *'
179191` ` `
192+
180193{% endtab %}
181194{% endtabs %}
182195
183- You can also run the filter from command line.
196+ You can also run the filter from command line:
184197
185- ` ` ` text
186- $ fluent-bit -i mem -o stdout -F record_modifier -p 'Allowlist_key=Mem.total' -p 'Allowlist_key=Mem.free' -p 'Allowlist_key=Mem.used' -m '*'
198+ ` ` ` shell copy
199+ fluent-bit -i mem -o stdout -F record_modifier -p 'Allowlist_key=Mem.total' -p 'Allowlist_key=Mem.free' -p 'Allowlist_key=Mem.used' -m '*'
187200```
188201
189- The output will be
202+ The output looks something like:
190203
191204``` python
192205[0 ] mem.local: [1492436998.000000000 , {" Mem.total" => 1016024 , " Mem.used" => 716672 , " Mem.free" => 295332 }]
193206```
194-
0 commit comments