Skip to content

Commit 3b3ca1e

Browse files
leandrodamascenarubenfonsecaheitorlessa
authored
feat(jmespath): new built-in envelopes to unwrap S3 events (#2169)
Co-authored-by: Ruben Fonseca <fonseka@gmail.com> Co-authored-by: Heitor Lessa <lessa@amazon.co.uk>
1 parent 4cd6a6a commit 3b3ca1e

File tree

4 files changed

+42
-23
lines changed

4 files changed

+42
-23
lines changed

‎aws_lambda_powertools/utilities/jmespath_utils/envelopes.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,3 +6,8 @@
66
CLOUDWATCH_EVENTS_SCHEDULED = EVENTBRIDGE
77
KINESIS_DATA_STREAM = "Records[*].kinesis.powertools_json(powertools_base64(data))"
88
CLOUDWATCH_LOGS = "awslogs.powertools_base64_gzip(data) | powertools_json(@).logEvents[*]"
9+
S3_SNS_SQS = "Records[*].powertools_json(body).powertools_json(Message).Records[0]"
10+
S3_SQS = "Records[*].powertools_json(body).Records[0]"
11+
S3_SNS_KINESIS_FIREHOSE = "records[*].powertools_json(powertools_base64(data)).powertools_json(Message).Records[0]"
12+
S3_KINESIS_FIREHOSE = "records[*].powertools_json(powertools_base64(data)).Records[0]"
13+
S3_EVENTBRIDGE_SQS = "Records[*].powertools_json(body).detail"

‎docs/utilities/jmespath_functions.md‎

Lines changed: 18 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -64,16 +64,24 @@ We provide built-in envelopes for popular AWS Lambda event sources to easily dec
6464

6565
These are all built-in envelopes you can use along with their expression as a reference:
6666

67-
| Envelope | JMESPath expression |
68-
| --------------------------------- | ------------------------------------------------------------- |
69-
| **`API_GATEWAY_REST`** | `powertools_json(body)` |
70-
| **`API_GATEWAY_HTTP`** | `API_GATEWAY_REST` |
71-
| **`SQS`** | `Records[*].powertools_json(body)` |
72-
| **`SNS`** | `Records[0].Sns.Message | powertools_json(@)` |
73-
| **`EVENTBRIDGE`** | `detail` |
74-
| **`CLOUDWATCH_EVENTS_SCHEDULED`** | `EVENTBRIDGE` |
75-
| **`KINESIS_DATA_STREAM`** | `Records[*].kinesis.powertools_json(powertools_base64(data))` |
76-
| **`CLOUDWATCH_LOGS`** | `awslogs.powertools_base64_gzip(data) | powertools_json(@).logEvents[*]` |
67+
| Envelope | JMESPath expression |
68+
| --------------------------------- | ------------------------------------------------------------------------------------------ |
69+
| **`API_GATEWAY_HTTP`** | `powertools_json(body)` |
70+
| **`API_GATEWAY_REST`** | `powertools_json(body)` |
71+
| **`CLOUDWATCH_EVENTS_SCHEDULED`** | `detail` |
72+
| **`CLOUDWATCH_LOGS`** | `awslogs.powertools_base64_gzip(data) | powertools_json(@).logEvents[*]` |
73+
| **`EVENTBRIDGE`** | `detail` |
74+
| **`KINESIS_DATA_STREAM`** | `Records[*].kinesis.powertools_json(powertools_base64(data))` |
75+
| **`S3_EVENTBRIDGE_SQS`** | `Records[*].powertools_json(body).detail` |
76+
| **`S3_KINESIS_FIREHOSE`** | `records[*].powertools_json(powertools_base64(data)).Records[0]` |
77+
| **`S3_SNS_KINESIS_FIREHOSE`** | `records[*].powertools_json(powertools_base64(data)).powertools_json(Message).Records[0]` |
78+
| **`S3_SNS_SQS`** | `Records[*].powertools_json(body).powertools_json(Message).Records[0]` |
79+
| **`S3_SQS`** | `Records[*].powertools_json(body).Records[0]` |
80+
| **`SNS`** | `Records[0].Sns.Message | powertools_json(@)` |
81+
| **`SQS`** | `Records[*].powertools_json(body)` |
82+
83+
???+ tip "Using SNS?"
84+
If you don't require SNS metadata, enable [raw message delivery](https://docs.aws.amazon.com/sns/latest/dg/sns-large-payload-raw-message-delivery.html){target="_blank"}. It will reduce multiple payload layers and size, when using SNS in combination with other services _(e.g., SQS, S3, etc)_.
7785

7886
## Advanced
7987

‎docs/utilities/validation.md‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -141,16 +141,16 @@ We provide built-in envelopes to easily extract the payload from popular event s
141141

142142
Here is a handy table with built-in envelopes along with their JMESPath expressions in case you want to build your own.
143143

144-
| Envelope name | JMESPath expression |
145-
| ------------------------------- | ------------------------------------------------------------- |
146-
| **API_GATEWAY_REST** | "powertools_json(body)" |
147-
| **API_GATEWAY_HTTP** | "powertools_json(body)" |
148-
| **SQS** | "Records[*].powertools_json(body)" |
149-
| **SNS** | "Records[0].Sns.Message | powertools_json(@)" |
150-
| **EVENTBRIDGE** | "detail" |
151-
| **CLOUDWATCH_EVENTS_SCHEDULED** | "detail" |
152-
| **KINESIS_DATA_STREAM** | "Records[*].kinesis.powertools_json(powertools_base64(data))" |
153-
| **CLOUDWATCH_LOGS** | "awslogs.powertools_base64_gzip(data) | powertools_json(@).logEvents[*]" |
144+
| Envelope | JMESPath expression |
145+
| --------------------------------- | ------------------------------------------------------------------------ |
146+
| **`API_GATEWAY_HTTP`** | `powertools_json(body)` |
147+
| **`API_GATEWAY_REST`** | `powertools_json(body)` |
148+
| **`CLOUDWATCH_EVENTS_SCHEDULED`** | `detail` |
149+
| **`CLOUDWATCH_LOGS`** | `awslogs.powertools_base64_gzip(data) | powertools_json(@).logEvents[*]` |
150+
| **`EVENTBRIDGE`** | `detail` |
151+
| **`KINESIS_DATA_STREAM`** | `Records[*].kinesis.powertools_json(powertools_base64(data))` |
152+
| **`SNS`** | `Records[0].Sns.Message | powertools_json(@)` |
153+
| **`SQS`** | `Records[*].powertools_json(body)` |
154154

155155
## Advanced
156156

‎examples/jmespath_functions/src/extract_data_from_builtin_envelope.py‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,18 @@
1+
from __future__ import annotations
2+
3+
from aws_lambda_powertools import Logger
14
from aws_lambda_powertools.utilities.jmespath_utils import (
25
envelopes,
36
extract_data_from_envelope,
47
)
58
from aws_lambda_powertools.utilities.typing import LambdaContext
69

10+
logger = Logger()
11+
712

813
def handler(event: dict, context: LambdaContext) -> dict:
9-
payload = extract_data_from_envelope(data=event, envelope=envelopes.SQS)
10-
customer_id = payload.get("customerId") # now deserialized
14+
records: list = extract_data_from_envelope(data=event, envelope=envelopes.SQS)
15+
for record in records: # records is a list
16+
logger.info(record.get("customerId")) # now deserialized
1117

12-
return {"customer_id": customer_id, "message": "success", "statusCode": 200}
18+
return {"message": "success", "statusCode": 200}

0 commit comments

Comments
 (0)