Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 14 additions & 19 deletions elastio-s3-changelog/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,31 +16,26 @@ Then, the Elastio `iscan` job reads those events to perform the scanning of new

**Important!** You can change the stack name, but it **MUST** start with `elastio-`. Otherwise, Elastio won't be able to access the created resources.

* [us-east-1](https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v3/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [us-east-2](https://us-east-2.console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v3/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [us-west-1](https://us-west-1.console.aws.amazon.com/cloudformation/home?region=us-west-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v3/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [us-west-2](https://us-west-2.console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v3/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [eu-central-1](https://eu-central-1.console.aws.amazon.com/cloudformation/home?region=eu-central-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v3/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [eu-west-1](https://eu-west-1.console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v3/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [eu-west-2](https://eu-west-2.console.aws.amazon.com/cloudformation/home?region=eu-west-2#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v3/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [eu-west-3](https://eu-west-3.console.aws.amazon.com/cloudformation/home?region=eu-west-3#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v3/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [ca-central-1](https://ca-central-1.console.aws.amazon.com/cloudformation/home?region=ca-central-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v3/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [ap-south-1](https://ap-south-1.console.aws.amazon.com/cloudformation/home?region=ap-south-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v3/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [ap-southeast-1](https://ap-southeast-1.console.aws.amazon.com/cloudformation/home?region=ap-southeast-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v3/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [ap-southeast-2](https://ap-southeast-2.console.aws.amazon.com/cloudformation/home?region=ap-southeast-2#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v3/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [me-central-1](https://me-central-1.console.aws.amazon.com/cloudformation/home?region=me-central-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v3/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [us-east-1](https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v4/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [us-east-2](https://us-east-2.console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v4/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [us-west-1](https://us-west-1.console.aws.amazon.com/cloudformation/home?region=us-west-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v4/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [us-west-2](https://us-west-2.console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v4/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [eu-central-1](https://eu-central-1.console.aws.amazon.com/cloudformation/home?region=eu-central-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v4/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [eu-west-1](https://eu-west-1.console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v4/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [eu-west-2](https://eu-west-2.console.aws.amazon.com/cloudformation/home?region=eu-west-2#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v4/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [eu-west-3](https://eu-west-3.console.aws.amazon.com/cloudformation/home?region=eu-west-3#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v4/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [ca-central-1](https://ca-central-1.console.aws.amazon.com/cloudformation/home?region=ca-central-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v4/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [ap-south-1](https://ap-south-1.console.aws.amazon.com/cloudformation/home?region=ap-south-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v4/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [ap-southeast-1](https://ap-southeast-1.console.aws.amazon.com/cloudformation/home?region=ap-southeast-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v4/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [ap-southeast-2](https://ap-southeast-2.console.aws.amazon.com/cloudformation/home?region=ap-southeast-2#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v4/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)
* [me-central-1](https://me-central-1.console.aws.amazon.com/cloudformation/home?region=me-central-1#/stacks/create/review?templateURL=https://elastio-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com/contrib/elastio-s3-changelog/v4/cloudformation-multiple-buckets.yaml&stackName=elastio-s3-changelog)

3. Fill in the main parameters:
* *BucketNames* - comma-separated list of S3 bucket names;

* *CrossAccount* - set this to true if the scan happens in a different AWS account than the
S3 bucket (i.e. you are using a centralized scanning account). If it is not set
to true in the cross-account case, an error about missing AWS Lambda functions
will occur during stack creation.

* *ScanExistingObjects* - if set to true, the full initial scan of the bucket will be performed.
If set to false, only new objects in the bucket will be scanned.
This option can only be enabled when CrossAccount is false. If it is not set
This option can only be enabled in non-cross-account scenario. If it is not set
to false in the cross-account case, an error about missing AWS Lambda functions
will occur during stack creation.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,6 @@ Transform:
Parameters:
BucketNames:
Type: String
CrossAccount:
Type: String
Default: "false"
ScanExistingObjects:
Type: String
Default: "false"
Expand All @@ -33,7 +30,6 @@ Resources:
Properties:
Parameters:
BucketName: !Ref BucketName
CrossAccount: !Ref CrossAccount
ScanExistingObjects: !Ref ScanExistingObjects
KeyPrefixes: !Ref KeyPrefixes
DisableEventCollection: !Ref DisableEventCollection
Expand Down
14 changes: 1 addition & 13 deletions elastio-s3-changelog/cloudformation-multiple-buckets.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,24 +12,14 @@ Parameters:
Comma-delimited list of S3 bucket names.
Example: examplebucket1, examplebucket2

CrossAccount:
Type: String
AllowedValues: ["true", "false"]
Default: "false"
Description: >
Set this to true if the scan happens in a different AWS account than the
S3 bucket (i.e. you are using a centralized scanning account). If it is not set
to true in the cross-account case, an error about missing AWS Lambda functions
will occur during stack creation.

ScanExistingObjects:
Type: String
AllowedValues: ["true", "false"]
Default: "false"
Description: >
If set to true, the full initial scan of the bucket will be performed.
If set to false, only new objects in the bucket will be scanned.
This option can only be enabled when CrossAccount is false. If it is not set
This option can only be enabled in non-cross-account scenario. If it is not set
to false in the cross-account case, an error about missing AWS Lambda functions
will occur during stack creation.

Expand Down Expand Up @@ -63,7 +53,6 @@ Metadata:
- Label: { default: Main Parameters }
Parameters:
- BucketNames
- CrossAccount
- ScanExistingObjects
- KeyPrefixes

Expand All @@ -78,7 +67,6 @@ Resources:
Properties:
Parameters:
BucketNames: !Join [",", !Ref BucketNames]
CrossAccount: !Ref CrossAccount
ScanExistingObjects: !Ref ScanExistingObjects
KeyPrefixes: !Join [",", !Ref KeyPrefixes]
DisableEventCollection: !Ref DisableEventCollection
Expand Down
38 changes: 1 addition & 37 deletions elastio-s3-changelog/cloudformation-single-bucket.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,24 +9,14 @@ Parameters:
ConstraintDescription: Bucket name must be at least 3 characters long.
Description: S3 bucket name.

CrossAccount:
Type: String
AllowedValues: ["true", "false"]
Default: "false"
Description: >
Set this to true if the scan happens in a different AWS account than the
S3 bucket (i.e. you are using a centralized scanning account). If it is not set
to true in the cross-account case, an error about missing AWS Lambda functions
will occur during stack creation.

ScanExistingObjects:
Type: String
AllowedValues: ["true", "false"]
Default: "false"
Description: >
If set to true, the full initial scan of the bucket will be performed.
If set to false, only new objects in the bucket will be scanned.
This option can only be enabled when CrossAccount is false. If it is not set
This option can only be enabled in non-cross-account scenario. If it is not set
to false in the cross-account case, an error about missing AWS Lambda functions
will occur during stack creation.

Expand Down Expand Up @@ -62,7 +52,6 @@ Metadata:
- Label: { default: Main Parameters }
Parameters:
- BucketName
- CrossAccount
- ScanExistingObjects
- KeyPrefixes

Expand All @@ -72,7 +61,6 @@ Metadata:
- EnableDlq

Conditions:
NotCrossAccount: !Equals [!Ref CrossAccount, "false"]
ScanExistingObjects: !Equals [!Ref ScanExistingObjects, "true"]
DisableEventCollection: !Equals [!Ref DisableEventCollection, "true"]
EnableDlq: !Equals [!Ref EnableDlq, "true"]
Expand Down Expand Up @@ -190,30 +178,6 @@ Resources:
Tags:
"elastio:resource": "true"

queueRefreshSchedule:
Type: AWS::Scheduler::Schedule
Condition: NotCrossAccount
Properties:
Description: A schedule to invoke the periodic changelog queue refresh lambda
ScheduleExpression: cron(0/15 * * * ? *)
FlexibleTimeWindow:
Mode: "OFF"
State: !If
- DisableEventCollection
- DISABLED
- ENABLED
Target:
Arn: !Sub "arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:elastio-s3-changelog-handler-refresh"
RoleArn: !Sub "arn:aws:iam::${AWS::AccountId}:role/ElastioS3ChangelogSchedulerExecution"
Input: !Sub >
{
"queue_url": "${queue}",
"queue_name": "${queue.QueueName}"
}
RetryPolicy:
MaximumEventAgeInSeconds: 60
MaximumRetryAttempts: 3

scanExistingObjectsCustom:
Type: Custom::ScanExistingObjects
Condition: ScanExistingObjects
Expand Down
2 changes: 1 addition & 1 deletion elastio-s3-changelog/version
Original file line number Diff line number Diff line change
@@ -1 +1 @@
v3
v4
Loading