Skip to content

Commit 25858d2

Browse files
author
Janez Justin
committed
code polish and README update
1 parent ba9525d commit 25858d2

3 files changed

Lines changed: 112 additions & 35 deletions

File tree

‎rpm/Makefile‎

Lines changed: 20 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,23 @@
1-
ZIPPED := s3rpm.py gnupg.py pyrpm/* pyrpm/tools/*
1+
ZIPPED_FILES := s3rpm.py requirements/gnupg.py # files to compress in root of zip
2+
ZIPPED_DIR := pyrpm # folders from requirements folder to compress to root of zip
23

4+
all: requires test package
35

4-
set: requires package
6+
help: ## displays this message
7+
@grep -E '^[a-zA-Z_/%\-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
58

6-
requires:
7-
pip3 install -t . -r requirements.txt --upgrade
8-
9-
package:
10-
zip code.zip $(ZIPPED)
9+
requires: ## installs required packages
10+
pip3 install -t ./requirements -r requirements.txt
11+
12+
package: ## creates zip of code
13+
zip -j code.zip $(ZIPPED_FILES)
14+
cd requirements && zip -r ../code.zip $(ZIPPED_DIR)
15+
16+
clean: ## cleans up the repository
17+
/bin/rm -rf code.zip
18+
/bin/rm -rf ./requirements
19+
20+
test: ## runs the tests
21+
cp s3rpm_test.py requirements
22+
cp s3rpm.py requirements
23+
cd requirements && python3.6 s3rpm_test.py

‎rpm/README.md‎

Lines changed: 91 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -2,36 +2,68 @@
22

33
Automatic YUM repository building inside S3 bucket using with lambda support
44

5-
## Setting up S3 and Lambda
6-
7-
Clone the repo and get all other required files
5+
## Readme contents
6+
7+
* [Setting up code, S3 and Lambda](#setting-up-code-s3-and-lambda)
8+
* [Getting the code](#getting-the-code)
9+
* [GPG key](#gpg-key)
10+
* [Environmental variables](#environmental-variables)
11+
* [Set up role](#set-up-role)
12+
* [Set up lambda with CLI](#set-up-lambda-with-cli)
13+
* [Set up lambda manually](#set-up-lambda-manually)
14+
* [The triggers](#the-triggers)
15+
* [Set up S3](#set-up-s3)
16+
* [Setting up yum](#setting-up-yum)
17+
* [First time set up](#first-time-set-up)
18+
* [Install/update](#installupdate)
19+
* [Notes](#notes)
20+
* [Tests](#tests)
21+
22+
## Setting up code, S3 and Lambda
23+
24+
### Getting the code
25+
Clone the repo, get all other required files and compress them
826
```
927
git clone https://github.com/tactycal/lambdaRepos.git
1028
cd lambdaRepos/rpm
11-
pip3 install -t . -r requirements.txt
29+
make all
1230
```
1331

14-
Compress all needed files
32+
### GPG key
33+
create your gpg key (skip to exporting your key, if you already have it)
1534
```
16-
zip code.zip s3rpm.py gnupg.py pyrpm/* pyrpm/tools/*
35+
gpg --gen-key
36+
# Follow the instructions
37+
# Create 'RSA and RSA' key - option 1
38+
# For maxium encryption it is recommended to make 4096 bits long key
39+
# Key should not expire
1740
```
1841

19-
Or just use `make set` instead of `zip` and `pip3` command
42+
export your key
2043

21-
Presuming you already have GPG key generated export secret key (you can skip this part if you don't want to GPG sign your repository)
2244
```
23-
gpg -a --export-secret-key > secret.key
45+
gpg --export-secret-key -a "User Name" > secret.key # exports secret key to secret.key
2446
```
2547

26-
Create new lambda function, set handler to **s3rpm.lambda_handler**, runtime to **python 3.6** and the triggers to:
48+
### Set up role
2749

28-
* Object Created, suffix 'rpm'
29-
* Object Removed, suffix 'rpm'
30-
* If you are using certain directory as a repo, set it as prefix
50+
Create new role with s3 write/read access
3151

32-
Upload `code.zip` to lambda function
52+
Here is a minimal requirement for the policy that is included in role:
53+
```
54+
{"Version": "2012-10-17",
55+
"Statement": [
56+
{"Sid": "<THIS IS UNIQE>",
57+
"Action": [
58+
"s3:GetObject",
59+
"s3:PutObject",
60+
"s3:PutObjectAcl"],
61+
"Effect": "Allow",
62+
"Resource": "arn:aws:s3:::<YOUR BUCKET NAME>/*"}]}
63+
```
3364

34-
Set the environmental variables
65+
### Environmental variables
66+
These are the environmental variables you will have to set:
3567

3668
| Key | Value |
3769
| --- | ---|
@@ -51,14 +83,44 @@ Set the environmental variables
5183

5284
**REPO_DIR** Path to repositroy from bucket root. If none is set, it is assumed root of repository is root of the bucket
5385

86+
### Set up lambda with CLI
87+
88+
[Install aws cli](http://docs.aws.amazon.com/cli/latest/userguide/installing.html)
89+
90+
Create new lambda function:
91+
```
92+
aws lambda create-function \
93+
--function-name <name the function> \
94+
--zip-file fileb://code.zip \
95+
--role <role's arn> \ # arn from role with S3 read/write access
96+
--handler s3rpm.handler \
97+
--runtime python3.6 \
98+
# Replace '<...>' with environmental variables
99+
--environment Variables='{PUBLIC=<bool>, GPG_KEY=<file>, GPG_PASS=<password>, BUCKET_NAME=<bucket name>, REPO_DIR=<dir>}'
100+
```
101+
102+
### Set up lambda manually
54103

104+
If CLI is not your thing, then you can upload code manaully
105+
106+
Create new lambda function, set handler to **s3rpm.lambda_handler**, runtime to **python 3.6**
107+
108+
Upload `code.zip` to lambda function
109+
110+
### The triggers
111+
112+
* Object Created(All), suffix 'rpm'
113+
* Object Removed(All), suffix 'rpm'
114+
* If you are using certain directory as a repo, set it as prefix
115+
116+
### Set up S3
55117
Upload secret key file to location you specified as GPG_KEY
56118

57-
Upload GPG SIGNED .rpm file to desired folder, lambda function should now keep your repository up to date
119+
Upload .rpm file to desired folder, lambda function should now keep your repository up to date
58120

59121
## Setting up yum
60122

61-
**First time set up**
123+
### First time set up
62124

63125
create `example.repo` file in `/etc/yum.repos.d/example.repo`
64126
```
@@ -78,23 +140,28 @@ gpgkey=<link to public key of key you used for signing metadata files>
78140
* You can do `repo_gpgcheck=0` to skip gpg verification when installing packages
79141
* You can do `gpgcheck=1` if you are uploading signed rpm packages(lambda does not sign them, it signs only metadata xml file)
80142

143+
### Install/update
81144
Install package
82145
```
83-
su
84-
yum install <package name>
146+
sudo yum install <package name>
85147
```
86148

87149
Upgrading package
88150
```
89-
su
90-
yum upgrade
151+
sudo yum upgrade
91152
```
92153

93154
## Notes
94155

95-
.rpm and repodata/* in repository directory are and should be publicly accessible
156+
* .rpm and repodata/* in repository directory are and should be publicly accessible for the
157+
158+
* Don't forget to increase the timeout of lambda function
96159

97-
Don't forget to increase the timeout of lambda function
160+
* If somebody tries to inject a malicious rpm file in your repo it will be automaticly added to repository. It is your job to make bucket secure enough for this not to happen.!!!
98161

99-
If somebody tries to inject a malicious rpm file in your repo it will be automaticly added to repository. It is your job to make bucket secure enough for this not to happen.!!!
162+
## Tests
100163

164+
To run unit tests:
165+
```
166+
make test
167+
```

‎rpm/s3rpm.py‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -15,10 +15,7 @@ def lambda_handler(event, context):
1515
repo = YumRepository('/tmp/repo/') # set repository
1616
prefix = '/'.join(key.split('/')[0:-1])+'/'
1717

18-
if os.environ['REPO_DIR'].endswith('/'):
19-
os.environ['REPO_DIR'] = os.environ['REPO_DIR'][:-1]
20-
if os.environ['REPO_DIR'].startswith('/'):
21-
os.environ['REPO_DIR'] = os.environ['REPO_DIR'][1:]
18+
os.environ['REPO_DIR'] = os.environ['REPO_DIR'].strip('/')
2219

2320
#make sure we are working with correct files
2421
if bucket == os.environ['BUCKET_NAME'] and key.endswith(".rpm") and prefix.startswith(os.environ['REPO_DIR']):

0 commit comments

Comments
 (0)