You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: 00-installation/README.md
+2-3Lines changed: 2 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,14 +1,13 @@
1
1
# Installing Singularity
2
-
Here we will install the latest tagged release from [GitHub](https://github.com/singularityware/singularity). If you prefer to install a different version or to install Singularity in a different location, see these [Singularity docs](https://sylabs.io/guides/3.5/admin-guide/installation.html).
2
+
Here we will install the latest tagged release from [GitHub](https://github.com/sylabs/singularity/releases). If you prefer to install a different version or to install Singularity in a different location, see these [Singularity docs](https://sylabs.io/guides/3.5/admin-guide/installation.html).
3
3
4
4
We're going to compile Singularity from source code. First we'll need to make sure we have some development tools and libraries installed so that we can do that. On Ubuntu, run these commands to make sure you have all the necessary packages installed.
In this exercise, we will build a container from scratch similar to the one we used to test the installation.
4
-
To build a singularity container, you must use the `build` command. The `build` command installs an OS, sets up your container's environment and installs the apps you need. To use the `build` command, we need a **recipe file** (also called a definition file). A Singularity recipe file is a set of instructions telling Singularity what software to install in the container.
3
+
In this exercise, we will build a container from scratch similar to the lolcow container we used to test the installation.
5
4
6
-
The Singularity source code contains several example definition files in the `/examples` subdirectory. Let's copy the ubuntu example to our home directory and inspect it.
5
+
We are going to use a standard development cycle (sometimes referred to as Singularity flow) to create this container. It consists of the following steps:
6
+
7
+
- create a writable container (called a `sandbox`)
8
+
- shell into the container and tinker with it interactively
9
+
- record changes that we like in our definition file
10
+
- rebuild the container from the definition file if we break it
11
+
- rinse and repeat until we are happy with the result
12
+
- rebuild the container as a read-only singularity image format (SIF) image for use in production
13
+
14
+
To build a singularity container, you must use the `build` command. The `build` command installs an OS, sets up your container's environment and installs the apps you need. To use the `build` command, we need a definition file. A [Singularity definition file](https://sylabs.io/guides/3.5/user-guide/definition_files.html) is a set of instructions telling Singularity what software to install in the container.
15
+
16
+
The Singularity source code contains several example definition files in the `/examples` subdirectory. Let's make a new directory, copy the Debian example definition file, and inspect it.
See the [Singularity docs](http://singularity.lbl.gov/docs-recipes) for an explanation of each of these sections.
44
+
See the [Singularity docs](https://sylabs.io/guides/3.5/user-guide/definition_files.html) for an explanation of each of these sections.
36
45
37
-
Now let's use this recipe file as a starting point to build our `lolcow.img` container. Note that the build command requires `sudo` privileges, when used in combination with a recipe file.
46
+
Now let's use this definition file as a starting point to build our `lolcow.img` container. Note that the build command requires `sudo` privileges. (We'll discuss some ways around this restriction later in the class.)
The `--sandbox` option in the command above tells Singularity that we want to build a special type of container for development purposes.
52
+
This is telling Singularity to build a container called `lolcow` from the `lolcow.def` definition file. The `--sandbox` option in the command above tells Singularity that we want to build a special type of container (called a sandbox) for development purposes.
44
53
45
-
Singularity can build containers in several different file formats. The default is to build a [squashfs](https://en.wikipedia.org/wiki/SquashFS)image. The squashfs format is compressed and immutable making it a good choice for reproducible, production-grade containers.
54
+
Singularity can build containers in several different file formats. The default is to build a SIF (singularity image format) container that uses [squashfs](https://en.wikipedia.org/wiki/SquashFS)for the file system. The squashfs format is compressed and immutable making it a good choice for reproducible, production-grade containers.
46
55
47
-
But if you want to shell into a container and tinker with it (like we will do here), you should build a sandbox (which is really just a directory). This is great when you are still developing your container and don't yet know what should be included in the recipe file.
56
+
But if you want to shell into a container and tinker with it (like we will do here), you should build a sandbox (which is really just a directory). This is great when you are still developing your container and don't yet know what to include in the definition file.
48
57
49
-
When your build finishes, you will have a basic Ubuntu container saved in a local directory called `lolcow`.
58
+
When your build finishes, you will have a basic Debian container saved in a local directory called `lolcow`.
50
59
51
-
# Using `shell` to explore and modify containers
60
+
##Using `shell` to explore and modify containers
52
61
53
62
Now let's enter our new container and look around.
54
63
@@ -60,10 +69,11 @@ Depending on the environment on your host system you may see your prompt change.
60
69
61
70
```
62
71
Singularity lolcow:~> cat /etc/os-release
63
-
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
72
+
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
64
73
NAME="Debian GNU/Linux"
65
-
VERSION_ID="9"
66
-
VERSION="9 (stretch)"
74
+
VERSION_ID="10"
75
+
VERSION="10 (buster)"
76
+
VERSION_CODENAME=buster
67
77
ID=debian
68
78
HOME_URL="https://www.debian.org/"
69
79
SUPPORT_URL="https://www.debian.org/support"
@@ -75,32 +85,31 @@ No matter what OS is running on your host, your container is running Debian Stab
75
85
Let's try a few more commands:
76
86
77
87
```
78
-
Singularity lolcow:~> whoami
88
+
Singularity> whoami
79
89
dave
80
90
81
-
Singularity lolcow:~> hostname
91
+
Singularity> hostname
82
92
hal-9000
83
93
```
84
94
85
-
This is one of the core features of Singularity that makes it so attractive from a security standpoint. The user remains the same inside and outside of the container.
95
+
This is one of the core features of Singularity that makes it so attractive from a security and usability standpoint. The user remains the same inside and outside of the container.
86
96
87
97
Let's try installing some software. I used the programs `fortune`, `cowsay`, and `lolcat` to produce the container that we saw in the first demo.
But even if we had installed `sudo` into the container and tried to run this command with it, or change to root using `su`, we would still find it impossible to elevate our privileges within the container:
106
+
The `sudo` command is not found. But even if we had installed `sudo` into the
107
+
container and tried to run this command with it, or change to root using `su`,
108
+
we would still find it impossible to elevate our privileges within the
109
+
container:
101
110
102
111
```
103
-
Singularity:~> sudo apt-get update
112
+
Singularity> sudo apt-get update
104
113
sudo: effective uid is not 0, is /usr/bin/sudo on a file system with the 'nosuid' option set or an NFS file system without root privileges?
105
114
```
106
115
@@ -109,41 +118,36 @@ Once again, this is an important concept in Singularity. If you enter a contain
109
118
Let's exit the container and re-enter as root.
110
119
111
120
```
112
-
Singularity lolcow:~> exit
121
+
Singularity> exit
113
122
114
123
$ sudo singularity shell --writable lolcow
115
124
```
116
125
117
126
Now we are the root user inside the container. Note also the addition of the `--writable` option. This option allows us to modify the container. The changes will actually be saved into the container and will persist across uses.
perl: warning: Please check that your locale settings:
142
-
LANGUAGE = (unset),
143
-
LC_ALL = (unset),
144
-
LANG = "en_US.UTF-8"
145
-
are supported and installed on your system.
146
-
perl: warning: Falling back to the standard locale ("C").
147
151
________________________________________
148
152
/ Keep emotionally active. Cater to your \
149
153
\ favorite neurosis. /
@@ -155,38 +159,32 @@ perl: warning: Falling back to the standard locale ("C").
155
159
|| ||
156
160
```
157
161
158
-
We're making progress, but we are now receiving a warning from perl. However, before we tackle that, let's think some more about the `$PATH` variable.
162
+
Great! Things are working properly now.
159
163
160
-
We changed our path in this session, but those changes will disappear as soon as we exit the container just like they will when you exit any other shell. To make the changes permanent we should add them to the definition file and re-bootstrap the container. We'll do that in a minute.
164
+
---
165
+
**NOTE**
161
166
162
-
Now back to our perl warning. Perl is complaining that the locale is not set properly. Basically, perl wants to know where you are and what sort of language encoding it should use. Should you encounter this warning you can probably fix it with the `locale-gen` command or by setting `LC_ALL=C`. Here we'll just set the environment variable.
167
+
If you receive warnings from the Perl language about the `locale` being incorrect, you can usually fix them with `export LC_ALL=C`.
163
168
164
-
```
165
-
Singularity lolcow:~> export LC_ALL=C
169
+
---
166
170
167
-
Singularity lolcow:~> fortune | cowsay | lolcat
168
-
_________________________________________
169
-
/ FORTUNE PROVIDES QUESTIONS FOR THE \
170
-
| GREAT ANSWERS: #19 A: To be or not to |
171
-
\ be. Q: What is the square root of 4b^2? /
172
-
-----------------------------------------
173
-
\ ^__^
174
-
\ (oo)\_______
175
-
(__)\ )\/\
176
-
||----w |
177
-
|| ||
178
-
```
171
+
---
172
+
**NOTE**
179
173
180
-
Great! Things are working properly now.
174
+
We changed our path in this session, but those changes will disappear as soon as we exit the container just like they will when you exit any other shell. To make the changes permanent we should add them to the definition file and re-bootstrap the container. We'll do that in a minute.
175
+
176
+
## Building the final production-grade SIF file
177
+
178
+
---
181
179
182
-
Although it is fine to shell into your Singularity container and make changes while you are debugging, you ultimately want all of these changes to be reflected in your recipe file. Otherwise if you need to reproduce it from scratch you will forget all of the changes you made.
180
+
Although it is fine to shell into your Singularity container and make changes while you are debugging, you ultimately want all of these changes to be reflected in your definition file. Otherwise if you need to reproduce it from scratch you will forget all of the changes you made. You will also want to rebuild you container into something more durable and robust than a directory.
183
181
184
182
Let's update our definition file with the changes we made to this container.
185
183
186
184
```
187
-
Singularity lolcow:~> exit
185
+
Singularity> exit
188
186
189
-
$ nano Singularity
187
+
$ nano lolcow.def
190
188
```
191
189
192
190
Here is what our updated definition file should look like.
Let's rebuild the container with the new definition file.
214
210
215
211
```
216
-
$ sudo singularity build lolcow.simg Singularity
212
+
$ sudo singularity build lolcow.sif lolcow.def
217
213
```
218
214
219
-
Note that we changed the name of the container. By omitting the `--sandbox` option, we are building our container in the standard Singularity squashfs file format. We are denoting the file format with the (optional) `.simg` extension. A squashfs file is compressed and immutable making it a good choice for a production environment.
215
+
Note that we changed the name of the container. By omitting the `--sandbox` option, we are building our container in the standard Singularity file format (SIF). We are denoting the file format with the (optional) `.sif` extension. A SIF file is compressed and immutable making it a good choice for a production environment.
220
216
221
-
Singularity stores a lot of [useful metadata](http://singularity.lbl.gov/docs-environment-metadata). For instance, if you want to see the recipe file that was used to create the container you can use the `inspect` command like so:
217
+
Singularity stores a lot of [useful metadata](https://sylabs.io/guides/3.5/user-guide/environment_and_metadata.html#container-metadata). For instance, if you want to see the definition file that was used to create the container you can use the `inspect` command like so:
Copy file name to clipboardExpand all lines: README.md
+7-7Lines changed: 7 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,14 +1,14 @@
1
1
# <b>Creating and running software containers with Singularity</b>
2
-
### <i>How to use [Singularity](http://singularity.lbl.gov)! </i>
2
+
### <i>How to use [Singularity](https://sylabs.io/guides/latest/user-guide/)! </i>
3
3
4
4
This is an introductory workshop on Singularity. It was originally taught by David Godlove at the [NIH HPC](https://hpc.nih.gov/), but the content has since been adapted to a general audience. For more information about the topics covered here, see the following:
## What IS a software container anyway? (And what's it good for?)
@@ -42,7 +42,7 @@ Containers and VMs are both types of virtualization. But it's important to unde
42
42
43
43
Because of their differences, VMs and containers serve different purposes and should be favored under different circumstances.
44
44
- VMs are good for long running interactive sessions where you may want to use several different applications. (Checking email on Outlook and using Microsoft Word and Excel).
45
-
- Containers are better suited to running one or two applicationsnon-interactively in their own custom environments.
45
+
- Containers are better suited to running one or two applications, often non-interactively, in their own custom environments.
46
46
47
47
## Docker
48
48
@@ -73,7 +73,7 @@ See https://docs.docker.com/engine/security/security/#docker-daemon-attack-surfa
73
73
- Not architected with security in mind
74
74
- Not built for HPC (but good for cloud)
75
75
76
-
Docker shines for DevOPs teams providing cloud-hosted micro-services to users.
76
+
Docker shines for DevOPs teams providing cloud-native micro-services to users.
77
77
78
78
## Singularity
79
79
@@ -88,8 +88,8 @@ Singularity assumes that you will have a build system where you are the root use
88
88
- Easy to learn and use (relatively speaking)
89
89
- Approved for HPC ([installed on some of the biggest HPC systems in the world](http://singularity.lbl.gov/citation-registration#clusters))
90
90
- Can convert Docker containers to Singularity and run containers directly from Docker Hub
91
-
-[Singularity Hub](https://singularity-hub.org/)!
92
-
- A place to build and host your containers similar to Docker Hub
0 commit comments