Skip to content

Commit a703960

Browse files
GodloveDGodloveD
committed
Rearranging content to prioritize _using_ pre-built containers before
_building_ containers. This follows a general shift in priority within Singularity and associated docs and ecosystem. It is now relatively common to find "consumers" of containers who never build a container themselves. I think the flow of the class is better too.
1 parent 03e487b commit a703960

File tree

4 files changed

+263
-124
lines changed

4 files changed

+263
-124
lines changed

‎00-installation/README.md‎

Lines changed: 4 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -70,33 +70,13 @@ Typing `singularity help <command>` will give you more detailed information abou
7070
You can test your installation like so:
7171

7272
```
73-
$ singularity run docker://godlovedc/lolcow
73+
$ singularity run library://godlovedc/funny/lolcow
7474
```
7575

7676
You should see something like the following.
7777

7878
```
79-
INFO: Converting OCI blobs to SIF format
80-
INFO: Starting build...
81-
Getting image source signatures
82-
Copying blob 9fb6c798fa41 done
83-
Copying blob 3b61febd4aef done
84-
Copying blob 9d99b9777eb0 done
85-
Copying blob d010c8cf75d7 done
86-
Copying blob 7fac07fb303e done
87-
Copying blob 8e860504ff1e done
88-
Copying config 73d5b1025f done
89-
Writing manifest to image destination
90-
Storing signatures
91-
2020/02/25 23:58:06 info unpack layer: sha256:9fb6c798fa41e509b58bccc5c29654c3ff4648b608f5daa67c1aab6a7d02c118
92-
2020/02/25 23:58:06 warn rootless{dev/agpgart} creating empty file in place of device 10:175
93-
[snip...]
94-
2020/02/25 23:58:09 info unpack layer: sha256:3b61febd4aefe982e0cb9c696d415137384d1a01052b50a85aae46439e15e49a
95-
2020/02/25 23:58:09 info unpack layer: sha256:9d99b9777eb02b8943c0e72d7a7baec5c782f8fd976825c9d3fb48b3101aacc2
96-
2020/02/25 23:58:09 info unpack layer: sha256:d010c8cf75d7eb5d2504d5ffa0d19696e8d745a457dd8d28ec6dd41d3763617e
97-
2020/02/25 23:58:09 info unpack layer: sha256:7fac07fb303e0589b9c23e6f49d5dc1ff9d6f3c8c88cabe768b430bdb47f03a9
98-
2020/02/25 23:58:09 info unpack layer: sha256:8e860504ff1ee5dc7953672d128ce1e4aa4d8e3716eb39fe710b849c64b20945
99-
INFO: Creating SIF file...
79+
INFO: Downloading library image
10080
_______________________________________
10181
/ Excellent day for putting Slinkies on \
10282
\ an escalator. /
@@ -110,6 +90,6 @@ You should see something like the following.
11090

11191
Your cow will likely say something different (and be more colorful), but as long as you see a cow your installation is working properly.
11292

113-
This command downloads, converts, and runs a container from [Docker Hub](https://hub.docker.com/r/godlovedc/lolcow/). You will see a lot of warnings that have to do with the fact that you've created this container without using root privileges. Pay them no mind.
93+
This command downloads and "runs" a container from [Singularity Container Library](https://cloud.sylabs.io/library). (We'll be talking more about what it means to "run" a container later on in the class.)
11494

115-
In the next exercise, we will learn how to build a similar container from scratch.
95+
In a following exercise, we will learn how to build a similar container from scratch. But right now, we are going to use this container to execute a bunch of basic commands and just get a feel for what it's like to use Singularity.

‎01-basic-usage/README.md‎

Lines changed: 257 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,257 @@
1+
# Downloading and interacting with containers
2+
3+
This section will be useful for container consumers. (i.e. those who really just want to use containers somebody else built.) The next chapter will explore topics more geared toward container producers (i.e. those who want/need to build containers from scratch).
4+
5+
You can find pre-built containers in lots of places. Singularity can convert and run containers in many different formats, including those built by Docker.
6+
7+
In this class, we'll be using containers from:
8+
9+
- [The Singularity Container Library](https://cloud.sylabs.io/library), developed and maintained by [Sylabs](https://sylabs.io/)
10+
- [Docker Hub](https://hub.docker.com/), developed and maintained by [Docker](https://www.docker.com/)
11+
12+
There are lots of other places to find pre-build containers too. Here are some of the more popular ones:
13+
14+
- [Singularity Hub](https://singularity-hub.org/), an early collaboration between Stanford University and the Singularity community
15+
- [Quay.io](quay.io), developed and maintained by Red Hat
16+
- [NGC](https://ngc.nvidia.com/catalog/all?orderBy=modifiedDESC&pageNumber=3&query=&quickFilter=&filters=), developed and maintained by NVIDIA
17+
- [BioContainers](https://biocontainers.pro/#/registry), develped and maintained by the Bioconda group
18+
- Cloud providers like Amazon AWS, Microsoft Azure, and Google cloud also have container registries that can work with Singularity
19+
20+
## Downloading containers
21+
22+
In the last section, we validated our Singularity installation by "running" a container from the Container Library. Let's download that container using the `pull` command.
23+
24+
```
25+
$ singularity pull library://godlovedc/funny/lolcow
26+
```
27+
28+
You'll see a warning about running `singularity verify` to make sure that the container is trusted. We'll talk more about that later.
29+
30+
For now, notice that you have a new file in your current working directory called `lolcow_latest.sif`
31+
32+
```
33+
$ ls lolcow_latest.sif
34+
lolcow_latest.sif
35+
```
36+
37+
This is your container. Or more precisely, it is a Singularity Image Format (SIF) file containing an image of a root level filesystem. This image is mounted to your host filesystem (in a new "mount namespace") and then entered when you run a Singularity command.
38+
39+
Note that you can download the Docker version of this same container from Docker Hub with the following command:
40+
41+
```
42+
$ singularity pull docker://godlovedc/lolcow
43+
```
44+
45+
Doing so may produce an error if the container already exists.
46+
47+
## Entering containers with `shell`
48+
49+
Now let's enter our new container and look around. We can do so with the `shell` command.
50+
51+
```
52+
$ singularity shell lolcow_latest.sif
53+
```
54+
55+
Depending on the environment of your host system you may see your shell prompt change. Let's look at what OS is running inside the container.
56+
57+
```
58+
$ cat /etc/os-release
59+
NAME="Ubuntu"
60+
VERSION="16.04.5 LTS (Xenial Xerus)"
61+
ID=ubuntu
62+
ID_LIKE=debian
63+
PRETTY_NAME="Ubuntu 16.04.5 LTS"
64+
VERSION_ID="16.04"
65+
HOME_URL="http://www.ubuntu.com/"
66+
SUPPORT_URL="http://help.ubuntu.com/"
67+
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
68+
VERSION_CODENAME=xenial
69+
UBUNTU_CODENAME=xenial
70+
```
71+
72+
No matter what OS is running on your host, your container is running Ubuntu 16.04 (Xenial Xerus)!
73+
74+
75+
Let's try a few more commands:
76+
77+
```
78+
Singularity> whoami
79+
dave
80+
81+
Singularity> hostname
82+
hal-9000
83+
```
84+
85+
This is one of the core features of Singularity that makes it so attractive from a security and usability standpoint. The user remains the same inside and outside of the container.
86+
87+
Regardless of whether or not the program `cowsay` is installed on your host system, you have access to it now because it is installed inside of the container:
88+
89+
```
90+
Singularity> which cowsay
91+
/usr/games/cowsay
92+
93+
Singularity> cowsay moo
94+
_____
95+
< moo >
96+
-----
97+
\ ^__^
98+
\ (oo)\_______
99+
(__)\ )\/\
100+
||----w |
101+
|| ||
102+
```
103+
104+
We'll be getting a lot of mileage out of this silly little program as we explore Linux containers.
105+
106+
This is the command that is executed when the container actually "runs":
107+
108+
```
109+
Singularity> fortune | cowsay | lolcat
110+
____________________________________
111+
/ A horse! A horse! My kingdom for a \
112+
| horse! |
113+
| |
114+
\ -- Wm. Shakespeare, "Richard III" /
115+
------------------------------------
116+
\ ^__^
117+
\ (oo)\_______
118+
(__)\ )\/\
119+
||----w |
120+
|| ||
121+
```
122+
123+
More on "running" the container in a minute. For now, don't forget to `exit` the container when you are finished playing!
124+
125+
```
126+
Singularity> exit
127+
exit
128+
```
129+
130+
## Executing containerized commands with `exec`
131+
132+
Using the `exec` command, we can run commands within the container from the host system.
133+
134+
```
135+
$ singularity exec lolcow_latest.sif cowsay 'How did you get out of the container?'
136+
_______________________________________
137+
< How did you get out of the container? >
138+
---------------------------------------
139+
\ ^__^
140+
\ (oo)\_______
141+
(__)\ )\/\
142+
||----w |
143+
|| ||
144+
```
145+
146+
In this example, singularity entered the container, ran the `cowsay` command with supplied arguments, displayed the standard output on our host system terminal, and then exited.
147+
148+
## "Running" a container with (and without) `run`
149+
150+
As mentioned several times you can "run" a container like so:
151+
152+
```
153+
$ singularity run lolcow_latest.sif
154+
_________________________________________
155+
/ Q: How many Bell Labs Vice Presidents \
156+
| does it take to change a light bulb? A: |
157+
| That's proprietary information. Answer |
158+
| available from AT&T on payment of |
159+
\ license fee (binary only). /
160+
-----------------------------------------
161+
\ ^__^
162+
\ (oo)\_______
163+
(__)\ )\/\
164+
||----w |
165+
|| ||
166+
```
167+
168+
So what actually happens when you run a container? There is a special file within the container called a `runscript` that is executed when a container is run. You can see this (and other meta-data about the container) using the inspect command.
169+
170+
```
171+
$ singularity inspect --runscript lolcow_latest.sif
172+
#!/bin/sh
173+
174+
fortune | cowsay | lolcat
175+
```
176+
177+
In this case the `runscript` consists of three simple commands with the output of each command piped to the subsequent command.
178+
179+
Because Singularity containers have pre-defined actions that they must carry out when run, they are actually executable. Note the default permissions when you download or build a container:
180+
181+
```
182+
$ ls -l lolcow_latest.sif
183+
-rwxr-xr-x 1 student student 93574075 Feb 28 23:02 lolcow_latest.sif
184+
```
185+
186+
This allows you to run execute a container like so:
187+
188+
```
189+
$ ./lolcow_latest.sif
190+
________________________________________
191+
/ It is by the fortune of God that, in \
192+
| this country, we have three benefits: |
193+
| freedom of speech, freedom of thought, |
194+
| and the wisdom never to use either. |
195+
| |
196+
\ -- Mark Twain /
197+
----------------------------------------
198+
\ ^__^
199+
\ (oo)\_______
200+
(__)\ )\/\
201+
||----w |
202+
|| ||
203+
```
204+
As we shall see later, this nifty trick can makes it easy to forget your applications are containerized and just run them like any old program.
205+
206+
## Pipes and redirection
207+
208+
Singularity does not try to isolate your container completely from the host system. This allows you to do some interesting things. For instance, you can use pipes and redirection to blur the lines between the container and the host system.
209+
210+
```
211+
$ singularity exec lolcow_latest.sif cowsay moo > cowsaid
212+
213+
$ cat cowsaid
214+
_____
215+
< moo >
216+
-----
217+
\ ^__^
218+
\ (oo)\_______
219+
(__)\ )\/\
220+
||----w |
221+
|| ||
222+
```
223+
224+
We created a file called `cowsaid` in the current working directory with the output of a command that was executed within the container. >_shock and awe_
225+
226+
We can also pipe things _into_ the container (and that is very tricky).
227+
228+
```
229+
$ cat cowsaid | singularity exec lolcow_latest.sif cowsay -n
230+
______________________________
231+
/ _____ \
232+
| < moo > |
233+
| ----- |
234+
| \ ^__^ |
235+
| \ (oo)\_______ |
236+
| (__)\ )\/\ |
237+
| ||----w | |
238+
\ || || /
239+
------------------------------
240+
\ ^__^
241+
\ (oo)\_______
242+
(__)\ )\/\
243+
||----w |
244+
|| ||
245+
```
246+
247+
We've created a meta-cow (a cow that talks about cows). ;-P
248+
249+
So pipes and redirects work as expected between a container and the host system. If, however, you need to pipe the output of one command in your container to another command in your container, things are slightly more complicated. Pipes and redirects are shell constructs, so if you don't want your host shell to interpret them, you have to hide them from it.
250+
251+
```
252+
$ singularity exec lolcow_latest.sif sh -c "fortune | cowsay | lolcat"
253+
```
254+
255+
The above invokes a new shell, but inside the container, and tells it to run the single command line `fortune | cowsay | lolcat`.
256+
257+
That covers the basics on how to download and use pre-built containers! In the next section we'll start learning how to build your own containers.

01-building/README.md renamed to 02-building/README.md

Lines changed: 2 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -57,42 +57,15 @@ But if you want to shell into a container and tinker with it (like we will do he
5757

5858
When your build finishes, you will have a basic Debian container saved in a local directory called `lolcow`.
5959

60-
## Using `shell` to explore and modify containers
60+
## Using `shell --writable` to explore and modify containers
6161

6262
Now let's enter our new container and look around.
6363

6464
```
6565
$ singularity shell lolcow
6666
```
6767

68-
Depending on the environment on your host system you may see your prompt change. Let's look at what OS is running inside the container.
69-
70-
```
71-
Singularity lolcow:~> cat /etc/os-release
72-
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
73-
NAME="Debian GNU/Linux"
74-
VERSION_ID="10"
75-
VERSION="10 (buster)"
76-
VERSION_CODENAME=buster
77-
ID=debian
78-
HOME_URL="https://www.debian.org/"
79-
SUPPORT_URL="https://www.debian.org/support"
80-
BUG_REPORT_URL="https://bugs.debian.org/"
81-
```
82-
83-
No matter what OS is running on your host, your container is running Debian Stable!
84-
85-
Let's try a few more commands:
86-
87-
```
88-
Singularity> whoami
89-
dave
90-
91-
Singularity> hostname
92-
hal-9000
93-
```
94-
95-
This is one of the core features of Singularity that makes it so attractive from a security and usability standpoint. The user remains the same inside and outside of the container.
68+
Depending on the environment on your host system you may see your prompt change.
9669

9770
Let's try installing some software. I used the programs `fortune`, `cowsay`, and `lolcat` to produce the container that we saw in the first demo.
9871

0 commit comments

Comments
 (0)