modularize the tutorial

inspired by @wresch

Author: 0xaf1f

00-installation/README.md

@@ -0,0 +1,78 @@
+### Install
+Here we will install the latest tagged release from [GitHub](https://github.com/singularityware/singularity). If you prefer to install a different version or to install Singularity in a different location, see these [Singularity docs](http://singularity.lbl.gov/docs-installation).
+
+We're going to compile Singularity from source code.  First we'll need to make sure we have some development tools installed so that we can do that.  On Ubuntu, run these commands to make sure you have all the necessary packages installed.
+
+```
+$ sudo apt-get update
+
+$ sudo apt-get -y install python build-essential debootstrap squashfs-tools
+```
+
+On CentOS, these commmands should get you up to speed.
+
+```
+$ sudo yum update 
+
+$ sudo yum groupinstall 'Development Tools'
+
+$ sudo yum install wget epel-release
+
+$ sudo yum install debootstrap.noarch squashfs-tools
+```
+
+Next we'll download a compressed archive of the source code (using the the `wget` command). Then we'll extract the source code from the archive (with the `tar` command).
+
+```
+$ wget https://github.com/singularityware/singularity/releases/download/2.4.2/singularity-2.4.2.tar.gz
+
+$ tar -xf singularity-2.4.2.tar.gz
+```
+
+Finally it's time to build and install!
+
+```
+$ cd singularity-2.4.2
+
+$ ./configure --prefix=/usr/local
+
+$ make 
+
+$ sudo make install
+```
+
+If you want support for tab completion of Singularity commands, you need to source the appropriate file and add it to the bash completion directory in `/etc` so that it will be sourced automatically when you start another shell.
+
+```
+$ . etc/bash_completion.d/singularity
+
+$ sudo cp etc/bash_completion.d/singularity /etc/bash_completion.d/
+```
+
+If everything went according to plan, you now have a working installation of Singularity.  You can test your installation like so:
+
+```
+$ singularity run docker://godlovedc/lolcow
+```
+
+You should see something like the following.
+
+```
+Docker image path: index.docker.io/godlovedc/lolcow:latest
+Cache folder set to /home/ubuntu/.singularity/docker
+[6/6] |===================================| 100.0%
+Creating container runtime...
+ _______________________________________
+/ Excellent day for putting Slinkies on \
+\ an escalator.                         /
+ ---------------------------------------
+        \   ^__^
+         \  (oo)\_______
+            (__)\       )\/\
+                ||----w |
+                ||     ||
+```
+
+Your cow will likely say something different (and be more colorful), but as long as you see a cow your installation is working properly.  
+
+This command downloads and runs a container from [Docker Hub](https://hub.docker.com/r/godlovedc/lolcow/).  During the next hour we will learn how to build a similar container from scratch.  

01-building/README.md

@@ -0,0 +1,249 @@
+## Hour 2 (Building and Running Containers)
+
+In the second hour we will build the preceding container from scratch. 
+
+Simply typing `singularity` will give you an summary of all the commands you can use. Typing `singularity help <command>` will give you more detailed information about running an individual command.
+
+### Building a basic container
+
+To build a singularity container, you must use the `build` command.  The `build` command installs an OS, sets up your container's environment and installs the apps you need.  To use the `build` command, we need a **recipe file** (also called a definition file). A Singularity recipe file is a set of instructions telling Singularity what software to install in the container.
+
+The Singularity source code contains several example definition files in the `/examples` subdirectory.  Let's copy the ubuntu example to our home directory and inspect it.
+
+```
+$ mkdir ../lolcow
+
+$ cp examples/ubuntu/Singularity ../lolcow/
+
+$ cd ../lolcow
+
+$ nano Singularity
+```
+
+Edit it until it looks like this:
+
+```
+BootStrap: debootstrap
+OSVersion: xenial
+MirrorURL: http://us.archive.ubuntu.com/ubuntu/
+
+
+%runscript
+    echo "This is what happens when you run the container..."
+
+
+%post
+    echo "Hello from inside the container"
+    sed -i 's/$/ universe/' /etc/apt/sources.list
+    apt-get update
+    apt-get -y install vim
+    apt-get clean
+
+```
+
+See the [Singularity docs](http://singularity.lbl.gov/docs-recipes) for an explanation of each of these sections.
+
+Now let's use this recipe file as a starting point to build our `lolcow.img` container. Note that the build command requires `sudo` privileges, when used in combination with a recipe file. 
+
+```
+$ sudo singularity build --sandbox lolcow Singularity
+```
+
+The `--sandbox` option in the command above tells Singularity that we want to build a special type of container for development purposes.  
+
+Singularity can build containers in several different file formats. The default is to build a [squashfs](https://en.wikipedia.org/wiki/SquashFS) image. The squashfs format is compressed and immutable making it a good choice for reproducible, production-grade containers.  
+
+But if you want to shell into a container and tinker with it (like we will do here), you should build a sandbox (which is really just a directory).  This is great when you are still developing your container and don't yet know what should be included in the recipe file.  
+
+When your build finishes, you will have a basic Ubuntu container saved in a local directory called `lolcow`.
+
+### Using `shell` to explore and modify containers
+
+Now let's enter our new container and look around.  
+
+```
+$ singularity shell lolcow
+```
+
+Depending on the environment on your host system you may see your prompt change. Let's look at what OS is running inside the container.
+
+```
+Singularity lolcow:~> cat /etc/os-release
+NAME="Ubuntu"
+VERSION="16.04 LTS (Xenial Xerus)"
+ID=ubuntu
+ID_LIKE=debian
+PRETTY_NAME="Ubuntu 16.04 LTS"
+VERSION_ID="16.04"
+HOME_URL="http://www.ubuntu.com/"
+SUPPORT_URL="http://help.ubuntu.com/"
+BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
+UBUNTU_CODENAME=xenial
+```
+
+No matter what OS is running on your host, your container is running Ubuntu 16.04!
+
+Let's try a few more commands:
+
+```
+Singularity lolcow:~> whoami
+dave
+
+Singularity lolcow:~> hostname
+hal-9000
+```
+
+This is one of the core features of Singularity that makes it so attractive from a security standpoint.  The user remains the same inside and outside of the container. 
+
+Let's try installing some software. I used the programs `fortune`, `cowsay`, and `lolcat` to produce the container that we saw in the first demo.
+
+```
+Singularity lolcow:~> sudo apt-get update && sudo apt-get -y install fortune cowsay lolcat
+bash: sudo: command not found
+```
+
+Whoops!
+
+Singularity complains that it can't find the `sudo` command.  But even if you try to install `sudo` or change to root using `su`, you will find it impossible to elevate your privileges within the container.  
+
+Once again, this is an important concept in Singularity.  If you enter a container without root privileges, you are unable to obtain root privileges within the container.  This insurance against privilege escalation is the reason that you will find Singularity installed in so many HPC environments.  
+
+Let's exit the container and re-enter as root.
+
+```
+Singularity lolcow:~> exit
+
+$ sudo singularity shell --writable lolcow
+```
+
+Now we are the root user inside the container. Note also the addition of the `--writable` option.  This option allows us to modify the container.  The changes will actually be saved into the container and will persist across uses. 
+
+Let's try installing some software again.
+
+```
+Singularity lolcow:~> apt-get update && apt-get -y install fortune cowsay lolcat
+```
+
+Now you should see the programs successfully installed.  Let's try running the demo in this new container.
+
+```
+Singularity lolcow:~> fortune | cowsay | lolcat
+bash: lolcat: command not found
+bash: cowsay: command not found
+bash: fortune: command not found
+```
+
+Drat! It looks like the programs were not added to our `$PATH`.  Let's add them and try again.
+
+```
+Singularity lolcow:~> export PATH=/usr/games:$PATH
+
+Singularity lolcow:~> fortune | cowsay | lolcat
+perl: warning: Setting locale failed.
+perl: warning: Please check that your locale settings:
+        LANGUAGE = (unset),
+        LC_ALL = (unset),
+        LANG = "en_US.UTF-8"
+    are supported and installed on your system.
+perl: warning: Falling back to the standard locale ("C").
+ ________________________________________
+/ Keep emotionally active. Cater to your \
+\ favorite neurosis.                     /
+ ----------------------------------------
+        \   ^__^
+         \  (oo)\_______
+            (__)\       )\/\
+                ||----w |
+                ||     ||
+```
+
+We're making progress, but we are now receiving a warning from perl.  However, before we tackle that, let's think some more about the `$PATH` variable.
+
+We changed our path in this session, but those changes will disappear as soon as we exit the container just like they will when you exit any other shell.  To make the changes permanent we should add them to the definition file and re-bootstrap the container.  We'll do that in a minute.
+
+Now back to our perl warning.  Perl is complaining that the locale is not set properly.  Basically, perl wants to know where you are and what sort of language encoding it should use.  Should you encounter this warning you can  probably fix it with the `locale-gen` command or by setting `LC_ALL=C`.  Here we'll just set the environment variable.
+
+```
+Singularity lolcow:~> export LC_ALL=C
+
+Singularity lolcow:~> fortune | cowsay | lolcat
+ _________________________________________
+/ FORTUNE PROVIDES QUESTIONS FOR THE      \
+| GREAT ANSWERS: #19 A: To be or not to   |
+\ be. Q: What is the square root of 4b^2? /
+ -----------------------------------------
+        \   ^__^
+         \  (oo)\_______
+            (__)\       )\/\
+                ||----w |
+                ||     ||
+```
+
+Great!  Things are working properly now.  
+
+Although it is fine to shell into your Singularity container and make changes while you are debugging, you ultimately want all of these changes to be reflected in your recipe file.  Otherwise if you need to reproduce it from scratch you will forget all of the changes you made.
+
+Let's update our definition file with the changes we made to this container.
+
+```
+Singularity lolcow:~> exit
+
+$ nano Singularity
+```
+
+Here is what our updated definition file should look like.
+
+```
+BootStrap: debootstrap
+OSVersion: xenial
+MirrorURL: http://us.archive.ubuntu.com/ubuntu/
+
+
+%runscript
+    echo "This is what happens when you run the container..."
+
+
+%post
+    echo "Hello from inside the container"
+    sed -i 's/$/ universe/' /etc/apt/sources.list
+    apt-get update
+    apt-get -y install fortune cowsay lolcat
+    apt-get clean
+
+%environment
+    export PATH=/usr/games:$PATH
+    export LC_ALL=C
+```
+
+Let's rebuild the container with the new definition file.
+
+```
+$ sudo singularity build lolcow.simg Singularity
+```
+
+Note that we changed the name of the container.  By omitting the `--sandbox` option, we are building our container in the standard Singularity squashfs file format.  We are denoting the file format with the (optional) `.simg` extension.  A squashfs file is compressed and immutable making it a good choice for a production environment.
+
+Singularity stores a lot of [useful metadata](http://singularity.lbl.gov/docs-environment-metadata).  For instance, if you want to see the recipe file that was used to create the container you can use the `inspect` command like so:
+
+```
+$ singularity inspect --deffile lolcow.simg
+BootStrap: debootstrap
+OSVersion: xenial
+MirrorURL: http://us.archive.ubuntu.com/ubuntu/
+
+
+%runscript
+    echo "This is what happens when you run the container..."
+
+
+%post
+    echo "Hello from inside the container"
+    sed -i 's/$/ universe/' /etc/apt/sources.list
+    apt-get update
+    apt-get -y install fortune cowsay lolcat
+    apt-get clean
+
+%environment
+    export PATH=/usr/games:$PATH
+    export LC_ALL=C
+```

02-io/README.md

@@ -0,0 +1,60 @@
+### Blurring the line between the container and the host system.
+
+Singularity does not try to isolate your container completely from the host system.  This allows you to do some interesting things.
+
+Using the exec command, we can run commands within the container from the host system.  
+
+```
+$ singularity exec lolcow.simg cowsay 'How did you get out of the container?'
+ _______________________________________
+< How did you get out of the container? >
+ ---------------------------------------
+        \   ^__^
+         \  (oo)\_______
+            (__)\       )\/\
+                ||----w |
+                ||     ||
+```
+
+In this example, singularity entered the container, ran the `cowsay` command, displayed the standard output on our host system terminal, and then exited. 
+
+You can also use pipes and redirection to blur the lines between the container and the host system.  
+
+```
+$ singularity exec lolcow.simg cowsay moo > cowsaid
+
+$ cat cowsaid
+ _____
+< moo >
+ -----
+        \   ^__^
+         \  (oo)\_______
+            (__)\       )\/\
+                ||----w |
+                ||     ||
+```
+
+We created a file called `cowsaid` in the current working directory with the output of a command that was executed within the container. 
+
+We can also pipe things _into_ the container.
+
+```
+$ cat cowsaid | singularity exec lolcow.simg cowsay -n
+ ______________________________
+/  _____                       \
+| < moo >                      |
+|  -----                       |
+|         \   ^__^             |
+|          \  (oo)\_______     |
+|             (__)\       )\/\ |
+|                 ||----w |    |
+\                 ||     ||    /
+ ------------------------------
+        \   ^__^
+         \  (oo)\_______
+            (__)\       )\/\
+                ||----w |
+                ||     ||
+```
+
+We've created a meta-cow (a cow that talks about cows). :stuck_out_tongue_winking_eye: