RFC: Support Official Published Database Images

[msmith-techempower]

Summary

• Formalize the database key in config.toml files to specify a published database image and tag
• Excise the Docker library portions of the Toolset to its own library repository
• Move configuration of a database into the Docker container start-up
• Formalize start_up interface for all database types
• Formalize build_up interface for all database types
• Formalize is_ready interface for all database types
• Formalize communication of the running container_id back to the Toolset

Motivation

Currently (and also in the legacy implementation), the database field of the configuration file specified one of a list of supported databases that were curated by the TechEmpower Framework Benchmarks team and community contributors. This implementation necessitates being able to build a database's Docker image dynamically, or the image being published by an external process and pulled at runtime.

This database key could technically be changed to allow specifying the published base image from which the database image will be built. For instance, a configuration could specify database = "mysql:5.7", and the Toolset et al could support pulling that published image from Dockerhub, configuring it, starting the container, connecting with a client, and issuing the build-up scripts for establishing the ready state of that database (creating the user/password, schema, tables, inserting data, etc).

Detailed Design

The proposed change would cause a verification of any test to become the following:

1. Start via tfb_toolset.exe -m verify --test gemini --type db
2. Parse frameworks/Java/gemini/config.toml and retrieve the database value of mysql:5.7
3. Toolset starts the verifier container in a "create database" mode, passing mysql:5.7 as an environment variable, and blocks until it terminates
4. Verifier runs in "create database" mode, splits the database into image_name and image_tag
   4a. Validates that image_name is a known database with a verifier implementation
   4b. Issues a Docker create image from "[image_name]:[image_tag]"
5. Verifier starts the database, optionally mounting configuration files
6. Verifier, via database interface, checks is_ready for a period of time
7. Verifier communicates container_id once database is ready back to Toolset which asynchronously stores the id
8. Verifier, via database interface, executes the build_up stage of the database
9. Verifier terminates
10. Toolset verifies

Benefits

• Use of officially published database images as opposed to our best-attempt images
• Individual tests must specify the exact database type and version allowing testing different databases over time in a controlled and visible way
• build_up is moved from creation scripts on the image to client scripts on the Verifier
• One source of truth for a given database type - the mysql folder under the Verifier is where all database about the mysql database type exists (read: no external repository; if the mysql folder is deleted, it is no longer supported; if a couchdb folder is added and the interfaces implemented, it is supported)

Drawbacks

• Database configuration files must live in an otherwise Rust repository
• Additional test time required to connect the client to the database instance and build_up on every invocation
• While database type and version are specified by the implementer, the configuration file is static to the Verifier

Alternatives

• Maintain an additional repository with folders for each database type (similar to how it is done in Legacy) and a single Dockerfile for each, publishing to Dockerhub under the TechEmpower repository for consumption by the Toolset.

Unresolved Questions

• Do all database types support configurations mounted as a volume? See Using a custom MySQL configuration file.
• Is there a way to have "versioned" configuration files other than having X files in the Verifier?