Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
171 lines (118 loc) · 5.71 KB

CONTRIBUTING.md

File metadata and controls

171 lines (118 loc) · 5.71 KB

Golem development

Prerequisites

To work on Golem you need to install (via package installers or manually) the following tools:

OSX

brew install rustup protobuf redis docker lnav nginx

Rust Installation

# latest **stable** rust compiler
rustup update stable
rustup default stable
rustup target add wasm32-wasip2
cargo install --force cargo-make

Everything else is managed by cargo-make.

Development workflow

Building

To compile everything use

cargo make build

It is recommended to do a full build before starting working on Golem and opening it with an IDE. During development it is usually enough to recompile only the crate you are working on, for example:

cargo build -p golem-worker-service-base

If cargo runs out of memory

Depending on the number of CPU cores and available memory, building everything can use a lot of memory. If cargo runs out of memory or just uses too much, you can limit the number of parallel jobs by providing a cargo config.toml file, for example:

[build]
jobs = 4

in ~/.cargo/config.toml. For other options and possibilities check the cargo documentation.

Running tests

Tests are using the test-r library.

During development you can run the involved tests in the usual ways: from the IDE, or using cargo test command selecting a specific crate and test module, for example:

cargo test -p golem-worker-executor api::promise -- --report-time

Running all unit tests

To run all unit tests use

cargo make unit-tests

Running all worker executor tests

The worker executor tests are testing the Golem Worker Executor standalone without any of the other services. These tests require redis. To run all of them use

cargo make worker-executor-tests

As there are many of these tests, they are organized into groups that are executed in parallel on CI. You can run only a specific group with cargo make, for example:

cargo make worker-executor-tests-group1

Running all integration tests

The integration tests are starting up several Golem services and testing them together. These tests also require docker and redis to be available.

To run all integration tests use

cargo make integration-tests

Running all the CLI tests

The CLI tests are similar to the integration tests but interact with the running services only through the Golem CLI application.

To run all CLI tests use

cargo make cli-tests

Running sharding tests

For sharding-related tests with file logging:

cargo make sharding-tests-debug

Using a debugger

When using a debugger with the tests, make sure to pass the --nocapture option to the test runner, otherwise the debugger will not be usable (when capturing is on, the test framework spawns child processes to run the actual tests).

Updating the REST API

Golem generates OpenAPI specs from the Rust code (using the poem-openapi crate, and the generated OpenAPI yaml file is also stored in the repository and a Rust Client crate is generated from it, used by the CLI app and also published into crates.io.

When changing anything that affects the user facing REST API, this YAML needs to be explicitly regenerated. If this does not happen, the CI process will fail and ask for doing it.

To regenerate the OpenAPI spec use

cargo make generate-openapi

Updating the config files

Service config files are also generated from code similarly to OpenAPI specs. When changing any of the service configuration data types, they have to be regeneraetd otherwise the CI process fails and asks for doing it.

To regenerate these files, use

cargo make generate-configs

Preparing the pull request

Golem CI checks the pull requests with rustfmt and cargo clippy. To make sure these checks pass, before opening a pull request run

cargo make fix

and fix any possible errors and warnings reported by it.

Running Golem locally

There are two ways now to run Golem locally:

Using cargo make run

By running cargo make run all services are going to be built and started as individual native processes. Ensure that lnav, nginx and redis commands are available in your PATH.

Using cargo make run-with-login-enabled

By running cargo make run-with-login-enabled, all services are started as individual native processes and the login system is enabled. In addition to the requirements for regular run, you'll need to set up GitHub OAuth.

  1. Go to https://github.com

  2. Navigate to Settings > Developer settings > OAuth Apps

  3. Create a new OAuth App. Use the following settings:

    Authorization callback URL: http://localhost

  4. Generate a new client secret.

  5. Export the following environment variables:

    GITHUB_CLIENT_ID: Client ID of the OAuth application created in step (3)

    GITHUB_CLIENT_SECRET: Client secret created in step (4)

After it is started you can configure the cli to use the local instance using golem profile new --component-url http://localhost:9881/ cloud cloud-local.