Packages

• pysc2 v1.2

• sc2client-proto v3.18.0

• pytorch v0.4.0

• StarCraft II (Linux) v.3.16.1

• tensorboardX (Tensorboard for Pytorch) v1.0

• common packages

  • psutil
  • cloudpickle

Python3 is required to resolve multiprocessing issue.

Detail installation steps.

Usage

• <project_path>: full path of this repo in local environment
• <hose_name>: host name of local environment (ex. localhost)

Train

To train agent in "FindAndDefeatZerglings" mini game with 8 different worker threads:

cd <project_path>
python -m rl.main --map-name FindAndDefeatZerglings --num-processes 8

Use python rl/main.py --help to see all available options.

To train with GPU, use --gpu_ids option. Default is CPU only.

# Use two GPUs for example
python -m rl.main --map-name FindAndDefeatZerglings --gpu_ids 0 1 --num-processes 8 

Monitor

To visualize training progress stats, run Tensorboard (tensorflow required).

tensorboard --logdir <project_path>/output/summaries

Then open the link http://<host_name>:6006 in browser.

Output

All output files are located in <project_path>/output by default.

• Trained models: <project_path>/output/models
• Logs/Temp files: <project_path>/output/logs
• Tensorboard summary logs: <project_path>/output/summaries

Installation

pytorch

Follow instruction here and chose OS, Package Manager, Python version and CUDA version accordingly.

• Linux

# check cuda version
nvcc --version
# use CUDA 9.1 as example
pip3 install http://download.pytorch.org/whl/cu91/torch-0.4.0-cp35-cp35m-linux_x86_64.whl 

• OS X

# no GPU as example
pip3 install torch

pysc2

pip install pysc2

tensorboardX

pip install git+https://github.com/lanpa/tensorboard-pytorch
# TensorFlow is required for dashboard visualization
pip install tensorflow

References

pytorch reinforcement learning

• pytorch-a3c - pytorch A3c implementation
• rl_a3c_pytorch - pytorch A3C with GPU

pysc2 integratioin

• pysc2-agents - pysc2 A3C agent with FullyConv model and epsilon greedy exploration by Tensorflow
• pysc2-rl-agents - pysc2 A2C agent with FullyConv model by Tensorflow