realtime quantization to workers #63
Description
in theory, if the master has the full res model, while streaming to workers it could quantize it automatically on the fly so that a worker can host a quantized version of the full model the master is using / orchestrating.