Torchgpipe

~~~~~~

Released on May 14, 2019 to evaluate usability and efficiency internally.

- Provided a functional GPipe implementation, including pipeline parallelism,
checkpointing, and deferred batch normalization.
- Supported Python 3.6+ and PyTorch 1.0.