Release highlights
* Starting from this release, all components of `hivemind.moe` use `libp2p` for communication. This comes with the same benefits as in averaging and DHT previously (simplified NAT traversal, better performance, etc.) and marks the end of gRPC usage in hivemind. The user API is mostly the same: if you were using abstractions like `RemoteMixtureOfExperts`, the code should not be changed, although cross-release training is not possible.
* If you need another way to reduce the network footprint during training with `hivemind.Optimizer`, you can now use [PowerSGD](https://arxiv.org/abs/1905.13727) for gradient averaging. This method decreases the communication costs by factorizing the gradients of the model and aggregating the factorized versions. To enable this method in your code, pass `grad_averager_factory=partial(PowerSGDGradientAverager, averager_rank=RANK)` when creating an instance of Optimizer. Here, RANK denotes the factorization rank; lower values give higher compression at the cost of the reconstruction quality.
* Similarly to `hivemind-server`, it is now possible to launch a dedicated DHT instance with a command-line tool. The tool, available via `hivemind-dht`, can be used to quickly create a lightweight peer that is used mostly for connecting others to the DHT (for example, on a publicly available server) or for DHT metadata replication.
* Previously, restarting a libp2p instance required generating a new P2P identity, which resulted in a new multiaddress. Thus, it was difficult to use the same command to connect to a peer in case of repeated launches, which is often the case during debugging. Now, you can store the persistent peer identity of a peer in a file and reuse it between launches: this is done by specifying the `--identity_path` argument, available both in the ALBERT example and CLI tools of hivemind.
Deprecations
* The parameters `quic`, `use_relay_hop`, and `use_relay_discovery` of hivemind.P2P are deprecated since our update of the libp2p dependency in the p2p daemon. They will be removed in the 1.2.0 release of hivemind
What's Changed
* Pin pytest version in requirements-dev, use file_descriptor in tests by justheuristic in https://github.com/learning-at-home/hivemind/pull/454
* Pin isort version, bump black by mryab in https://github.com/learning-at-home/hivemind/pull/456
* Clean compression/__init__.py by borzunov in https://github.com/learning-at-home/hivemind/pull/460
* Do not use offload_optimizer with local_updates by deafult by foksly in https://github.com/learning-at-home/hivemind/pull/462
* Add PowerSGD for compressed gradient averaging by artek0chumak in https://github.com/learning-at-home/hivemind/pull/432
* Bump Black to 22.3.0, pin Golang version by mryab in https://github.com/learning-at-home/hivemind/pull/466
* use_local_updates in optimizer by justheuristic in https://github.com/learning-at-home/hivemind/pull/468
* Update p2pd to v0.3.8 (and libp2p to v0.17.0) by borzunov in https://github.com/learning-at-home/hivemind/pull/469
* Generate new private key if identity file doesn't exist by borzunov in https://github.com/learning-at-home/hivemind/pull/473
* Convert hivemind.server to libp2p backend by GreenFatGuy in https://github.com/learning-at-home/hivemind/pull/470
* Implement a CLI for hivemind.DHT by mryab in https://github.com/learning-at-home/hivemind/pull/465
* Use PeerID exclusively to address MoE experts by justheuristic in https://github.com/learning-at-home/hivemind/pull/479
* Remove deprecated code in hivemind.optim and hivemind.averaging before the 1.1.0 release by mryab in https://github.com/learning-at-home/hivemind/pull/480
* Fix shape validation in GradientAverager by mryab in https://github.com/learning-at-home/hivemind/pull/481
* Change expiration time in declare_experts, fix update_period discrepancy by justheuristic in https://github.com/learning-at-home/hivemind/pull/482
* Add identity_path option for MoE.Server runners by GreenFatGuy in https://github.com/learning-at-home/hivemind/pull/484
* Simplify ExpertBackend interface by justheuristic in https://github.com/learning-at-home/hivemind/pull/483
* Clean up imports, remove unused utils by mryab in https://github.com/learning-at-home/hivemind/pull/486
* finish renaming experts -> module_backends in ConnectionHandler by justheuristic in https://github.com/learning-at-home/hivemind/pull/487
* Remove gRPC services and grpcio requirement by mryab in https://github.com/learning-at-home/hivemind/pull/485
New Contributors
* GreenFatGuy made their first contribution in https://github.com/learning-at-home/hivemind/pull/470
**Full Changelog**: https://github.com/learning-at-home/hivemind/compare/1.0.1...1.1.0