Very shortly, the dev branch will switch to 0.8 beta and we will continue to support hotfixes into 0.7 as needed here: https://github.com/OpenMined/pysyft/tree/0.7.0
We will continue the weekly beta releases of 0.8 however there might be some stability issues in the short term while we merge in some larger pending changes
Going forwards, as we test on all PRs and do Nightly smoke tests the dev branch / 0.8, beta builds will be quite stable and are dogfooded regularly by our team and partners
Padawan Program
A big thank you to all our Padawan Program contributors: amdjedbens, annareithmeir, callezenwaka, faustyang, kanak8278, majauhar, mihirdcoder, mikaelapisani, Nilanshrajput, OSobky, rAlexandre00, saffronh, sim017, the-elancier, tthoraldson, vinalb-oblivious, yashgorana, znreza and anyone in the OpenMined Community who helped but doesn’t have a github account.
If you wan't to join the Padawan Program you can see more here:
https://blog.openmined.org/work-on-ais-most-exciting-frontier-no-phd-required/
What's Changed
README.md
- New README
- Updated Docs
Platform Support
- Apple Silicon
- Python 3.10
- linux/arm64 containers on dockerhub
- Test deployments to Azure and AWS k8s clusters
HAGrid
- Support for prebuilt containers with –tag=$DOCKER_HUB_TAG
- Added quickstart mode
- Added install wizard
- Deployment support to GCP with `gcloud`
- Support for deploying multiple nodes at once to Azure
- Added `check` command to check node health / status
- Added password randomization during deployment
- Optional JupyterLab on deployed Domain
Grid
- Improved container start times by removing db migration race condition
- Kubernetes with k3d and full integration test compatibility
- Refactored Docker Traefik to use static and dynamic yaml
- Tailscale VPN Sidecar is optional
- Network Association allows for custom routes
- Added threads to sync endpoint
- Improved Network message handling and forwarding
CI / CD
- Weekly CD for Syft and HAGrid
- Refactored Integration tests to use common tox task
Permissions
- Added composable RBAC decorators for granular security
Storage and Serialization
- Support for >2GB objects via `capnp`
- Added SeaweedFS Blob Storage for fast transfer and storage of large tensors
- Added redis for fast key value store access
Tensor Types
- FixedPrecisionTensor
- PhiTensor and GammaTensor
- Optional Public fields for TensorPointers
Numpy-like API
- Increased support for numpy-like API and Tensor Operations
DP
- Refactored DP system to be more scalable
- Our new Automatic DP system now uses JAX’s Automatic Differentiation replacing primes and our ScalarManager
- Replaced scalar / element wise data structures with performant Tensors that work at the scale of 1B rows
- Added lazy evaluation for sparse DP min and max values for smaller size and faster computation
- Added several additional methods to the Syft Tensor API
- Added RDP Constants Ledger
SMPC
- Floating Point Support with the inclusion of FixedPrecisionTensor
- Moved SMPC Action Messages to ShareTensor Context
- Included SMPC support for various ring sizes(2**16,2**32,2**64)
- Reduced the communication overhead of operations(PRZS Action, Beaver Action) to a single message per operation
- SMPC division support for public and private tensor values
DP+SMPC
- Initial End-to-End DP+SMPC Model Training developed for an XOR Example
- Support for numpy scalar data types