Grokmirror

-------------------
- Add support to use git plumbing for objstore operations, via enabling
core.objstore_uses_plumbing. This allows to significantly speed up
fetching objects into objstore during pull operations. Fsck operations
will continue to use porcelain "git fetch", since speed is less important
in those cases and it's best to opt for maximum safety. As a benchmark,
with remote.preload_bundle_url and core.objstore_uses_plumbing settings
enabled, cloning a full replica of git.kernel.org takes less than an hour
as opposed to over a day.

-------------------
- Refuse to delete ffonly repos
- Add new experimental bundle_preload feature for generating objstore
repo bundles and using them to preload objstores on the mirrors

-------------------
- Provide pi-piper utility for piping new messages from public-inbox
repositories. It can be specified as post_update_hook:
post_update_hook = /usr/bin/grok-pi-piper -c ~/.config/pi-piper.conf
- Add -r option to grok-manifest to ignore specific refs when calculating
repository fingerprint. This is mostly useful for mirroring from gerrit.

-------------------
- fix potential corruption when migrating repositories with existing
alternates to new object storage format
- improve grok-fsck console output to be less misleading for large repo
collections (was misreporting obstrepo/total repo numbers)
- use a faster repo search algorithm that doesn't needlessly recurse
into git repos themselves, once found

-------------------
Major rewrite to improve shared object storage and replication for VERY
LARGE repository collections (codeaurora.org is ~30,000 repositories,
which are mostly various forks of Android).

See UPGRADING.rst for the upgrade strategy.

Below are some major highlights.

- Drop support for python < 3.6
- Introduce "object storage" repositories that benefit from git-pack
delta islands and improve overall disk storage footprint (depending on
the number of forks).
- Drop dependency on GitPython, use git calls directly for all operations
- Remove progress bars to slim down dependencies (drops enlighten)
- Make grok-pull operate in daemon mode (with -o) (see contrib for
systemd unit files). This is more efficient than the cron mode when
run very frequently.
- Provide a socket listener for pubsub push updates (see contrib for
Google pubsubv1.py).
- Merge fsck.conf and repos.conf into a single config file. This
requires creating a new configuration file after the upgrade. See
UPGRADING.rst for details.
- Record and propagate HEAD position using the manifest file.
- Add grok-bundle command to create clone.bundle files for CDN-offloaded
cloning (mostly used by Android's repo command).
- Add SELinux policy for EL7 (see contrib).

-------------------
- Small bugfixes
- Generate commit-graph file if the version of git is new
enough to support it. This is done during grok-fsck any time we
decide that the repository needs to be repacked. You can force
this off by setting commitgraph=never in config.