This is a standard quarterly feature and bugfix release.
New Features
Redis Cluster Support
Baseplate.py now has an instrumented client for the [redis-py-cluster] library.
This allows your service to interact with redis in cluster mode.
For more information see [the documentation][cluster-docs] or 573.
[redis-py-cluster]: https://github.com/Grokzen/redis-py-cluster/
[cluster-docs]: https://baseplate.readthedocs.io/en/v2.1.0/api/baseplate/clients/redis_cluster.html
Deadline Budget Headers
If service A sets a time budget on calling service B and service B takes too
long, it's wasteful for service B to keep processing that request long after
service A has given up and stopped waiting for a response. To prevent this
waste, Baseplate.py will now send deadline budget headers automatically when
you specify a time budget on Thrift RPCs. Thrift servers will understand this
header and set their internal [server timeout] accordingly.
For example, the following call:
python
with context.service_b.retrying(attempts=3, budget=2) as svc:
svc.some_method()
Will retry the `some_method()` call up to 3 times as long as the total time
spent is less than 2 seconds (though note that the socket may block longer).
Each call to service B will carry information about how much time in the
budget remains, which will also be carried onto any services that service B
itself calls!
For more information, see 574 and 578.
[server timeout]: https://baseplate.readthedocs.io/en/v2.1.0/api/baseplate/observers/timeout.html
Changes
* Don't emit JSON logs to TTYs, this should be easier to read when running baseplate-serve directly. (also in v2.0.1, 584)
* Add runtime metrics for memcache pools. (586)
* Support SQLAlchemy 1.4 (also in v2.0.1, 587—please [read these notes before upgrading][sqlalchemy])
* Support publishing v2j batches in the event publisher sidecar. (also in v2.0.3, 590)
* Add a drain period to server shutdown. This allows the application to avoid getting routed traffic while shutting down. (591, 594)
Bug Fixes
* Add timeouts to HTTP calls made by secrets fetcher sidecar. (also in v2.0.2, 588)
* Fix metrics not being emitted from spans that were children of local spans (also in v2.0.3, 592)
Known Issues
* Returning an HTTPException subclass as a response in a Pyramid application may result in no body content being returned to the client. This bug was introduced in v1.5.0. See 575.
Upgrading
There should not be any breaking changes in this release. As always, use the [baseplate.py-upgrader](https://github.com/reddit/baseplate.py-upgrader) to get automatic upgrades and advice on things that need to be done manually.
If you want to reduce error rates during deploys, you can take advantage of the
drain period by making your healthcheck endpoint return a failure for
`READINESS` checks when the server is in the shutting down state. Here's an
example for an HTTP service:
diff
-1,6 +1,9
from baseplate.frameworks.pyramid import BaseplateRequest
from baseplate.frameworks.pyramid import get_is_healthy_probe
+from baseplate.server import SERVER_STATE
+from baseplate.server import ServerLifecycle
from baseplate.thrift.ttypes import IsHealthyProbe
+from pyramid.httpexceptions import HTTPServiceUnavailable
from pyramid.response import Response
from pyramid.view import view_config
-16,6 +19,9
return {}
elif probe_type == IsHealthyProbe.READINESS:
the readiness probe wants to know if we're ready to handle traffic.
+ if SERVER_STATE.state == ServerLifecycle.SHUTTING_DOWN:
+ raise HTTPServiceUnavailable(detail="no more please, I'm shutting down")
+
return {}
else: pragma: nocover
not sure what the future holds for us. let's play it safe.
[sqlalchemy]: https://docs.sqlalchemy.org/en/14/changelog/migration_14.html