Baseplate

Bug Fixes
- Fix crash in secrets fetcher startup

New Features

EdgeRequestContext/AuthenticationToken unification

This isn't a new addition, but a **breaking** rework of authentication context in Baseplate. Authentication token propagation and access is now fully integrated into the edge request context. Authentication tokens are propagated inside the edge context header and the API for applications built on Baseplate is unified. See below for details on how to use this.

For context, here are the original release notes on the authentication system:

> Authentication tokens provided by the authentication service can now be automatically propagated between services when making Thrift calls. This allows internal services to securely and accurately understand on whose behalf a given request is being made so they can decide if the requester is authorized for a particular action. The context is passed implicitly, in request headers, so no extra parameters need be added to service IDLs. Baseplate provides APIs for validating and accessing the tokens from within request context and will automatically pass upstream credentials to downstream services without extra work.

This should now be stable and ready for wide use.

Kubernetes authentication backend for Vault secrets fetcher

Baseplate's secret fetcher daemon can now authenticate to Vault using Kubernetes as its proof of identity. This allows Baseplate's Vault sidecar to be used inside Kubernetes pods.

Histogram metrics

You can now use Baseplate's [metrics API](https://baseplate.readthedocs.io/en/stable/baseplate/metrics.html#baseplate.metrics.Client.histogram) to send arbitrary integers to StatsD to be collected into [Histograms](https://en.wikipedia.org/wiki/Histogram). This is useful for monitoring the distribution of a metric over a time interval.

thrift_pool_from_config

A new helper for creating [`ThriftConnectionPool`](https://baseplate.readthedocs.io/en/stable/baseplate/thrift_pool.html#baseplate.thrift_pool.ThriftConnectionPool) objects from configuration without the boilerplate. See "Upgrading" below for details of how to switch over.

Changes

* Baseplate now emits a new [`ServerSpanInitialized`](https://baseplate.readthedocs.io/en/stable/baseplate/integration/index.html#baseplate.integration.pyramid.ServerSpanInitialized) event in Pyramid applications after it is done initializing a request's span context but before request handler starts. This allows you to hook in to the request lifecycle but have access to all the context attributes you have registered with Baseplate.
* Baseplate CLI tools now use `/usr/bin/env python` for compatibility with environments like virtualenv.
* The setup script (setup.py) now triggers the thrift build process. This makes `python setup.py sdist` and descendants work cleanly.

Bug Fixes

* Don't retry publishing forever when v2 event fails to validate.
* Use correct path for authentication token public key in Vault.

Upgrading

EdgeRequestContext

Most of the workings of this system are under the hood in Baseplate, but you'll need to configure it at application startup. The way this looks is a little different depending on where in the call graph your service sits.

**For "landlocked" services inside the cluster** that don't interact directly with clients, we assume that upstream services will propagate edge request context (including authentication) to us and all we need to do is verify this. Create an [`EdgeRequestContextFactory`](https://baseplate.readthedocs.io/en/stable/baseplate/core.html#baseplate.core.EdgeRequestContextFactory) passing in the secrets store for your application and then pass that along to your application's Baseplate integration. For an example Thrift service:

diff
--- a
+++ b/
-90,6 +90,7

+from baseplate.core import EdgeRequestContextFactory
from baseplate.secrets import secrets_store_from_config

--- a
+++ b
-115,9 +115,15 def make_wsgi_app(app_config):
+ edge_context_factory = EdgeRequestContextFactory(secrets)
+
handler = Handler()
processor = {{ cookiecutter.service_name }}.ContextProcessor(handler)
- event_handler = BaseplateProcessorEventHandler(logger, baseplate)
+ event_handler = BaseplateProcessorEventHandler(
+ logger,
+ baseplate,
+ edge_context_factory=edge_context_factory,
+ )
processor.setEventHandler(event_handler)
return processor


Now you can use `context.request_context` ([`EdgeRequestContext`](https://baseplate.readthedocs.io/en/stable/baseplate/core.html#baseplate.core.EdgeRequestContext)) and all of its properties to access the context provided by upstream edge services.

**For services at the edge**, that is ones that interact with external clients directly, we need to collect the right information from the external request and put it into the context to be propagated to downstream services. Create an [`EdgeRequestContextFactory`](https://baseplate.readthedocs.io/en/stable/baseplate/core.html#baseplate.core.EdgeRequestContextFactory) and use it to make fresh context on each request. For a Pyramid service, the new [`ServerSpanInitialized`](https://baseplate.readthedocs.io/en/stable/baseplate/integration/index.html#baseplate.integration.pyramid.ServerSpanInitialized) event is particularly useful here because we can use any services managed through Baseplate in the event handler.

diff
--- a
+++ b/
-90,6 +90,7

+from baseplate.core import EdgeRequestContextFactory
+from baseplate.integration.pyramid import ServerSpanInitialized
from baseplate.secrets import secrets_store_from_config

--- a
+++ b
-115,9 +115,15 def make_wsgi_app(app_config):

+ edge_context_factory = EdgeRequestContextFactory(secrets)
+
+ def add_edge_context(event):
+ bearer_token = event.request.headers[...]
+ authn_response = event.request.authn_service.authenticate_oauth2_bearer_token(bearer_token)
+ edge_context = edge_context_factory.new(
+ ... fill in details from the external request ...
+ )
+ edge_context.attach_context(event.request)
+
+ configurator.add_subscriber(add_edge_context, ServerSpanInitialized)

`

All of this context will be available in your application immediately and will also show up automatically in downstream services.

**Both types of services** will need access to the authentication service's public key to be able to validate authentication tokens. This is as simple as adding `secret/authentication/public-key` to the list of secrets managed by your service's secret fetcher daemon.

See [the documentation](https://baseplate.readthedocs.io/en/stable/baseplate/core.html#edge-request-context) for more details on all this.

thrift_pool_from_config

If your application was doing its own configuration parsing to build a Thrift connection pool, you can now use this function instead.

Before:

python
cfg = config.parse_config(app_config, {
"example_service": {
"endpoint": config.Endpoint,
"timeout": config.Timespan,
},
})

...

example_pool = ThriftConnectionPool(
cfg.example_service.endpoint,
timeout=cfg.example_service.timeout.total_seconds(),
)

...


After:

python
example_pool = thrift_pool_from_config(app_config, prefix="example_service.")

Changes

* Encode AuthenticationContext token
* Change baseplate tween position in pyramid integration

Changes

* EdgeRequestContext.create now supports `loid_id=None`.

New Features

Edge Request Context Propagation

The EdgeRequestContext provides an interface into both authentication and context information about the original request from a user. For edge services, it provides helpers to create the initial object and serialize the context information into the appropriate headers. Once this object is created and attached to the context, Baseplate will automatically forward the headers to downstream services so they can access the authentication and context data as well.

The first step to this getting wide use will be to get the services closest to clients wired up to get authentication tokens from the authentication service and to start parsing the LoID and Session Tracker values from cookies/headers, then internal services will automatically get the request context brought along to them.

Changes

* The Experiments client works with the new Edge Request Context objects. You can pass a User object (`context.request_context.user`) directly to the call to `Experiments.variant` and the client will extract the standard user inputs from that object and pass those to the call to `experiment.variant` as well as include any User event fields in the bucketing event. The Experiments client will also pull event fields out of the Edge Request Context that is attached to the current context.
* The experiment config format has been updated. The "expires" parameter has been deprecated and several new fields have been added. Configs that still use the "expires" field will continue to work as before with a warning that the field is deprecated.

Bug Fixes

* The formatting string for the expiration date of an experiment was incorrect.

Upgrading

Authentication Context

Direct use of the Authentication Context object is now deprecated. You should use the Edge Request Context and it's wrappers for the Authentication Context rather than using the Authentication Context directly. Also, baseplate will only forward the authentication token if it is included in the Edge Request Context, it will not forward the token from an Authentication Context object that is attached directly to the context.

New Features

Authentication Context Propagation

Authentication tokens provided by the authentication service can now be automatically propagated between services when making Thrift calls. This allows internal services to securely and accurately understand on whose behalf a given request is being made so they can decide if the requester is authorized for a particular action. The context is passed implicitly, in request headers, so no extra parameters need be added to service IDLs. Baseplate provides APIs for validating and accessing the tokens from within request context and will automatically pass upstream credentials to downstream services without extra work.

The first step to this getting wide use will be to get the services closest to clients wired up to get authentication tokens, and then internal services will automatically get the authentication context brought along to them.

v2 Events Support

Baseplate's event system now supports the Thrift-based schemaful v2 events system. Event publishers can be configured to send the new V2 style event payloads. Please see below for instructions on upgrading.

Changes

* Many bug fixes, compatibility improvements, and performance boosts to the experiments framework based on testing in r2. However, it's still an experimental experiment framework so please hold off on using it in your services yet.
* For same-machine `MessageQueue` use cases that need more buffering room, a `MessageQueue` implementation backed by Redis has been added.

Bug Fixes

* Spans created as children of local spans are now properly wired up to the instrumentation systems. Metrics, tracing, etc did not work before in that context but will now.

Upgrading

Dependencies

Baseplate now has a new hard dependency on `PyJWT`. This should be automatically installed in new development environments going forward, but for existing ones you'll want to install the package inside your development VM:


$ sudo apt install python-jwt python3-jwt


V2 Events

In your application's entry point, add a V2 event queue:

diff
-2,7 +2,7
-from baseplate.events import Event, EventQueue
+from baseplate.events import Event, EventQueue, serialize_v2_event

-97,7 +97,7 def make_wsgi_app(app_config):
baseplate.add_to_context("events_production", EventQueue("production"))
baseplate.add_to_context("events_test", EventQueue("test"))
+ baseplate.add_to_context("events_v2", EventQueue("v2", event_serializer=serialize_v2_event))


Install the event schemas IDL files in your application and then you can then instantiate Thrift-based events and put them into the queue like normal.

python
from event_schemas.event.ttypes import Event

def some_handler(request):
event = Event(
source="baseplate",
action="test",
noun="baseplate",
client_timestamp=time.time() * 1000,
uuid=str(uuid.uuid4()),
)
request.events_v2.put(ev2)


There will not be any v2 test publishers on production machines going forward. Instead, the v2 queue will publish to test or production depending on if the application is running in staging or production. You can remove the v1 test queue if you like.

Authentication Context

Until edge services are wired up to the authentication service, there won't be any authentication contexts flowing around, but this is what integration looks like when it's time.

For a service deep inside the call graph, all you need to do is check the authentication.

python
def some_handler(request):
if not request.authentication.valid:
raise NotAuthenticatedError

if request.authentication.account_id in allowed_ids:
...


Services at the edge need to do some extra work to get a token from the authentication service and then add it to the context. For this and other deeper use cases, check out [the docs](http://baseplate.readthedocs.io/en/stable/baseplate/core.html#authentication).