Localstack

Change log

1. New Features

* add ability to import keys into KMS
* enable and refactor rotation logic and version stages in SecretsManager
* support "Accept" header to switch response format
* add provider for S3Control service

2. Enhancements

* migrate services to ASF: EC2, KMS, Route53, STS
* skip installing -ext dependencies in CI tests
* lazily load subclasses for instance manager to enable plugin loading at runtime
* add missing UpdateDomainConfig for OpenSearch and Elasticsearch
* pin werkzeug version to fix Transfer-Encoding "chunked" on empty responses
* apply filter pattern for log subscriptions
* skip "online-only" tests, fix some tests
* use valid references for EC2 resources to fix CloudFormation tests
* enhance handling of KMS key pairs for SIGN_VERIFY keys
* add CloudFormation support for IAM::ServiceLinkedRole
* reintroduce Lambda log output str casting
* source API Gateway domain name and prefix from invocation context
* pin airspeed version and remove patch code
* add CloudFormation support for AWS::CDK::Metadata
* remove SERVICES from default docker-compose.yaml
* round DynamoDB ApproximateCreationDateTime to seconds
* add GetAtt support for AWS::EC2::VPC DefaultSecurityGroup and DefaultNetworkAcl
* minor update in patch decorator to allow multiple/composite patches of methods

3. Bug Fixes

* fix PaginatedList nextToken generation
* fix MessageGroupId for publish batch to SQS fifo queue
* fix elliptic curve key specs for KMS keys
* fix broken transitive imports of quart from jinja
* fix Terraform creation of EC2 Security Groups with default egress
* fix eager deserialization for 404 responses returned from Lambda
* fix CloudFormation Route53 deployment without ResourceRecords
* fix ACM DomainValidationOptions to support waiters
* fix Flask app routes for Lambda API
* fix broken patch for moto S3 is_delete_keys(..)
* fix CFn name autogeneration for SQS fifo queues
* fix CFn Route53 deployment
* fix CFn default name for Kinesis streams
* fix logs subscription filter
* fix query request parser for operations without input shape
* fix nested stack outputs in CloudFormation
* fix handling encoded `$` for default stage in API Gateway path regex
* fix redirect to index file in S3 website

Change log

1. New Features

* push Docker images to public ECR registry on CI build
* add internal endpoint to output diagnostics data
* add ability to build, tag, and push images using Docker client
* add API Gateway SNS integration
* add OpenSearch destination to Firehose
* add support for CloudWatch set_alarm_state and trigger actions

2. Enhancements

* implement in-memory request dispatching to moto
* implement generic HTTP route dispatching
* migrate several services to ASF (ACM, CloudWatch, CloudWatch Logs, ConfigService, DynamoDB Streams, Redshift, ResourceGroups, SecretsManager, SSM, Support, SWF)
* migrate from requirements.txt/setup.py to setup.cfg
* CircleCI pipeline parallelization
* add GitHub action to automatically update ASF APIs
* migrate resourcegroupstagging API to ASF
* split up Docker client, remove obsolete `docker` client binary from image
* refactor handling of API Gateway request/response templates
* introduce CODEOWNERS file for code ownership
* add timeout to status services cli command
* add DynamoDB Streams integration for PartiQL query updates
* add ability to extract cookies from API Gateway invocation context
* update generated ASF APIs to latest version
* add ability to specify timeout and max content length for HTTP server
* improve cli startup performance
* refactor utils imports in utils and services packages
* patch moto Lambda backend to access LocalStack Lambdas directly
* add ability to extract tgz archives via download_and_extract utility
* allow specifying working directory when running Docker exec
* implement regex rules for the http Router
* migrate from localstack-plugin-loader to plux
* add support for define function in API Gateway velocity templates
* add a GHA workflow to track first-time pull request and issue creators
* rework docker capabilities, remove intermediate containers after building images in docker-sdk client
* add optimizing and sorting of imports to ASF scaffold
* add support for response templates in API Gateway MOCK responses
* add mock ECR CFN resource and integration test for CDK bootstrap template v10
* add flag to call_moto to exclude the ResponseMetadata
* upgrade airspeed dependency and remove custom patches
* add helpers to unpause Docker containers
* reorganize utility package
* add deep copy of handler results to avoid unwanted mutations in parser
* add error check for fifo attribute when creating SQS fifo queues
* update renovatebot config to ignore Dockerfile.rh
* add ApiGateway S3 integration using action name
* add method to allow calling moto with a new request in request dispatcher
* add resolve_hostname(..) util function, refactor some utils into sub-modules
* improve performance of localstack status command
* remove `six` library from dependencies
* add failsafe with try/except for creating log streams to guard against potential duplicates
* add the ability to snapshot running Docker containers
* make autocreating log groups optional
* allow service restart in ServiceContainer from STOPPED state
* add support for DefinitionS3Location in StepFunctions::StateMachine model
* remove superfluous CloudWatch response processing
* add PaginatedList for paginating and filtering response lists
* add default entries in CloudFormation stack template dict to avoid KeyError
* docs: refactor third party software tools docs and remove announcements in README
* refactor some tests to pytest (e.g., Lambda, EC2)
* configure test coverage exclusions
* make Lambda output truncation configurable

3. Bug Fixes

* fix ASF parser and serializer issues
* fix custom error responses from lambda when handler throws an exception
* fix moto dispatching for trailing slashes and response headers
* fix OpenSearch data dir issue that would block startup in pro
* fix encoding problem for AlarmDescription in put-metric-alarm
* fix making typing_extensions import dependent on python version
* fix ApiGateway v2 path-base style requests without stages
* fix several ASF parser, serializer, and scaffold issues
* fix replacement of AWS::NoValue, add default props for Elasticsearch::Domain resources
* fix service provider configuration override
* fix edge route mapping for STS AssumeRole
* fix API Gateway integration with Kinesis
* fix linting issues by pinning flake8-black
* fix python 3.7 compatibility with TypedDict for cli
* fix ASF xml-rest parser location- and payload-trait handling
* fix CloudFormation stack class to return latest raw template if change sets are present
* fix OpenSearch cluster shutdown in tests
* fix parameter for StateMachine type in CFN model
* fix uncapped recursion bug in parser

Announcements

- **Breaking changes:**
- This version introduces the external service port range in LocalStack. This port range has been used in LocalStack Pro before, but has now been moved to the community edition. While doing so, the port range has been broadened (it now ranges from 4510 to 4559). Please make sure to adjust your Docker configurations to expose this port range (such that external services started within the container can be reached) or use the latest version of the LocalStack CLI to manage your LocalStack container.
- When using ElasticSearch with the `ES_ENDPOINT_STRATEGY=port` (formerly `off`) and `ES_MULTI_CLUSTER=0`, a port from the external service port range is used (instead of `PORT_ELASTICSEARCH` or `4571` by default).
- **Upcoming breaking change:** This version deprecates the `ES_ENDPOINT_STRATEGY`, `ES_MULTI_CLUSTER`, and `ES_CUSTOM_BACKEND`. Instead, the `OPENSEARCH_*` variables should be used. The `ES_ENDPOINT_STRATEGY=off` is being renamed to `port` (i.e. please use `OPENSEARCH_ENDPOINT_STRATEGY=port` from now on).

Change log

1. New Features

* add initial support for SNS PublishBatch API
* add support for email-json protocol in SNS subscriptions
* add support for AWS OpenSearch
* support multi arch for local-kms provider
* support invocation of local API URLs via stepfunctions

2. Enhancements

* bump moto-ext to latest version
* minor refactoring in SNS get_message_attributes() method
* add utilities for migrating services to ASF
* minor refactoring in selecting handler in edge.py for better extensibility
* add check for supported protocols on SNS subscription
* adjust routing to enable using anonymous boto3 client to fetch S3 object
* refactor Lambda executors to allow modification of container settings using hooks
* minor adjustments in patch decorator for bound methods of class instances
* hardcode default tmp and data paths
* Add handling for DynamoDB stream iterator expiration, bump kinesis-mock to 0.2.2
* remove obsolete moto imports in CloudFormation resource classes
* remove obsolete patch for CloudWatch metrics filtering
* cleanup CloudFormation logic and remove old/obsolete code
* add utility to manage / reserve ports for external services
* invalidate builder stage in dockerfile on package version changes
* use base_url instead of url for determining SQS queue URL
* add cachetools to CLI dependencies
* set MessageGroupId and MessageDeduplicationId on SQS message from SNS subscription
* Introduce lazily-created localstack-internal-awssdk utility lambda function
* capture X-AMAZON-APIGATEWAY-ANY-METHOD for cases where route key is for integrations
* provide better log info when CloudFormation resources cannot be found
* clean up old/unused CloudFormation logic
* refactor SQS queue-url generation
* extend SNS publish batch support for FIFO topics
* remove monkey patching and circular dependency
* refactor sqs queue resolving and url rendering
* minor refactoring of CFN logging code to dump resources as JSON string
* clean up aws_stack utils, remove obsolete global cached boto3 resources
* add missing CloudFormation parameters for Firehose delivery streams
* migrate firehose to ASF
* stop setting environment variable if cognito_identity if empty
* Kinesis refactor server abstraction
* Allow container client to connect/disconnect containers to/from a network
* handle SSE for sqs
* refactor config and rip out global port/url variables
* add connection parameters and credentials to EventBridge API destinations
* replace function calls with literal syntax and remove unnecessary comprehensions
* broaden the external service port range, adjust docker-compose
* update Docker base image (python:3.8.12-slim-buster) digest to 3544d06
* add util function for deletion of None-values from dicts
* combine OpenSearch and Elasticsearch backends
* update exposed ports and docker-compose


3. Bug Fixes

* fix CloudFormation deployment of SNS::TopicPolicy
* fix handling of empty prefix for Firehose S3 uploads to properly trigger event notification
* fix external host setting for sqs
* fix acronym handling in camel_to_snake_case
* fix API Gateway to match the most specific path in the presence of "{proxy+}" placeholders
* fix asf scaffold to generate TypedDict declarations correctly
* fix sqs port extraction from headers and other related issues
* fix ASF SQS provider to allow direct calls to queue URLs
* fix DynamoDB local/localhost region for NoSQL Workbench
* fix ASF SQS xml response encoding
* fix request parser to enable X-Amz-Target strings with multiple dots
* fix expectations around Kinesis record encoding, always re-encode records for shard subscribers
* fix CloudFormation deployer to recursively resolve string placeholders
* fix ASF REST operation lookup with conflicting path params
* fix DomainValidationOptions property in ACM Certificate CFN deployment
* fix access to listener in logic for restarting Kinesis
* fix issue rendering VTL templates for API Gateway by flattening the input
* fix accessing external SQS port for intra-service communication
* fix positional args in CloudFormation find_change_set(..) method
* fix SNS PublishBatch logic for optional Subject atttribute
* fix Kinesis installation via LPM, add Docker check for StepFunctions installation
* fix docker client ctx manager and json decode issues on windows
* fix passing Conditions on stack updates, refactor/simplify CFN function arguments
* fix EC2 route tables and service backend initializations for recent upstream moto changes
* add config for host path to fix mounting of libs_dir
* minor fix in SNS tests to resolve stale PR merge
* minor fix in printing result for local node.js Lambda executor
* ASF rest-json parser fixes
* minor fix in request parsing to match path parameters with slashes
* replace get_data() with response.content to fix logic in SNS HTTP subscriptions
* fix gzip handling in SSL proxy

Change Log

1. New Features

* add initial ASF (AWS Service Framework) SQS provider implementation
* add automatic Docker network detection for Lambda containers
* add support for dynamic SSM/SecretsManager references in CloudFormation templates
* add version info to internal health HTTP endpoint

2. Enhancements

* add support for gzipped responses in SSL proxy via Accept-Encoding headers
* lowercase 'accept' headers in API Gateway for parity with AWS
* add negative test for SNS FIFO topic validation, migrate SNS tests from unittest to pytest
* add ability to customize default IAM/STS user via TEST_IAM_USER_ID/TEST_IAM_USER_NAME
* add integration test that passes binary data via APIGateway to Lambda
* refactor StepFunctions multi-region support to use upstream changes
* add proper validations for SNS/SQS integration with fifo queues
* allow provider overrides to be passed to the started LocalStack container
* ensure SequenceNumber is present in Lambda events from DynamoDB Streams
* support updating of API Gateway resources via PATCH operations
* refactor logging code to replace string interpolation with passing arguments to log methods
* refactor code to remove unnecessary list comprehensions
* refactor code to remove mutable default arguments
* pull out subtypes instance manager into separate util class for reusability
* update and clean up outdated documentation
* replace dict and list function calls to literal syntax
* remove `uname` command in system check for windows compatibility

3. Bug Fixes

* fix API gateway proxy resources
* fix passing of request parameters to API Gateway HTTP integrations
* fix Firehose-ElasticSearch integration, allow S3Backup AllDocuments with ElasticSearchDestination
* fix multiple service container creation
* fix extraction of filter values for describing EC2 prefix lists
* fix single-space env default in docker-compose
* fix RenovateBot config to only enable patch updates

Announcements

* **Security fixes**: This release upgrades **log4j** dependencies to version `2.17.0` to fix the critical security vulnerabilities [CVE-2021-44228](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44228), [CVE-2021-45046](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-45046)

Change Log

1. New Features

* add initial support for region-based in-place partition rewriting
* add "--offline" pytest flag, skip "online-only" tests, fix some tests
* add SKIP_SSL_CERT_DOWNLOAD option to allow skipping download of SSL cert

2. Enhancements

* bump version of Java utils to 0.2.18 to fix log4j vulnerability CVE-2021-44228, CVE-2021-45046
* refactor README.md to make it crisp and readable
* refactor Lambda logic to remove local executor Callable from model entities
* automatically pull non-cached Docker images on image inspection
* replace dict calls with comprehensions
* small fixes to allow running LocalStack with podman
* reduce log level of edge port configuration hook
* correctly wait for stay-open port to be available, fix fallback to exec mode
* add compatibility checks for S3 copy object with metadata
* add proper error response message to reject empty SNS messages
* minor refactoring to use negative index -1 to get the last element of sequence
* minor refactoring of Lambda API for better extensibility

3. Bug Fixes

* fix updating of HTTP method in message handler chain to avoid None methods
* fix misc. tests failing for non-default region
* fix Terraform test issue related to SQS changes in the AWS provider
* fix association of VPCs in Route53 HostedZone responses
* fix CloudFormation updates for EC2::Instance with empty SecurityGroups property
* fix listing of KMS signing keys for asymmetric key pairs

Announcements

* Upcoming breaking change: This version introduces the `LEGACY_PERSISTENCE` config, which determines whether the legacy persistence mechanism (based on API calls record&replay) should be used. Currently still defaults to `LEGACY_PERSISTENCE=1`, but the logic will be disabled by default and may get removed entirely in a future release. (Please note: this is only affecting the legacy persistence in Community - if you're using the Pro version, you can ignore this message.)

Change Log

1. New Features

* add LocalStack Package Manager (lpm) CLI
* add plugin-based hooks for bootstrapping and infra startup
* add functionality for custom Elasticsearch backends
* add support for stay-open mode for docker-reuse Lambda executor
* add cli command to show current config
* add initial integration of Route53Resolver API
* add multi-region support for StepFunctions state machines

2. Enhancements

* refactor handler chain and add MessageModifyingProxyListener to enable modification of requests/responses
* add Directories config object and introduce directory structure
* use unittest.mock.patch.object for overriding config params
* publish logs per metric and remove faulty metric filter behavior in CloudWatch API
* add Docker tags for minor and patch versions
* exclude Elasticsearch >=7.14.0 client versions for OpenSearch compatibility
* add SIGINT signal handler for CLI to enable clean shutdown across different operating systems
* clean up obsolete patches for event_rules in EventBridge API
* slightly refactor StepFunctions install logic for better extensibility
* add flag to check port availability in start_proxy_server(..) to fix test flakes
* increase timeout when waiting for edge port to become available
* allow redirection of var libs to static libs inside the container / move dependencies to correct folders
* update startup hook that initializes and restores persistence
* download test-jar for Lambda integration tests with `make init-testlibs`
* filter warnings about "tail unrecognized file system" from Lambda logs
* add basic Architectures support to Lambda API
* get hostname by using gethostname() instead of reading /etc/hostname
* add patch utility
* add .localstack config directory and config profiles
* add initial support for KMS Sign API
* skip time expiration validation for presigned S3 URL when `S3_SKIP_SIGNATURE_VALIDATION=True`
* remove dead event logging code
* move machine id cache to new cache directory
* add missing attributes for Lambda::Function CloudFormation resources
* add "is running" check before restarting Kinesis and StepFunctions services
* add module init file for localstack.runtime
* add support for SNS delivery logs stored to CloudWatch Logs
* explicitly handle S3 OPTIONS request with "Access-Control-Request-Method" header
* add psutil as CLI requirement
* strip extra xmlns attributes in PutBucketNotification responses for AWS Rust SDK compatibility
* update Makefile to use new cli detach flag for smoke test
* patch CFN stack outputs for API Gateway
* make can_use_sudo use non-interactive mode
* migrate several tests from unittest -> pytest
* re-arrange Dockerfile commands to decrease image size and increase number of cache hits
* pin Docker base images, enable DependaBot/RenovateBot to update once a week
* apply boolean lowercase conversion in IAM responses for all clients (not only Node.js SDK)
* enable Docker buildkit cache inlining, use remote Docker layer cache
* patch `LogStream.filter_log_events` to use `get_pattern_matcher`
* minor refactoring of EventBridge utils, fix location of `EVENTS_TMP_DIR`
* minor: improve parameter checks and error responses for SES SendEmail

3. Bug Fixes

* fix returning formatted date string for requestTime in Lambda API GW events
* fix edge routing for API Gateway invocation URLs when Signature= query param is passed
* fix SNS pagination to support large CloudFormation stacks with very large number of topics
* fix lpm to return non-zero exit code if package installation fails
* fix using custom Docker images for nodejs14.x/python3.9 only if no custom registry is provided
* fix MD5 check on S3 requests with "chunk-signature="
* fix "localhost" region in requests headers for compatibility with NoSQL Workbench
* fix Docker flags parsing in configure_container
* fix tests and remove obsolete patch for CloudWatch metrics filtering
* fix setting HMAC/non-HMAC credentials when injecting internal Authorization headers in requests
* fix OldImage/NewImage in DynamoDB->Kinesis event payloads on updating/deletion of items
* fix creation of multi route table associations in CFN EC2::SubnetRouteTableAssociation resources
* fix deleting objects from non-existing S3 buckets
* fix text/xml content-type header in STS responses
* fix install_go_lambda_runtime for multi-arch build
* fix forwarding of unprintable chars to SQS DLQ
* upgrade pyopenssl version to fix OpenSSL issue, refactor SSL cert generation
* minor fix checking for dict/CaseInsensitiveDict in merge_recursive(..) util function