Ecs-composex

============================

Improving environment variables settings and setup autoscaling on DynamoDB Table and Indexes.

New features
-------------

* ebbe1af feature: x-dynamodb.Scaling (615) (John Preston)
* 063f9ee Support services.environment: x-resource attribute mapping (609) (John Preston)

Improvement / Maintenance
--------------------------

* 24e9684 Updated dependencies (John Preston)


Fixes
---------

* 96a2687 Updated deps (John Preston)
* 141ef32 Fix x-rds.Lookup SecretsMappings (John Preston)

====================

New features
--------------

* 063f9ee Support services.environment: x-resource attribute mapping (609) (John Preston)

=====================

New Features
--------------

* 3d8d602 New: x-kinesis_firehose (603) (John Preston)

Fixes
--------

* 67ec8d8 Patching up documentation for x-route53 (John Preston)
* f160319 Fix x-rds instance Engine properties and secrets mappings (598) (John Preston)
* 6f5e488 Fix no x-vpc defined in file to upgrade (John Preston)

=====================

It has been a long time since 0.17 has been released, and the subsequent patch releases have been resilient enough
to guide us to this point.

So May the 4th be with us all on this new release, packed with bug fixes, new features, and more to come!

New features
----------------

Some very exciting new features have come into this new version, and although only one new AWS Resource has
made it to the project, the most exciting change is the use of a module manager which if going to dynamically
load the core ECS Compose-X modules as well as extensions that anyone can write on their own, to support
further use-cases.

* c36a853 Adding documentation on creating new modules Further docs corrections (John Preston)
* 91a3962 Adding -p option, equivalent to -n, for project name (John Preston)
* eafdbb4 Adding label on tags parameters (John Preston)
* 47021cd TaskCompute class to manage CPU/RAM settings (John Preston)
* f1bdb5f Added x-kms to x-sqs support (John Preston)
* 8cb0edd x-cloudmap for x-resources (584) (John Preston)
* b953169 Allows to define ports to allow for ext_sources and aws_sources (582) (John Preston)
* bfb1a74 Added feature for RuntimePlatform Task definition settings (John Preston)
* 5a300eb Route53 stack created for records (John Preston)
* 9525f05 Added x-kms:: mapping to S3.BucketEncryption (John Preston)
* b1d6de2 Added x-neptune.Lookup (565) (John Preston)

Breaking changes
--------------------

This new version comes with a few breaking changes :

* Deprecation of ``Use`` for resources, such as x-vpc, x-cluster, x-s3 and so on. The Use functionality was too limited.
* Deprecation of ``x-dns``, replaced with ``x-route53`` and ``x-cloudmap``
* ec24dd1 Remove prefix list given max size is immutable when set (John Preston)

To help with the transition to using the 0.18 version, an upgrade script has been created.

To use it, simply do

.. code-block:: bash

python3 -m venv compose-x
source compose-x/bin/activate
pip install pip poetry -U
git clone https://github.com/compose-x/ecs_composex
cd ecs_composex/; poetry install
./upgrade_scripts/upgrade_to_0.18.py -h

for example
./upgrade_scripts/upgrade_to_0.18.py -f docker-compose.yaml


Geneal Improvements
-----------------------

These improvements have been made to make ecs-compose-x more reliable and consistent at validating
itself and getting closer to a proper production-grade tool.

* 9d9c57e Simplified JSON schema loading (John Preston)
* cb76c1a Using pyupgrade pre-commit hook (John Preston)
* e7ea8f0 (John Preston)
* Allowing Env resources with _to_ecs to apply
* Enforce x-cluster deprecation of Use * Improve migration script
* f3518be Refactoring x-route53 code into smaller modules (John Preston)
* ef0bca0 Lint code. Change x-cluster to add the exec bucket/key into x-s3/x-kms (John Preston)
* 1860d43 Linted code (John Preston)
* ac27461 Refactor to use modules more and cleanup params RES/MOD key (John Preston)
* 0b5d87b (John Preston)
* Refactored resources stack to use the module from manager
* Refactored x-cluster bucket/kms key to use x-kms/x-s3 properly
* Fixed up use case tests * Refactored x-sns to not use x-sns.Topics{}
* a6b3685 Refactors and renames of ecs packages to improve ECS Family configuration (John Preston)
* c33d63f Using published first, target second when creating ingress rules (589) (John Preston)
* bc2787d Refactor schemas files to be within module (587) (John Preston)
* e32f92b Updated deps and NOTICES (John Preston)
* a11254b Simple upgrade script to 0.18 syntax (John Preston)
* bfab153 Updated test files with upgrade script (John Preston)
* ff6acd4 When secret JSON keys given, only expose those, remove default secret value (John Preston)
* 97907a1 Precaution for Name value in x-events (John Preston)
* 43c24be Removing tests for deprecated feature (John Preston)
* d558645 No more Zones defined in settings (John Preston)
* d7233b1 Refactored x_dependencies for x-rds (John Preston)
* b6d57de Updated JSON Specs (John Preston)
* 8f3b4b8 Refactored function to link x-resource to services for IAM and environment variables. Added typing for resource to service linking Link resource to services function deals with new vs lookup on its own (John Preston)
* 52d0771 Testing troposphere 4.0.0 beta Refactored env vars, only the Ref value is exposed by default (John Preston)
* 3b41ad6 Refactored to_ecs for RDS like resources (John Preston)
* 926ce99 Refactor x-alarms to x mapping (John Preston)
* 77b9dbd Refactored x_dependencies for x-rds (John Preston)
* 39ef236 Using retry on CFN validate template (John Preston)
* 29fea25 Updating CICD. Macro will be moved elsewhere (John Preston)
* b52a568 Updated neptune for creation and added test case (John Preston)
* 3c9cc03 Reworked lookup resources.kms policies assignment (John Preston)
* 8d345b1 Simplified _to_ecs functions and added tests cases (John Preston)
* 9ec1dde Refactored x-s3 to use generic IAM policies functions (John Preston)
* 233d973 Strenghtening Lookup JSON model (John Preston)
* 14bcb48 pre-commit cleanup (John Preston)
* 5057944 Updated copyright dates (John Preston)
* c55e27c Updated userpool mappings (John Preston)
* ce6b049 Updated ACM, cloudmap and other settings (John Preston)
* 361ac79 Reworked x-route53 with ACM and ELBv2 (John Preston)
* 78e3ced Reworked x-dns to x-route53 and x-cloudmap (John Preston)
* 3de79c5 Refactoring ELBv2 for external support (John Preston)
* 0cf307a Reworked ECS IAM Roles and Family init (John Preston)
* 9891e4f Reworking the XResources classes (John Preston)
* 2df8b24 Re-instating default PrivateNamespace to support all DNS features (571) (John Preston)
* 2745038 Refactoring / cleaning the compose and ECS services related settings (568) (John Preston)
* 4bac941 Use official nginx-prometheus-exporter image (570) (Luca Comellini)


Bug Fixes
----------

A number of these bug fixes are the result of changes in the general improvements above,
which mostly were due to restructuring of the code and classes.

* fcddf63 Fix ECS Log group name (John Preston)
* bf44bfd Fixed x-cluster logging configuration (595) (John Preston)
* cbd1546 Fix for x-route53 circular imports (John Preston)
* c6c5db6 Fix for duplicate secret var names (John Preston)
* 690c55a Fixing x-rds.Lokup.db and x-neptune.Lookup (593) (John Preston)
* 88f0697 Fix networks{} to subnets association (John Preston)
* 023a555 Fix cloudmap to ecs (John Preston)
* 84c7cc5 Fix RAM GB conversion to MB (John Preston)
* a445e6b Fix imports (John Preston)
* a96d565 Network feature and compute settings fixes. (591) (John Preston)
* 2cfd6f3 Fixing logging. Working traefik public e2e (John Preston)
* 431309d Fixing code smells (John Preston)
* eb432a6 Fix Launch Type and set it early Fix min CPU for ECS Auto-fix feature for ECS cluster providers Common class for sidecars (John Preston)
* 4ce25d9 Split refactor of ecs_prometheus and sidecars (John Preston)
* ee5d386 Fixing a non-problem for non-secret value (John Preston)
* 23ec4c8 Split x-elbv2 into modules and fix for env vars (John Preston)
* fd1d0bc Fix services add and split-refactor compose.x_resources (John Preston)
* ea9b56f Fixing port mappings, adding protocol support and fargate default (588) (John Preston)
* f00f0af Fix services scaling and improve input validation (John Preston)
* 72f1cd2 Fix x-events input from services output (John Preston)
* 5a3a92e Fixing up condition where template is in fact not needed (John Preston)
* ab327cb Align the code to the JSON Schema specs (John Preston)
* 1798a25 Fix x-events multi events on same service and efs bug (John Preston)
* db57b4b Fixed x-alarms to x-elbv2 Dimensions (John Preston)
* 69db70a Fixing up RDS and DB Version for testing (John Preston)
* b4a8e5d Fixed ELBv2 - Alarms (John Preston)
* 0c0e60c Fixed ELBv2 - Cognito mapping (John Preston)
* 6fec8fc Fix and simplified resource to services container env vars (John Preston)
* dc351a8 Fix SSM ARN parameter (John Preston)
* a09f17e Fixing volumes settings and handle host config (John Preston)
* d2de0a6 Fixes in RDS like resources (John Preston)
* 73f43d1 Fixed up x-acm to x-elbv2. Got generic algorithm for x-to-x resources (John Preston)
* 96ad4c5 Fixes and log formatting (John Preston)
* 6edf0f2 Fix x-ssm_parameter (John Preston)
* 3980975 Fixed x-events and x-elbv2 (John Preston)

====================

This new release comes with a lot of changes and fixes that aim to both give more CFN native support and equally
allow for future features to be integrated in a better way.

A lot of changes on the modules one want to implement to support new AWS resources is greatly simplified.


Breaking changes
-----------------

Compute platform options
^^^^^^^^^^^^^^^^^^^^^^^^^^

In this new version we have deprecated the --spot-fleet (501) option. Users who want to use EC2 for the deployment
of their services will be in charge of settings it all up on their own to fit their requirements.

This was then done after implementing (500) which will allow through Lookup to detect the ECS Cluster compute
settings and automatically set the Launch Platform for the services appropriately. Users can, if they have
a cluster with multiple capacity providers override and set what capacity provider to use for the service.
Again, if that is not available in the cluster (using Lookup) then it will either fail or fallback to a working
capacity provider.


IAM stack created at the root
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This change **should not** be incurring any change to existing stacks **if the IAM permissions were stricly managed under Compose-X**.
A new *iam* stack is created with all the IAM roles of the services (Task and Execution Role) which is then passed on to the other
nested stacks that will need it.

This change is necessary for upcoming features support and changes.

That change also helps with a simpler way to manage IAM policies for the roles and a more flexible way to set permissions that does
not require to wait for the service nested stack to complete to possibly set IAM permissions or get the roles names/arns.


New features
-------------

* 2fcd1ba Added x-alarms and x-elbv2 cross-support Updated x-alarms docs (John Preston)
* 65d4123 Feature x-ecs enable exec (539) (John Preston)
* 51a4628 Allow to add custom and predefined bucket policies (John Preston)
* 599b5ad Refactor IAM to set roles in their own stack (532) (John Preston)
* c9564ee Added support for Conditions in Listener target to cover native CFN implementation (John Preston)
* 8bdf95d Allowing for resource to have an ARN extension from policy definition (535) (John Preston)
* b39859c Added option to store SSM parameter as Base64 to avoid invalid characters (527) (John Preston)
* 3cc0908 x-elbv2 Target Group Attributes support (518) (John Preston)
* 41d393f Feature - docker working_dir (509) (John Preston)
* ee82fef Feature services.x-ecs to enable services level capacity providers (507) (John Preston)
* 5ec5c1f Option to enforce secure connection to S3 bucket for IAM role (504) (John Preston)
* f5ca17d Added ecs.ephemeral.storage label to extend Fargate local storage (503) (John Preston)
* 6fe4880 JSON validation for more x-resources (502) (John Preston)
* a399344 Deprecating --spot-fleet (501) (John Preston)
* 0141544 Adds deploy label ecs.compute.platform to override from FARGATE (500) (John Preston)


Fixes
-------

* 7202741 Fix release 0.16.10 (530) (John Preston)
* 84c3716 Fix ACM and Rules conditions (529) (John Preston)
* 8c20f82 Fix SSM ARN and volumes path (528) (John Preston)
* aac5811 Fix release/0.16.9 (526) (John Preston)
* 5cd37a5 Fixing IAM issues (525) (John Preston)
* 455b3d4 Fix release/0.16.8 (524) (John Preston)
* 84cd54d Fix missing lookup mappings (523) (John Preston)
* 760e804 Fix release - v0.16.7 (522) (John Preston)
* c8e8882 Fix for kms key alias in x-s3 Lookup (521) (John Preston)
* f223bb5 Fix for x-kms Lookup (John Preston)
* 900b03e Fix release 0.16.4 (517) (John Preston)
* b9c4ac5 Fix settings typo (John Preston)
* 79012d7 [FIX] ecs placements condition (514) (John Preston)
* 898ec07 Fix release to 0.16.3 (511) (John Preston)
* 88ce644 Fixes for ECS and Cognito (510) (John Preston)
* 197bf3b Fixing docker-compose commands for any env (John Preston)


Improvements
--------------

* 84f7216 Update deps (John Preston)
* f8bdd6c When services have an expose set of ports, allowing self-ingress (John Preston)
* 403a652 Updated docs (John Preston)
* decd5c2 Flake8'd the code (John Preston)
* 2fcd1ba Added x-alarms and x-elbv2 cross-support Updated x-alarms docs (John Preston)
* 22cbd5a Import ecs_composex_specs schemas in main application (538) (John Preston)
* ea510b4 Policies Uniqueness (John Preston)
* 6a6409d Working pre-defined bucket policies (John Preston)
* 1820fc3 Ensures the IAM policies are created before the service is (534) (John Preston)
* 887d31b Reworked docs and added to docs (519) (John Preston)
* bfed008 Refactor the services input (516) (John Preston)
* 66e2733 Update issue templates (John Preston)
* dc7b713 Refactor - perform x-resources.Lookup early (508) (John Preston)
* 8dfa8a3 Adding non-docker command (John Preston)
* 40153f7 Using poetry to manage dependencies (John Preston)

======================

This release adds features that revolve around the monitoring of applications by supporting
Prometheus and AWS CloudWatch integration integration making it easy for users to collect metrics
for Prometheus enabled applications.

It also improves the docker experience for users that wish to automatically use the docker image digest instead of tags,
and for users of AWS ECR, allows to perform an image scan of the indicated image prior to continue the process.

Fixed docs and trying to steer towards a CLI usage with docker for new starters to avoid python environments problems.

New features
--------------

* b81d444 x-alarms schema validation (494) (John Preston)
* 604dbfe docker images digest substitution (492) (John Preston)
* a2c740e Battletesting prometheus and EMF processors (491) (John Preston)
* f19899b Adding docs, tests, and lib dependency to enable prometheus (488) (John Preston)
* c67d3c9 Adding some prometheus support (472) (John Preston)
* d92a1ca Adding x-ssm_parameters macro parameters (John Preston)
* e956203 Better ECR display conditions (John Preston)
* ae10da7 Add successful notice output for ECR Scan (John Preston)
* b307f6d Feature - x-ssm (486) (John Preston)
* de345c1 Feature x-ecr interpolate digest (482) (John Preston)
* c448650 Adding ECR Scan at execution time (478) (John Preston)
* cd441d8 Adding x-dashboards feature (476) (John Preston)
* 6c2e95e Cognito ALB app profile creation (475) (John Preston)
* 9ae02ff Feature - Lookup codeguru profiler (468) (John Preston)
* 56156b0 Using codeguru at top level (462) (John Preston)
* 1e6016a CLI Feature: `plan` (459) (John Preston)

Improvements
-------------

* 381aab9 Adding ECR Scan reporter lib to CLI and macro (John Preston)
* 3650792 Matching PEP0440 RC syntax (John Preston)
* dc903fb Changing docs theme (John Preston)
* 25b3e7b Using poetry env commands to make life easier (John Preston)
* 507b917 Pyproject black settings update (John Preston)
* 024852f Fixes and new features to help with life comfort (John Preston)
* 2d778d2 Updated deps (John Preston)
* fadab75 Updating dependency (John Preston)
* fd99dbb Using more of common compose_x lib (John Preston)
* cef8f1e Removing cognito init override (John Preston)
* 6aa54ea Not using sphinx-material to generate sitemap.xml (John Preston)
* b5e1d63 Using common lib for keyisset and keypresent (John Preston)
* 043d787 use Poetry and pyproject.toml (483) (John Preston)
* 7aff79e Added x-ecr docs for scans (479) (John Preston)
* 79c3346 Addind DL stats. (John Preston)
* 280b0f6 Newer docker image source (John Preston)
* dbfd70c Docs improvements (467) (John Preston)


Fixes
------

* 923ee23 Fixing docs(497) (John Preston)
* 2a7cc3e Adding exception for bucket init creation in us-east-1 (496) (John Preston)
* 8c6a159 Fix/subnets must belong to same vpc (493) (John Preston)
* d2d9ba4 Fix missing return and outputs for new SSM Parameters (John Preston)
* 42b442f Fix docs buildspec (John Preston)
* 6297604 Fix layer buildspec (John Preston)
* 7c72014 Fixing build for docs and manifest (John Preston)
* 7da9538 Indentation fix (John Preston)
* 2390d23 Fixing loop and scan report return (480) (John Preston)
* 14de30c Fixing setup.py for extra (John Preston)
* 3b30cf7 Fixing pyproject version (John Preston)
* 3131973 Bug fixes (473) (John Preston)
* 40d0195 Fixed missing env vars via lookup (466) (John Preston)
* 6306ed0 Fixing S3 perms bug and adding s3 to JSON specs (464) (John Preston)