[1.11.0](https://github.com/ladybug-tools/queenbee/compare/v1.10.0...v1.11.0) (2020-05-21)
Bug Fixes
* **baked recipe:** fix inherited validators to work with recipe and baked recipe ([f0676cb](https://github.com/ladybug-tools/queenbee/commit/f0676cb503aa7ca2bf3b86cce1f0bf53218fbb35))
* **cli:** clean code and fix docs ([d8169e7](https://github.com/ladybug-tools/queenbee/commit/d8169e722717c601bdd0c0d7e083a54aeb9617e2))
* **cli:** replace recipe dependency with simple install ([ec1eafd](https://github.com/ladybug-tools/queenbee/commit/ec1eafd81b653c854ded11ea8465a59fcc315474))
* **conf.py:** create `_static/schemas` folder if doesn't exist ([b209fa6](https://github.com/ladybug-tools/queenbee/commit/b209fa6d9a3ae90d1a0a4d25ff8e2ef337765da6))
* **dependency:** fix package_url ([5b6b0c0](https://github.com/ladybug-tools/queenbee/commit/5b6b0c0f21e571f6f43d9536cb85c2a55ccdb9f7))
* **function:** fix command validation ([95001eb](https://github.com/ladybug-tools/queenbee/commit/95001ebaff6f038d5a697f0266e2e20b7c175b99))
* **functions:** reference value check ([7671336](https://github.com/ladybug-tools/queenbee/commit/7671336a7c6c5a2bc7100e96c446080b85823497))
* **lists:** sort schema lists deterministically ([84143e3](https://github.com/ladybug-tools/queenbee/commit/84143e3a67bf13f5ffedfe05948bc0210d938ce6))
* **operator:** use snake_case for schema keys ([1f1ce5a](https://github.com/ladybug-tools/queenbee/commit/1f1ce5a2adc9d8ddb845f95dec96ce5be22e8ea0)), closes [#50](https://github.com/ladybug-tools/queenbee/issues/50)
* **packaging:** lenient packaging path location ([85264ab](https://github.com/ladybug-tools/queenbee/commit/85264ab0ef2bf4919143810254d4abb6251f0b46))
* **recipe:** better validation and validation messages ([7af3c2d](https://github.com/ladybug-tools/queenbee/commit/7af3c2d3f5ebfaceac3db49eb6304335cfe2a122))
* **recipe:** check template names ([26783d7](https://github.com/ladybug-tools/queenbee/commit/26783d7a73131d362184bbaea6d77c1d72408f56))
* **recipe:** don't lock dependencies when writin folder ([00f26d9](https://github.com/ladybug-tools/queenbee/commit/00f26d916b59a046805e591140cedd462fddde2c))
* **recipe:** fix dependency lock and Windows separator ([d509da7](https://github.com/ladybug-tools/queenbee/commit/d509da7e0fe371ac1c172c5637d8bb96462b1560))
* **recipe:** fix entrypoint check to work with classes inheriting from Recipe ([4b7d8e8](https://github.com/ladybug-tools/queenbee/commit/4b7d8e80f97c6f2565025cbb7900b085bf0bb2b0))
* **recipe:** read/write from/to folders ([c65ef7f](https://github.com/ladybug-tools/queenbee/commit/c65ef7fd8a32503720a17034b2ed973181add2c8))
* **repository:** generate from folder, enable merge ([4425365](https://github.com/ladybug-tools/queenbee/commit/44253657c81e4502b817e679367665896998f131))
* **serialization:** remove "Enum" classes from dict and pass exclude_unset option ([3cf2e22](https://github.com/ladybug-tools/queenbee/commit/3cf2e22109d4f5fe5f96e6b6301c218437e62355))
* **workflow:** nest recipe inside of workflow instead of inheriting ([b6e0542](https://github.com/ladybug-tools/queenbee/commit/b6e0542a5ed0850fae6f954a5c05b50795ec4b9c))
Features
* **cli:** add repository serve and init commands ([7f8705f](https://github.com/ladybug-tools/queenbee/commit/7f8705ff810cda48a46ef6c82eb808dadd741120))
* **cli:** commands to generate/manage recipes, operators and repositories ([6448627](https://github.com/ladybug-tools/queenbee/commit/6448627b7796124815403195dd1569f5ca3f793e)), closes [#47](https://github.com/ladybug-tools/queenbee/issues/47)
* **functions:** update function and operator schema ([b596a5b](https://github.com/ladybug-tools/queenbee/commit/b596a5b0fdeaf534295ae6e6280e749338eaedef))
* **schema:** mega refactor! ([1171778](https://github.com/ladybug-tools/queenbee/commit/11717786633bbeed4e3ded3a62f07ba802c15cc3)), closes [#39](https://github.com/ladybug-tools/queenbee/issues/39) [#41](https://github.com/ladybug-tools/queenbee/issues/41) [#42](https://github.com/ladybug-tools/queenbee/issues/42)
* **schemas:** create repository schema, read/write from folder and create archives ([1cb53e5](https://github.com/ladybug-tools/queenbee/commit/1cb53e559178d08dcd224a8f969baa912dbc0ff3)), closes [#47](https://github.com/ladybug-tools/queenbee/issues/47) [#42](https://github.com/ladybug-tools/queenbee/issues/42)
* **workflow:** generate workflow from a baked recipe ([183561a](https://github.com/ladybug-tools/queenbee/commit/183561af418500fed5252c2f804872767738a6fc))
Performance Improvements
* **folders:** write yaml folders without empty values ([9f3e9d7](https://github.com/ladybug-tools/queenbee/commit/9f3e9d7177531be2c268685bd73c4bbcd337e40e))
Reverts
* **artifact location:** kill it with :fire: ([194862e](https://github.com/ladybug-tools/queenbee/commit/194862ec6964582d6026772f91d4b7fd8e7da7f9))
Details
Separate `Workflow` into `Operators`, `Recipes` and `Workflows`
Two main issues led us to split the old `Workflow` object into three separate objects:
* Confusion between Workflow template and Workflow object executed by Argo or Luigi 39
* Re-usability of Workflows
For this reason we introduce the three following Queenbee Objects:
* **Operator:** A list of `functions` (templated commands) for 1 cli tool (eg: `radiance-operator` would template command for radiance)
* **Recipe:** A reusable DAG definition of Operator Functions or other Recipes (42) (ie: a recipe can use another recipe inside of itself)
* **Workflow:** A Recipe + Input Arguments run by an executor (ie: luigi or argo). The schema will not be used by Queenbee explicitely but serves as a common definitions or interface for the different execution engines to report progress of an executed Recipe.
Introduce Dependency Management
Despite [all warnings](https://medium.com/sdboyer/so-you-want-to-write-a-package-manager-4ae9c17d9527) we went ahead and built a package manager... This was a requirement for `Recipes` to be shareable and modular (ie: a recipe could refer to a certain operator without having to copy all the operator data).
To do so we drew heavy inspiration from the [Helm project](https://helm.sh/docs/topics/chart_repository/) (#47 ). This means that Queenbee packaging and sharing remains open sourced and freely available for those who wish to create their own registries of Recipes and Operators.
Artifact Path Context (remove `artifact_location`)
Artifact Locations have been removed to make recipes more reusable. Hard coding the location to persist a file inside a DAG lead to re-usability issues when nesting Recipes inside each other. It was therefore preferable to establish the concept of `artifact path context`:
Function Path Context
When an artifact path is declared within an `Operator Function`, this path is only relevant or valid in the context of this function. In practice this means that a Function doesn't care where an artifact is persisted when it is executed because it knows that the executor (argo or luigi) will move copy the artifact to the path it expects it to be in the command.
yaml
name: ray-tracing
description: Run ray tracing using some input data!
inputs:
parameters:
- name: radiance-parameters
description: a string of radiance parameters
default: -b 5
artifacts:
- name: grid
path: grid.pts
- name: scene-file
path: scene.oct
command: rtrace -I -h {{inputs.parameters.radiance-parameters}} scene.oct < grid.pts > grid.res
outputs:
artifacts:
- name: result-file
path: grid.res
Recipe Path Context
When an artifact path is declared in a `Recipe DAG` it points to a folder or S3 bucket location where it expects the artifact to be pulled from (`input`) or pushed to (`output`). This path does not overwrite the one in the `Operator Function` context. Instead the executor (argo or luigi) will copy the artifact from the `Recipe Context` to the `Function Context` for `inputs` and vice versa for `outputs`
yaml
name: main
inputs:
parameters:
- name: sensor-grid-count
description: The maximum number of grid points per parallel execution
default: 100
- name: radiance-parameters
description: The radiance parameters for ray tracing
default: -I -ab 2 -h
artifacts:
- name: model
description: A Honeybee model with radiance properties
required: true
- name: input-grid
description: A grid file
required: true
tasks:
...
- name: daylight-factor-simulation
template: honeybee-radiance/ray-tracing
dependencies:
- split-grid
- create-octree
loop:
from:
type: tasks
name: split-grid
variable: grid-list
sub_folder:
- item.name
arguments:
parameters:
- name: radiance-parameters
from:
type: inputs
variable: radiance-parameters
artifacts:
- name: grid
from:
type: tasks
name: split-grid
variable: output-grids-folder
subpath: '{{item.name}}'
- name: scene-file
from:
type: tasks
name: create-octree
variable: scene-file
outputs:
artifacts:
This file is persisted in a loop sub-folder so the name won't clash
- name: result-file
path: grid.res
...
outputs:
artifacts:
- name: data
from:
type: tasks
name: post-process
variable: post-process-folder
parameters:
- name: average
from:
type: tasks
name: post-process
variable: daylight-factor-average
Overhaul Documentation
The previous documentation page consisted of a Redoc template pointing to the old `Workflow` schema. As we have introduced new separate schemas as well as development practices for Operator and Recipe creation/management we needed to introduce some written tutorials and more in depth exploration pages for each schema. We therefore overhauled the Redoc page and went for a full Sphinx page that matches styling from other ladybug tools packages.