v0.7 includes a major change to how Calliope internally operates.
Along with this, there are multiple changes to how Calliope models are defined and configured.
This requires adapting models to work with 0.7.
We group changes into those that are primarily user-facing and relevant for all Calliope users, and those that are primarily internal, and relevant only for Calliope developers.
User-facing changes
This section gives a brief summary of changes.
For more detail, see our migrating from v0.6 to v0.7 section in our [documentation](https://calliope.readthedocs.io/en/latest/migrating/).
|new| Storage buffers available in all technology base classes.
|new| Multiple carriers and different carriers in/out available in all technology base classes.
|new| `node_groups` added to match `tech_groups`.
|new| technology efficiencies split into inflow and outflow efficiencies.
|new| Technology capacities and efficiencies can be differentiated between technology carriers.
|new| Parameters can be defined outside the scope of `nodes` and `techs` using the top-level `parameters` key in YAML.
|new| Any parameters can be indexed over arbitrary dimensions, both core Calliope dimensions and new, user-defined dimensions.
|new| Non-timeseries data can be loaded from CSV files or in-memory Pandas DataFrames using the top-level `data_sources` key in YAML.
|new| User-defined mathematical formulations using the new Calliope math syntax can be loaded when creating a model.
|new| Shadow prices obtained from a dual LP problem can be read by using `model.backend.shadow_prices.get("constraint_name")`.
|changed| |backwards-incompatible| Updated to support Python versions >= 3.10.
|changed| |backwards-incompatible| Updated to Pandas >= v2.1, Pyomo >= v6.4, Xarray >= v2023.10.
|changed| |backwards-incompatible| Flat technology definitions, removing the distinction between `essentials`, `constraints` and `costs`.
|changed| |backwards-incompatible| Timeseries data is defined under the `data_sources` top-level key, not with `file=`/`df=` at the technology level.
|changed| |backwards-incompatible| Demand and carrier consumption values are strictly positive instead of strictly negative.
|changed| |backwards-incompatible| `model.run()` method → two-stage `model.build()` + `model.solve()` methods.
|changed| |backwards-incompatible| `model` and `run` top-level keys → `config.init`/`.build`/`.solve`.
|changed| |backwards-incompatible| `locations` top-level key and `loc` data dimensions → `nodes`.
|changed| |backwards-incompatible| `parent` technology parameter → `base_tech` + `inherit`.
|changed| |backwards-incompatible| Cost parameters are flattened and use the indexed parameter syntax.
|changed| |backwards-incompatible| `links` top-level key → transmission links defined in `techs`.
|changed| |backwards-incompatible| Various parameters/decision variables renamed (namely `energy_` → `flow_`, `carrier_prod`/`_con` → `flow_out`/`_in`, and `resource` → `source_use`/`sink_use`).
|changed| |backwards-incompatible| Various configuration options renamed.
|changed| |backwards-incompatible| `force_resource` technology parameter → `source_use_equals` / `sink_use_equals`.
|changed| |backwards-incompatible| `units` + `purchased` decision variables → `purchased_units`.
|changed| |backwards-incompatible| Parameters added to explicitly trigger MILP and storage decision variables/constraints.
|changed| |backwards-incompatible| Structure of input and output data within a Calliope model updated to remove concatenated `loc::tech::carrier` sets and technology subsets (e.g. `techs_supply`) in favour of sparse arrays indexed over core dimensions only (`nodes`, `techs`, `carriers`, `timesteps`).
|changed| |backwards-incompatible| `coordinates.lat`/`lon` node parameter → `latitude`/`longitude`.
|changed| |backwards-incompatible| Distance units default to kilometres and can be reverted to metres with `config.init.distance_unit`.
|changed| |backwards-incompatible| `operate` mode input parameters now expected to match `plan` mode decision variable names (e.g., `flow_cap`).
|changed| |backwards-incompatible| Cyclic storage is defined per-technology, not as a top-level configuration option.
|changed| Documentation has been overhauled.
|removed| `_equals` constraints.
Use both `_min` and `_max` constraints to the same value.
|removed| `x`/`y` coordinates.
Use geographic lat/lon coordinates (in `EPSG:4326` projection) instead.
|removed| Comma-separated node definitions.
Inheriting duplicate definitions from `node_groups` instead.
|removed| `supply_plus` and `conversion_plus` technology base classes.
Use `supply` and `conversion` technology base classes instead.
|removed| `carrier` key.
Use `carrier_in` and `carrier_out` instead.
|removed| `carrier_tiers` and `carrier_ratios`.
Use indexed parameter definitions for `flow_out_eff` and your own math instead.
|removed| `calliope.Model.get_formatted_array`.
The Calliope internal representation of data now matches the format achieved by calling this method in v0.6.
|removed| Group constraints.
Use your own math instead.
|removed| Configuration options for features we no longer support.
|removed| Plotting.
See our documentation for example of how to visualise your data with Plotly.
|removed| Clustering.
Cluster your data before creating your Calliope model.
Mapping of timeseries dates to representative days is still possible.
Internal changes
|new| Automatic release uploads to PyPI and new accompanying pre-release pipeline.
|new| Choice of issue templates.
|new| YAML schema to catch the worst offences perpetrated in the model definition / configuration.
This schema is also rendered as a reference page in the documentation, replacing `defaults`/`config` tables.
|new| The model mathematical formulation (constraints, decision variables, objectives) is stored in a YAML configuration file: `math/base.yaml`.
Equation expressions and the logic to decide on when to apply a constraint/create a variable etc. are given in string format.
These strings are parsed according to a set of documented rules.
|changed| Development environment installation instructions (they're now simpler!).
|changed| Documentation has been ported to Markdown pages and is built using MKDocs and the Material theme.
|changed| Pre-processed model data checks are conducted according to a YAML configuration, instead of a hard-coded set of python functions.
An API will be created in due course to allow the user to add their own checks to the configuration.
|changed| Costs are now Pyomo expressions rather than decision variables.
|changed| When a model is loaded into an active session, configuration dictionaries are stored as dictionaries instead of serialised YAML strings in the model data attributes dictionary.
Serialisation and de-serialisation only occur on saving and loading from NetCDF, respectively.
|changed| Backend interface has been abstracted to enable non-Pyomo solver interfaces to be implemented in due course.
|changed| Repository structure has been re-configured to use the `src` layout, to rely on the `pyproject.toml` configuration file for most config, and to use only `.txt` requirements files (for pip+conda cross-compatibility)
|changed| CI moved from Azure pipelines (back) to GitHub Actions.
|changed| Stronger reliance on `pre-commit`, including a CI check to run it in Pull Requests.