Openfisca-core

- When a variable file is loaded twice in the same python interpreter, make sure the second loading doesn't corrupt the first one.
- This fixes a bug introduced in 21.0.2, which could lead to a corruption of the tax and benefit in rare edge cases

- Fix two bugs that appeared with 21.2.0:
- Properly encode the result of a formula returning an Enum value
- Enable storing an Enum value on disk

New features

- Improve [`holder.get_memory_usage`]((https://openfisca.org/doc/openfisca-python-api/holder.html#openfisca_core.holders.Holder.get_memory_usage)):
- Add `nb_requests` and `nb_requests_by_array` fields in the memory usage stats for traced simulations.

- Enable intermediate data storage on disk to avoid memory overflow
- Introduce `memory_config` option in `Simulation` constructor
- This allows fine tuning of memory management in OpenFisca

For instance:


from openfisca_core.memory_config import MemoryConfig

simulation = ... create a Simulation object

config = MemoryConfig(
max_memory_occupation = 0.95, When 95% of the virtual memory is full, switch to disk storage
priority_variables = ['salary', 'age'], Always store these variables in memory
variables_to_drop = ['age_elder_for_family_benefit'] Do not store the value of these variables
)

simulation.memory_config = config

New features

- Improve `Tracer`:

- Introduce an `aggregate` option in [`tracer.print_computation_log`](https://openfisca.org/doc/openfisca-python-api/tracer.html#openfisca_core.tracers.Tracer.print_computation_log) to handle large population simulations.
- Introduce [`tracer.usage_stats`](https://openfisca.org/doc/openfisca-python-api/tracer.html#openfisca_core.tracers.Tracer.usage_stats) to keep track of the number of times a variable is computed.

- Introduce methods to keep track of memory usage:

- Introduce [`holder.get_memory_usage`](https://openfisca.org/doc/openfisca-python-api/holder.html#openfisca_core.holders.Holder.get_memory_usage)
- Introduce `entity.get_memory_usage`
- Introduce `simulation.get_memory_usage`

- Improve `Holder` public interface:

- Enhance [`holder.delete_arrays`](https://openfisca.org/doc/openfisca-python-api/holder.html#openfisca_core.holders.Holder.get_memory_usage) to be able to remove known values only for a specific period
- Introduce [`holder.get_known_periods`](https://openfisca.org/doc/openfisca-python-api/holder.html#openfisca_core.holders.Holder.get_known_periods)

- Introduce [`variable.get_formula`](https://openfisca.org/doc/openfisca-python-api/variables.html#openfisca_core.variables.Variable.get_formula)

- Re-introduce `taxscales.combine_tax_scales` to combine several tax scales.

Deprecations

- Deprecate `requested_period_added_value` base function, as it had no effect.

Bug fix

- Fix API response encoding from ascii to utf-8
* Improve user message by displaying `UnicodeDecodeError` information

_Note: the 21.0.1 and 21.0.0 versions have been unpublished due to performance issues_

Breaking changes

Change the way enumerations (Enum) are defined when coding variables

Before:

py
HOUSING_OCCUPANCY_STATUS = Enum([
u'Tenant',
u'Owner',
u'Free lodger',
u'Homeless'])


Now:

py
class HousingOccupancyStatus(Enum):
tenant = u'Tenant'
owner = u'Owner'
free_lodger = u'Free lodger'
homeless = u'Homeless'


> Each Enum item has:
> - a `name` property that contains its key (e.g. `tenant`)
> - a `value` property that contains its description (e.g. `"Tenant or lodger who pays a monthly rent"`)

- Enum variables must now have an explicit default value

py
class housing_occupancy_status(Variable):
possible_values = HousingOccupancyStatus,
default_value = HousingOccupancyStatus.tenant
entity = Household
definition_period = MONTH
label = u"Legal housing situation of the household concerning their main residence"



- In a formula, to compare an Enum variable to a fixed value, use `housing_occupancy_status == HousingOccupancyStatus.tenant`

- To access a parameter that has a value for each Enum item (e.g. a value for `zone_1`, a value for `zone_2` ... ), use fancy indexing

> For example, if there is an enum:
> py
> class TypesZone(Enum):
> z1 = "Zone 1"
> z2 = "Zone 2"
>
> And two parameters `parameters.city_tax.z1` and `parameters.city_tax.z2`, they can be dynamically accessed through:
> py
> zone = numpy.asarray([TypesZone.z1, TypesZone.z2, TypesZone.z2, TypesZone.z1])
> zone_value = parameters.rate._get_at_instant('2015-01-01').single.owner[zone]
>
> returns
> py
> [100, 200, 200, 100]
>
>

Change the simulation inputs and outputs for enumeration variables

Web API and YAML tests

- When setting the value of an input Enum variable for a simulation, the user must now send the **string identifier** (e.g. `free_lodger`).
- The item index (e.g. `2`) is not accepted anymore
- The value (e.g. `Free lodger`) is not accepted anymore.

- When calculating an Enum variable through the web API, the output will now be the string identifier.

Python API

- When using the Python API (`set_input`), the three following inputs are accepted:
- The enum item (e.g. HousingOccupancyStatus.tenant)
- The enum string identifier (e.g. "tenant")
- The enum item index, though this is not recommanded.
- If you rely on index, make sure to specify an `__order__` attribute to all your enums to make sure each intem has the right index. See the enum34 [doc](https://pypi.python.org/pypi/enum34/1.1.1).

> Example:
py
holder = simulation.household.get_holder('housing_occupancy_status')
Three possibilities
holder.set_input(period, numpy.asarray([HousingOccupancyStatus.owner]))
holder.set_input(period, numpy.asarray(['owner']))
holder.set_input(period, numpy.asarray([0])) Highly not recommanded


- When calculating an Enum variable, the output will be an [EnumArray](https://openfisca.org/doc/openfisca-python-api/enum_array.html#module-openfisca_core.indexed_enums).