Smac

Major Changes

* Support for Successive Halving and Hyperband as new instensification/racing strategies.
* Improve the SMAC architecture by moving from an architecture where new candidates are passed to the racing algorithm
to an architecture where the racing algorithm requests new candidates, which is necessary to implement the
[BOHB](http://proceedings.mlr.press/v80/falkner18a.html) algorithm (#551).
* Source code is now PEP8 compliant. PEP8 compliance is checked by travis-ci (565).
* Source code is now annotated with type annotation and checked with mypy.

Minor Changes

* New argument to directly control the size of the initial design (553).
* Acquisition function is fed additional arguments at update time (557).
* Adds new acquisition function maximizer which goes through a list of pre-specified configurations (558).
* Document that the dependency pyrfr does not work with SWIG 4.X (599).
* Improved error message for objects which cannot be serialized to json (453).
* Dropped the random forest with HPO surrogate which was added in 0.9.
* Dropped the EPILS facade which was added in 0.6.
* Simplified the interface for constructing a runhistory object.
* removed the default rng from the Gaussian process priors (554).
* Adds the possibility to specify the acquisition function optimizer for the random search (ROAR) facade (563).
* Bump minimal version of `ConfigSpace` requirement to 0.4.9 (578).
* Examples are now rendered on the website using sphinx gallery (567).

Bug fixes

* Fixes a bug which caused SMAC to fail for Python function if `use_pynisher=False` and an exception was raised
(437).
* Fixes a bug in which samples from a Gaussian process were shaped differently based on the number of dimesions of
the `y`-array used for fitting the GP (556).
* Fixes a bug with respect saving data as json (555).
* Better error message for a sobol initial design of size `>40` ( 564).
* Add a missing return statement to `GaussianProcess._train`.

Changes

* Updated the default hyperparameters of the Gaussian process facade to follow recent research (529)
* Enabled `flake8` code style checks for newly merged code (525)

Major changes

* Local search now starts from observed configurations with high acquisition function values, low cost and the from
unobserved configurations with high acquisition function values found by random search (509)
* Reduces the number of mandatory requirements (516)
* Make Gaussian processes more resilient to linalg error by more aggressively adding noise to the diagonal (511)
* Inactive hyperparameters are now imputed with a value outside of the modeled range (-1) (508)
* Replace the GP library George by scikit-learn (505)
* Renames facades to better reflect their use cases (492), and adds a table to help deciding which facade to use (495)
* SMAC facades now accept class arguments instead of object arguments (486)

Minor changes

* Vectorize local search for improved speed (500)
* Extend the Sobol and LHD initial design to work for non-continuous hyperparameters as well applying an idea similar
to inverse transform sampling (494)

Bug fixes

* Fixes a regression in the validation scripts (519)
* Fixes a unit test regression with numpy 1.17 (523)
* Fixes an error message (510)
* Fixes an error making random search behave identical for all seeds

Major changes

* ADD further acquisition functions: PI and LCB
* SMAC can now be installed without installing all its dependencies
* Simplify setup.py by moving most thing to setup.cfg

Bug fixes

* RM typing as requirement
* FIX import of authors in setup.py
* MAINT use json-file as standard pcs format for internal logging

Major changes
* ADD multiple optional initial designs: LHC, Factorial Design, Sobol
* ADD fmin interface know uses BORF facade (should perform much better on continuous, low-dimensional functions)
* ADD Hydra (see "Hydra: Automatically Configuring Algorithms for Portfolio-Based Selection" by Xu et al)
* MAINT Not every second configuration is randomly drawn, but SMAC samples configurations randomly with a given probability (default: 0.5)
* MAINT parsing of options

Interface changes
* ADD two new interfaces to optimize low dimensional continuous functions (w/o instances, docs missing)
* BORF facade: Initial design + Tuned RF
* BOGP interface: Initial design + GP
* ADD options to control acquisition function optimization
* ADD option to transform function values (log, inverse w/ and w/o scaling)
* ADD option to set initial design

Minor changes
* ADD output of estimated cost of final incumbent
* ADD explanation of "deterministic" option in documentation
* ADD save configspace as json
* ADD random forest with automated HPO (not activated by default)
* ADD optional linear cooldown for interleaving random configurations (not active by default)
* MAINT Maximal cutoff time of pynisher set to UINT16
* MAINT make SMAC deterministic if function is deterministic, the budget is limited and the run objective is quality
* MAINT SLS on acquisition function (plateau walks)
* MAINT README
* FIX abort-on-first-run-crash
* FIX pSMAC input directory parsing
* FIX fmin interface with more than 10 parameters
* FIX no output directory if set to '' (empty string)
* FIX use `np.log` instead of `np.log10`
* FIX No longer use law of total variance for uncertainty prediction for RFs as EPM, but only variance over trees (no variance in trees)
* FIX Marginalize over instances inside of each tree of the forest leads to better uncertainty estimates (motivated by the original SMAC implementation)

Major changes

* Upgrade to ConfigSpace (0.4.X), which is not backwards compatible. On the plus
side, the ConfigSpace is about 3-10 times faster, depending on the task.
* FIX 240: improved output directory structure. If the user does not specify
an output directory a SMAC experiment will have the following structure:
`smac_/run_<run_id>/*.json`. The user can specify a output directory, e.g.
`./myExperiment` or `./myExperiment/` which results in
`./myExperiment/run_<run_id>/*.json`.
* Due to changes in AnaConda's compiler setup we drop the unit tests for
python3.4.

Interface changes

* Generalize the interface of the acquisition functions to work with
ConfigSpaces's configuration objects instead of numpy arrays.
* The acquisition function optimizer can now be passed to the SMBO object.
* A custom SMBO class can now be passed to the SMAC builder object.
* `run_id` is no longer an argument to the Scenario object, making the interface
a bit cleaner.

Minor changes

* 333 fixes an incompability with `uncorrelated_mo_rf_with_instances`.
* 323 fixes 324 and 319, which both improve the functioning of the built-in
validation tools.
* 350 fixes random search, which could accidentaly use configurations found my
a local acquisition function optimizer.
* 336 makes validation more flexible.