Botorch

Compatibility
* Require GPyTorch >= 1.8.1 (1347).

New Features
* Support batched models in `RandomFourierFeatures` (1336).
* Add a `skip_expand` option to `AppendFeatures` (1344).

Other Changes
* Allow `qProbabilityOfImprovement` to use batch-shaped `best_f` (1324).
* Make `optimize_acqf` re-attempt failed optimization runs and handle optimization
errors in `optimize_acqf` and `gen_candidates_scipy` better (1325).
* Reduce memory overhead in `MARS.set_baseline_Y` (1346).

Bug Fixes
* Fix bug where `outcome_transform` was ignored for `ModelListGP.fantasize` (1338).
* Fix bug causing `get_polytope_samples` to sample incorrectly when variables
live in multiple dimensions (1341).

Documentation
* Add more descriptive docstrings for models (1327, 1328, 1329, 1330) and for other
classes (1313).
* Expanded on the model documentation at [botorch.org/docs/models](https://botorch.org/docs/models) (#1337).

Compatibility
* Require PyTorch >=1.10 (1293).
* Require GPyTorch >=1.7 (1293).

New Features
* Add MOMF (Multi-Objective Multi-Fidelity) acquisition function (1153).
* Support `PairwiseLogitLikelihood` and modularize `PairwiseGP` (1193).
* Add in transformed weighting flag to Proximal Acquisition function (1194).
* Add `FeasibilityWeightedMCMultiOutputObjective` (1202).
* Add outcome_transform to `FixedNoiseMultiTaskGP` (1255).
* Support Scalable Constrained Bayesian Optimization (1257).
* Support `SaasFullyBayesianSingleTaskGP` in `prune_inferior_points` (1260).
* Implement MARS as a risk measure (1303).
* Add MARS tutorial (1305).

Other Changes
* Add `Bilog` outcome transform (1189).
* Make `get_infeasible_cost` return a cost value for each outcome (1191).
* Modify risk measures to accept `List[float]` for weights (1197).
* Support `SaasFullyBayesianSingleTaskGP` in prune_inferior_points_multi_objective (1204).
* BotorchContainers and BotorchDatasets: Large refactor of the original `TrainingData` API to allow for more diverse types of datasets (1205, 1221).
* Proximal biasing support for multi-output `SingleTaskGP` models (1212).
* Improve error handling in `optimize_acqf_discrete` with a check that `choices` is non-empty (1228).
* Handle `X_pending` properly in `FixedFeatureAcquisition` (1233, 1234).
* PE and PLBO support in Ax (1240, 1241).
* Remove `model.train` call from `get_X_baseline` for better caching (1289).
* Support `inf` values in `bounds` argument of `optimize_acqf` (1302).

Bug Fixes
* Update `get_gp_samples` to support input / outcome transforms (1201).
* Fix cached Cholesky sampling in `qNEHVI` when using `Standardize` outcome transform (1215).
* Make `task_feature` as required input in `MultiTaskGP.construct_inputs` (1246).
* Fix CUDA tests (1253).
* Fix `FixedSingleSampleModel` dtype/device conversion (1254).
* Prevent inappropriate transforms by putting input transforms into train mode before converting models (1283).
* Fix `sample_points_around_best` when using 20 dimensional inputs or `prob_perturb` (1290).
* Skip bound validation in `optimize_acqf` if inequality constraints are specified (1297).
* Properly handle RFFs when used with a `ModelList` with individual transforms (1299).
* Update `PosteriorList` to support deterministic-only models and fix `event_shape` (1300).

Documentation
* Add a note about observation noise in the posterior in `fit_model_with_torch_optimizer` notebook (1196).
* Fix custom botorch model in Ax tutorial to support new interface (1213).
* Update MOO docs (1242).
* Add SMOKE_TEST option to MOMF tutorial (1243).
* Fix `ModelListGP.condition_on_observations`/`fantasize` bug (1250).
* Replace space with underscore for proper doc generation (1256).
* Update PBO tutorial to use EUBO (1262).

New Features
* Implement `ExpectationPosteriorTransform` (903).
* Add `PairwiseMCPosteriorVariance`, a cheap active learning acquisition function (1125).
* Support computing quantiles in the fully Bayesian posterior, add `FullyBayesianPosteriorList` (1161).
* Add expectation risk measures (1173).
* Implement Multi-Fidelity GIBBON (Lower Bound MES) acquisition function (1185).

Other Changes
* Add an error message for one shot acquisition functions in `optimize_acqf_discrete` (939).
* Validate the shape of the `bounds` argument in `optimize_acqf` (1142).
* Minor tweaks to `SAASBO` (1143, 1183).
* Minor updates to tutorials (24f7fda7b40d4aabf502c1a67816ac1951af8c23, 1144, 1148, 1159, 1172, 1180).
* Make it easier to specify a custom `PyroModel` (1149).
* Allow passing in a `mean_module` to `SingleTaskGP/FixedNoiseGP` (1160).
* Add a note about acquisitions using gradients to base class (1168).
* Remove deprecated `box_decomposition` module (1175).

Bug Fixes
* Bug-fixes for `ProximalAcquisitionFunction` (1122).
* Fix missing warnings on failed optimization in `fit_gpytorch_scipy` (1170).
* Ignore data related buffers in `PairwiseGP.load_state_dict` (1171).
* Make `fit_gpytorch_model` properly honor the `debug` flag (1178).
* Fix missing `posterior_transform` in `gen_one_shot_kg_initial_conditions` (1187).

New Features
* Implement SAASBO - `SaasFullyBayesianSingleTaskGP` model for sample-efficient high-dimensional Bayesian optimization (1123).
* Add SAASBO tutorial (1127).
* Add `LearnedObjective` (1131), `AnalyticExpectedUtilityOfBestOption` acquisition function (1135), and a few auxiliary classes to support Bayesian optimization with preference exploration (BOPE).
* Add BOPE tutorial (1138).

Other Changes
* Use `qKG.evaluate` in `optimize_acqf_mixed` (1133).
* Add `construct_inputs` to SAASBO (1136).

Bug Fixes
* Fix "Constraint Active Search" tutorial (1124).
* Update "Discrete Multi-Fidelity BO" tutorial (1134).

New Features
* Use `BOTORCH_MODULAR` in tutorials with Ax (1105).
* Add `optimize_acqf_discrete_local_search` for discrete search spaces (1111).

Bug Fixes
* Fix missing `posterior_transform` in qNEI and `get_acquisition_function` (1113).

New Features
* Add `Standardize` input transform (1053).
* Low-rank Cholesky updates for NEI (1056).
* Add support for non-linear input constraints (1067).
* New MOO problems: MW7 (1077), disc brake (1078), penicillin (1079), RobustToy (1082), GMM (1083).

Other Changes
* Support multi-output models in MES using `PosteriorTransform` (904).
* Add `Dispatcher` (1009).
* Modify qNEHVI to support deterministic models (1026).
* Store tensor attributes of input transforms as buffers (1035).
* Modify NEHVI to support MTGPs (1037).
* Make `Normalize` input transform input column-specific (1047).
* Improve `find_interior_point` (1049).
* Remove deprecated `botorch.distributions` module (1061).
* Avoid costly application of posterior transform in Kronecker & HOGP models (1076).
* Support heteroscedastic perturbations in `InputPerturbations` (1088).

Performance Improvements
* Make risk measures more memory efficient (1034).

Bug Fixes
* Properly handle empty `fixed_features` in optimization (1029).
* Fix missing weights in `VaR` risk measure (1038).
* Fix `find_interior_point` for negative variables & allow unbounded problems (1045).
* Filter out indefinite bounds in constraint utilities (1048).
* Make non-interleaved base samples use intuitive shape (1057).
* Pad small diagonalization with zeros for `KroneckerMultitaskGP` (1071).
* Disable learning of bounds in `preprocess_transform` (1089).
* Fix `gen_candidates_torch` (4079164489613d436d19c7b2df97677d97dfa8dc).
* Catch runtime errors with ill-conditioned covar (1095).
* Fix `compare_mc_analytic_acquisition` tutorial (1099).