Gpboost

- Support space-time (‘matern_space_time’) and anisotropic ARD (‘matern_ard’, ‘gaussian_ard’) covariance functions
- support ‘negative_binomial’ likelihood
- support FITC aka modified predictive process approximation (‘fitc’) and full scale approximation with tapering (‘full_scale_tapering’) with ‘cholesky’ decomposition and ‘iterative’ methods
- add optimizer_cov option 'lbfgs', and make this the default for (generalized) linear effects models
- faster prediction for multiple grouped random effects and non-Gaussian likelihoods
- allow for duplicate locations / coordinates for Vecchia approximation for non-Gaussian likelihoods
- support vecchia approximation for space-time and ARD covariance functions with correlation-based neighbor selection
- support offset in GLMMs
- add safeguard against too large step sizes for linear regression coefficients
- change default initial values for (i) (marginal) variance and error variance to var(y)/2 for Gaussian likelihoods and (ii) range parameters such that the effective range is half the average distance
- add backtracking line search for mode finding in Laplace approximation
- add option ‘reuse_learning_rates_gp_model’ for GPBoost algorithm -> faster learning
- add option ‘line_search_step_length’ for GPBoost algorithm. This corresponds to the optimal choice of boosting learning rate as in e.g. Friedman (2001)
- support optimzer_coef = ‘wls’ when optimizer_cov = ‘lbfgs’ for Gaussian likelihood, make this the default

- support iterative methods for Vecchia-Laplace approximation (non-Gaussian data and gp_approx=”vecchia”)
- faster model construction and prediction for compactly supported covariance functions
- add metric 'test_neg_log_likelihood'
- change handling of 'objective' parameter for GPBoost algorithm: only ‘likelihood’ in ‘GPModel()’ needs to be set
- change API for parameters ‘vecchia_pred_type’ and ‘num_neighbors_pred’

- faster gradient calculation for
1. Multiple / multilevel grouped random effects for non-Gaussian likelihoods
2. GPs with Vecchia approximation for non-Gaussian likelihoods
3. GPs with compactly supported covariance functions / tapering

- enable estimation of shape parameter in gamma likelihood
- predict_training_data_random_effects: enable for Vecchia approximation and enable calculation of variances
- change API for Vecchia approximation and tapering
- correction in nearest neighbor search for Vecchia approximation
- show GPModel parameters on original and not transformed scale when trace = true
- change initial intercept for bernoulli_probit, gamma, and poisson likelihood
- change default value for ‘delta_rel_conv’ to 1e-8 for nelder_mead
- avoid unrealistically large learning rates for gradient descent

- cap too large gradient descent steps on log-scale for covariance parameters, GLMMs: reset small learning rates for covariance parameters and regression parameters if the other parameters change
- add gaussian_neg_log_likelihood as validation metric
- add function ‘get_nested_categories‘ for nested grouped random effects
- prediction: remove nugget variance from predictive (co)variances when predict_response = false for Gaussian likelihoods
- set default value for predict_response to true in prediction function of GPModel
- NA’s and Inf’s are not allowed in label
- correct prediction if Vecchia approximation for non-Gaussian likelihoods

- Reduce memory usage for Vecchia approximation
- [R-package] add function for creating interaction partial dependence plots
- Add function ‘predict_training_data_random_effects’ for predicting (=‘estimating’) training data random effects
- [R-package][python-package] predict function: rename ‘raw_score’ argument to ‘pred_latent’ and unify handling of Gaussian and non-Gaussian data
- (G)LMMs: better initialization of intercept, change internal scaling of covariates, change default value of ‘lr_coef’ to 0.1
- Add ‘adam’ as optimizer option
- allow for grouped random coefficients without random intercept effects
- [R-package][python-package] nicer summary function

- make predictions faster and more memory efficient when having multiple grouped random effects
- set “nelder_mead” as automatic fallback option if problems in optimization occur
- (generalized) linear mixed effects models: scale covariate data for linear predictor internally for optimization using gradient descent
- add “bfgs” as optimizer option