Approximate Bayesian Optimisation for Neural Networks

A novel Bayesian Optimization method based on a linearized link-function to accounts the under-presented class by using a GP surrogate model. This method is based on Laplace’s method and Gauss-Newton approximations to the Hessian. Our method can improve generalization and be useful when validation data is unavailable (e.g., in nonstationary settings) to solve heteroscedastic behaviours. Our experiments demonstrate that our BO by Gauss-Newton approach competes favorably with state-of-the-art blackbox optimization algorithms.

Kronecker-factored approximation (KFAC) of the Laplace-GGN for Continual Learning

Publication highlighting how Catastrophic Forgetting can be solved by Laplace Gauss-Newton approximation without alternating the complexity cost.

Orion-Asynchronous Distributed Hyperparameter Optimization

Oríon is a black-box function optimization library with a key focus on usability and integrability for its users. As a researcher, you can integrate Oríon to your current workflow to tune your models but you can also use Oríon to develop new optimization algorithms and benchmark them with other algorithms in the same context and conditions.