Название: Bayesian Optimization
Автор: Roman Garnett
Издательство: Cambridge University Press
Год: 2023
Страниц: 375
Язык: английский
Формат: pdf
Размер: 20.9 MB

Bayesian optimization is a methodology for optimizing expensive objective functions that has proven success in the sciences, engineering, and beyond. This timely text provides a self-contained and comprehensive introduction to the subject, starting from scratch and carefully developing all the key ideas along the way. This bottom-up approach illuminates unifying themes in the design of Bayesian optimization algorithms and builds a solid theoretical foundation for approaching novel situations. The core of the book is divided into three main parts, covering theoretical and practical aspects of Gaussian process modeling, the Bayesian approach to sequential decision making, and the realization and computation of practical and effective optimization policies. Following this foundational material, the book provides an overview of theoretical convergence results, a survey of notable extensions, a comprehensive history of Bayesian optimization, and an extensive annotated bibliography of applications.

The book is divided into three main parts. Chapters 2–4 cover theoretical and practical aspects of modeling with Gaussian processes. This class of models is the overwhelming favorite in the Bayesian optimization literature, and the material contained within is critical for several following chapters. It was daunting to write this material in light of the many excellent references already available, in particular the aforementioned Gaussian Processes for Machine Learning. However, I heavily biased the presentation in light of the needs of optimization, and even experts may find something new.

Chapters 5–7 develop the theory of sequential decision making and its application to optimization. Although this theory requires a model of the objective function and our observations of it, the presentation is agnostic to the choice of model and may be read independently from the preceding chapters on Gaussian processes.

These threads are unified in Chapters 8–10, which discuss the particulars of Bayesian optimization with Gaussian process models. Chapters 8–9 cover details of computation and implementation, and Chapter 10 discusses theoretical performance bounds on Bayesian optimization algorithms, where most results depend intimately on a Gaussian process model of the objective function or the associated reproducing kernel Hilbert space.

The nuances of some applications require modifications to the basic sequential optimization scheme that is the focus of the bulk of the book, and Chapter 11 introduces several notable extensions to this basic setup. Each is systematically presented through the unifying lens of Bayesian decision theory to illustrate how one might proceed when facing a novel situation.

Finally, Chapter 12 provides a brief and standalone history of Bayesian optimization.