Bringing it all Together Sample Clauses

Bringing it all Together. ‌ For now, we have not described how to actually pull all the parts together to allow for the optimization. Most off-the-shelf optimizers (minimizers) accept an objective function f(x) and gradient evaluation df(x). Thus, we need to transform the parameters, which are currently simple variables, into known parameters for the optimization and have an effective and simple way to pulling them together. In GPy, we use the package called paramz [99], which handles parameterized optimization in an efficient manner. Paramz is part of the contributions of this thesis and will be discussed in detail in Section 3.12. It provides routines for pa- rameter constraining, printing, getting and setting and automatic model updates. One optimization step (iteration) is a cycle through all parameters, updating each part individually and calling the inference on the updated parameters. It keeps pa- rameters in memory in one place (collated together into one array), so that getting and setting parameters individually, automatically updates the variable x of the objective function. This reduces overload in getting and setting parameters, as well as minimizes memory requirements of the model. The paramz package also provides that gradients of variables are stored directly on the parameters themselves (also memory controlled), so that each of the parts can control their respective gradients, allowing for separation of code. One opti- mization cycle (iteration) goes as follows: