chassis

class MuyGPyS.optimize.chassis.OptimizeFn(optimize_fn, make_obj_fn)[source]

Outer-loop optimization functor class.

MuyGPyS-compatible optimization functions are objects of this class. Creating a new outer-loop optimization function is as simple as initializing a new OptimizeFn object with conforming optimize_fn and make_obj_fn functions.

Parameters:

optimize_fn (Callable) – A Callable with the signature (muygps, obj_fn, verbose=verbose, **kwargs) -> MuyGPS.
make_obj_fn (Callable) – A Callable taking the following objects, in order: loss_fn, kernel_fn, mean_fn, var_fn, scale_fn, pairwise_diffs, crosswise_diffs batch_nn_targets, batch_targets batch_features, loss_kwargs.

make_obj_fn(muygps, batch_targets, batch_nn_targets, crosswise_diffs, pairwise_diffs, batch_features=None, target_mask=None, loss_fn=<MuyGPyS.optimize.loss.LossFn object>, loss_kwargs={}, **kwargs)[source]

Returns the objective function specified by the training batch and model choices.

Parameters:

muygps (MuyGPS) – The model to be optimized.
batch_targets (ndarray) – Matrix of floats of shape (batch_count,) [+ (response_count,)] listing the expected (possibly multivariate) responses for each batch element.
batch_nn_targets (ndarray) – Tensor of floats of shape (batch_count, nn_count) [+ (response_count,)] containing the expected (possibly multivariate) response for each nearest neighbor of each batch element.
crosswise_diffs (ndarray) – A tensor of shape (batch_count, nn_count) [+ (feature_count,)] containing the crosswise distances or feature-dimension-wise differences (extra feature_count dimension) between the batch elements and each of their nearest neighbors.
pairwise_diffs (ndarray) – A tensor of shape (batch_count, nn_count, nn_count) [+ (feature_count,)] containing the pairwise distances or feature-dimension-wise differences (extra feature_count dimension) between all pairs of nearest neighbors for each batch element.
loss_fn (LossFn) – The loss functor used to evaluate model performance.
target_mask (Optional[ndarray]) – An array of indices, listing the output dimensions of the prediction to be used for optimization.
loss_kwargs (Dict) – A dictionary of additional keyword arguments to apply to the LossFn. Loss function specific.
kwargs – Additional keyword arguments to be passed to the wrapper optimizer.

Return type:

Callable

Returns:

A Callable function that evaluates the objective function for a given value of the free parameters.

MuyGPyS.optimize.chassis.Bayes_optimize = <MuyGPyS.optimize.chassis.OptimizeFn object>

Optimize a MuyGPS model using Bayesian optimization.

See the following example, where we have already created a batch_indices vector and a batch_nn_indices matrix using MuyGPyS.neighbors.NN_Wrapper, a crosswise_diffs matrix using MuyGPyS.gp.tensors.crosswise_tensor() and pairwise_diffs using MuyGPyS.gp.tensors.pairwise_tensor(), and initialized a MuyGPS model muygps.

Example

>>> from MuyGPyS.optimize import Bayes_optimize
>>> muygps = Bayes_optimize(
...         muygps,
...         batch_targets,
...         batch_nn_targets,
...         crosswise_diffs,
...         pairwise_diffs,
...         train_responses,
...         loss_fn=lool_fn,
...         verbose=True,
... )
parameters to be optimized: ['nu']
bounds: [[0.1 5. ]]
initial x0: [0.92898658]
|   iter    |  target   |    nu     |
-------------------------------------
| 1         | 1.826e+03 | 0.929     |
| 2         | 2.359e+03 | 2.143     |
| 3         | 1.953e+03 | 3.63      |
| 4         | 614.4     | 0.1006    |
| 5         | 2.309e+03 | 1.581     |
| 6         | 1.707e+03 | 0.8191    |
| 7         | 1.48e+03  | 5.0       |
| 8         | 2.202e+03 | 2.83      |
| 9         | 2.373e+03 | 1.883     |
| 10        | 2.373e+03 | 1.996     |
| 11        | 2.375e+03 | 1.938     |
| 12        | 2.375e+03 | 1.938     |
| 13        | 2.375e+03 | 1.938     |
| 14        | 2.375e+03 | 1.938     |
| 15        | 2.375e+03 | 1.938     |
| 16        | 2.375e+03 | 1.938     |
| 17        | 2.375e+03 | 1.938     |
| 18        | 2.375e+03 | 1.945     |
| 19        | 2.375e+03 | 1.927     |
| 20        | 2.375e+03 | 1.95      |
| 21        | 2.375e+03 | 1.926     |
=====================================

Parameters:

muygps – The model to be optimized.
batch_targets – Matrix of floats of shape (batch_count,) [+ (response_count,)] listing the expected (possibly multivariate) responses for each batch element.
batch_nn_targets – Tensor of floats of shape (batch_count, nn_count) [+ (response_count,)] containing the expected (possibly multivariate) response for each nearest neighbor of each batch element.
crosswise_diffs – A tensor of shape (batch_count, nn_count) [+ (feature_count,)] containing the crosswise distances or feature-dimension-wise differences (extra feature_count dimension) between the batch elements and each of their nearest neighbors.
pairwise_diffs – A tensor of shape (batch_count, nn_count, nn_count) [+ (feature_count,)] containing the pairwise distances or feature-dimension-wise differences (extra feature_count dimension) between all pairs of nearest neighbors for each batch element.
loss_fn – The loss functor used to evaluate model performance.
loss_kwargs – A dictionary of additional keyword arguments to apply to the LossFn. Loss function specific.
verbose – If True, print debug messages.
kwargs – Additional keyword arguments to be passed to the wrapper optimizer.

Returns:

A new MuyGPs model whose specified hyperparameters have been optimized.

MuyGPyS.optimize.chassis.L_BFGS_B_optimize = <MuyGPyS.optimize.chassis.OptimizeFn object>

Optimize a MuyGPS model using the L-BFGS-B algorithm.

See the following example, where we have already created a batch_indices vector and a batch_nn_indices matrix using MuyGPyS.neighbors.NN_Wrapper, a crosswise_diffs matrix using MuyGPyS.gp.tensors.crosswise_tensor() and pairwise_diffs using MuyGPyS.gp.tensors.pairwise_tensor(), and initialized a MuyGPS model muygps.

Example

>>> from MuyGPyS.optimize import L_BFGS_B_optimize
>>> muygps = L_BFGS_B_optimize(
...         muygps,
...         batch_targets,
...         batch_nn_targets,
...         crosswise_diffs,
...         pairwise_diffs,
...         train_responses,
...         loss_fn=lool_fn,
...         verbose=True,
... )
parameters to be optimized: ['nu']
bounds: [[0.1 1. ]]
sampled x0: [0.8858425]
optimizer results:
    fun: 0.4797763813693626
hess_inv: <1x1 LbfgsInvHessProduct with dtype=float64>
    jac: array([-3.06976666e-06])
message: b'CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL'
    nfev: 16
    nit: 5
    njev: 8
status: 0
success: True
        x: array([0.39963594])

Parameters:

muygps – The model to be optimized.
batch_targets – Matrix of floats of shape (batch_count,) [+ (response_count,)] listing the expected (possibly multivariate) responses for each batch element.
batch_nn_targets – Tensor of floats of shape (batch_count, nn_count) [+ (response_count,)] containing the expected (possibly multivariate) response for each nearest neighbor of each batch element.
crosswise_diffs – A tensor of shape (batch_count, nn_count) [+ (feature_count,)] containing the crosswise distances or feature-dimension-wise differences (extra feature_count dimension) between the batch elements and each of their nearest neighbors.
pairwise_diffs – A tensor of shape (batch_count, nn_count, nn_count) [+ (feature_count,)] containing the pairwise distances or feature-dimension-wise differences (extra feature_count dimension) between all pairs of nearest neighbors for each batch element.
loss_fn – The loss functor used to evaluate model performance.
loss_kwargs – A dictionary of additional keyword arguments to apply to the LossFn. Loss function specific.
verbose – If True, print debug messages.
kwargs – Additional keyword arguments to be passed to the wrapper optimizer.

Returns:

A new MuyGPs model whose specified hyperparameters have been optimized.