chassis

Convenience functions for optimizing MuyGPS objects

The functions optimize_from_indices() and optimize_from_tensors() wrap different optimization packages to provide a simple interface to optimize the hyperparameters of MuyGPS objects.

Currently, opt_method="scipy" wraps scipy.optimize.opt multiparameter optimization using L-BFGS-B algorithm using the objective function MuyGPyS.optimize.objective.loo_crossval().

Currently, opt_method="bayesian" (also accepts "bayes" and "bayes_opt") wraps bayes_opt.BayesianOptimization. Unlike the scipy version, BayesianOptimization can be meaningfully modified by several kwargs. MuyGPyS assigns reasonable defaults if no settings are passed by the user. See the BayesianOptimization documentation for details.

MuyGPyS.optimize.chassis.optimize_from_tensors(muygps, batch_targets, batch_nn_targets, crosswise_diffs, pairwise_diffs, batch_features=None, loss_method='mse', obj_method='loo_crossval', opt_method='bayes', sigma_method='analytic', loss_kwargs={}, verbose=False, **kwargs)[source]

Find the optimal model using existing difference matrices.

See the following example, where we have already created a batch_indices vector and a batch_nn_indices matrix using MuyGPyS.neighbors.NN_Wrapper, a crosswise_diffs matrix using MuyGPyS.gp.tensors.crosswise_tensor() and pairwise_diffs using MuyGPyS.gp.tensors.pairwise_tensor(), and initialized a MuyGPS model muygps.

Example

>>> from MuyGPyS.optimize.chassis import optimize_from_tensors
>>> muygps = optimize_from_tensors(
...         muygps,
...         batch_indices,
...         batch_nn_indices,
...         crosswise_diffs,
...         pairwise_diffs,
...         train_responses,
...         loss_method='mse',
...         obj_method='loo_crossval',
...         opt_method='scipy',
...         verbose=True,
... )
parameters to be optimized: ['nu']
bounds: [[0.1 1. ]]
sampled x0: [0.8858425]
optimizer results:
      fun: 0.4797763813693626
 hess_inv: <1x1 LbfgsInvHessProduct with dtype=float64>
      jac: array([-3.06976666e-06])
  message: b'CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL'
     nfev: 16
      nit: 5
     njev: 8
   status: 0
  success: True
        x: array([0.39963594])
Parameters:

  • muygps (MuyGPS) – The model to be optimized.

  • batch_targets (ndarray) – Matrix of floats of shape (batch_count, response_count) whose rows give the expected response for each batch element.

  • batch_nn_targets (ndarray) – Tensor of floats of shape (batch_count, nn_count, response_count) containing the expected response for each nearest neighbor of each batch element.

  • crosswise_diffs (ndarray) – A tensor of shape (batch_count, nn_count, feature_count) whose last two dimensions list the difference between each feature of each batch element element and its nearest neighbors.

  • pairwise_diffs (ndarray) – A tensor of shape (batch_count, nn_count, nn_count, feature_count) containing the (nn_count, nn_count, feature_count)-shaped pairwise nearest neighbor difference tensors corresponding to each of the batch elements.

  • loss_method (str) – Indicates the loss function to be used.

  • obj_method (str) – Indicates the objective function to be minimized. Currently restricted to "loo_crossval".

  • opt_method (str) – Indicates the optimization method to be used. Currently restricted to "bayesian" (alternately "bayes" or "bayes_opt") and "scipy".

  • sigma_method (Optional[str]) – The optimization method to be employed to learn the sigma_sq hyperparameter.

  • loss_kwargs (Dict) – A dictionary of additional keyword arguments to apply to the loss function. Loss function specific.

  • verbose (bool) – If True, print debug messages.

  • kwargs – Additional keyword arguments to be passed to the wrapper optimizer.

Return type:

MuyGPS

Returns:

A new MuyGPs model whose specified hyperparameters have been optimized.