objective

Objective and Loss Function Handling

MuyGPyS includes predefined loss functions and convenience functions for indicating them to optimization.

MuyGPyS.optimize.objective.cross_entropy_fn(predictions, targets)

Cross entropy function.

Computes the cross entropy loss the predicted versus known response. Transforms predictions to be row-stochastic, and ensures that targets contains no negative elements.

@NOTE[bwp] I don’t remember why we hard-coded eps=1e-6. Might need to revisit.

Parameters

• predictions (ndarray) – The predicted response of shape (batch_count, response_count).

• targets (ndarray) – The expected response of shape (batch_count, response_count).

Return type

float

Returns

The cross-entropy loss of the prediction.

MuyGPyS.optimize.objective.get_loss_func(loss_method)

Select a loss function based upon string key.

Currently supports strings "log" or "cross-entropy" for MuyGPyS.optimize.objective.cross_entropy_fn() and "mse" for MuyGPyS.optimize.objective.mse_fn().

Parameters

• predictions – The predicted response of shape (batch_count, response_count).

• targets – The expected response of shape (batch_count, response_count).

Return type

Callable

Returns

The loss function Callable.

Raises

NotImplementedError – Unrecognized strings will result in an error.

MuyGPyS.optimize.objective.loo_crossval(x0, loss_fn, kernel_fn, predict_fn, pairwise_dists, crosswise_dists, batch_nn_targets, batch_targets)

Leave-one-out cross validation.

Returns leave-one-out cross validation performance for a set MuyGPS object. Predicts on all of the training data at once. This function is designed for use with MuyGPyS.optimize.chassis.optimize_from_tensors() with opt_method="scipy", and embeds the optimization parameters into the x0 vector.

Parameters

• x0 (ndarray) – Current guess for hyperparameter values of shape (opt_count,).

• loss_fn (Callable) – The loss function to be minimizes. Can be any function that accepts two numpy.ndarray objects indicating the prediction and target values, in that order.

• kernel_fn (Callable) – A function that realizes kernel tensors given a list of the free parameters.

• predict_fn (Callable) – A function that realizes MuyGPs prediction given an epsilon value. The given value is unused if epsilon is fixed.

• pairwise_dists (ndarray) – Distance tensor of floats of shape (batch_count, nn_count, nn_count) whose second two dimensions give the pairwise distances between the nearest neighbors of each batch element.

• crosswise_dists (ndarray) – Distance matrix of floats of shape (batch_count, nn_count) whose rows give the distances between each batch element and its nearest neighbors.

• batch_nn_targets (ndarray) – Tensor of floats of shape (batch_count, nn_count, response_count) containing the expected response for each nearest neighbor of each batch element.

• batch_targets (ndarray) – Matrix of floats of shape (batch_count, response_count) whose rows give the expected response for each batch element.

Return type

float

Returns

The evaluation of objective_fn on the predicted versus expected response.

MuyGPyS.optimize.objective.loo_crossval_kwargs(loss_fn, kernel_fn, predict_fn, pairwise_dists, crosswise_dists, batch_nn_targets, batch_targets, **kwargs)

Leave-one-out cross validation.

Returns leave-one-out cross validation performance for a set MuyGPS object. Predicts on all of the training data at once. This function is designed for use with MuyGPyS.optimize.chassis.optimize_from_tensors() with opt_method="bayesian", and the optimization parameters as additional kwargs.

Parameters

• loss_fn (Callable) – The loss function to be minimizes. Can be any function that accepts two numpy.ndarray objects indicating the prediction and target values, in that order.

• kernel_fn (Callable) – A function that realizes kernel tensors given a list of the free parameters.

• predict_fn (Callable) – A function that realizes MuyGPs prediction given an epsilon value. The given value is unused if epsilon is fixed.

• pairwise_dists (ndarray) – Distance tensor of floats of shape (batch_count, nn_count, nn_count) whose second two dimensions give the pairwise distances between the nearest neighbors of each batch element.

• crosswise_dists (ndarray) – Distance matrix of floats of shape (batch_count, nn_count) whose rows give the distances between each batch element and its nearest neighbors.

• batch_nn_targets (ndarray) – Tensor of floats of shape (batch_count, nn_count, response_count) containing the expected response for each nearest neighbor of each batch element.

• batch_targets (ndarray) – Matrix of floats of shape (batch_count, response_count) whose rows give the expected response for each batch element.

• kwargs – Hyperparameter values to be optimized, e.g. nu=0.32.

Return type

float

Returns

The evaluation of objective_fn on the predicted versus expected response.

MuyGPyS.optimize.objective.make_loo_crossval_fn(loss_fn, kernel_fn, predict_fn, pairwise_dists, crosswise_dists, batch_nn_targets, batch_targets)

Prepare a leave-one-out cross validation function as a function purely of the hyperparameters to be optimized.

This function is designed for use with MuyGPyS.optimize.chassis.optimize_from_tensors() with opt_method="scipy", and assumes that the optimization parameters will be passed in an (optim_count,) vector.

Parameters

• loss_fn (Callable) – The loss function to be minimizes. Can be any function that accepts two numpy.ndarray objects indicating the prediction and target values, in that order.

• kernel_fn (Callable) – A function that realizes kernel tensors given a list of the free parameters.

• predict_fn (Callable) – A function that realizes MuyGPs prediction given an epsilon value. The given value is unused if epsilon is fixed.

• pairwise_dists (ndarray) – Distance tensor of floats of shape (batch_count, nn_count, nn_count) whose second two dimensions give the pairwise distances between the nearest neighbors of each batch element.

• crosswise_dists (ndarray) – Distance matrix of floats of shape (batch_count, nn_count) whose rows give the distances between each batch element and its nearest neighbors.

• batch_nn_targets (ndarray) – Tensor of floats of shape (batch_count, nn_count, response_count) containing the expected response for each nearest neighbor of each batch element.

• batch_targets (ndarray) – Matrix of floats of shape (batch_count, response_count) whose rows give the expected response for each batch element.

Return type

Callable

Returns

A Callable objective_fn as a function of only an (optim_count,) vector.

MuyGPyS.optimize.objective.make_loo_crossval_kwargs_fn(loss_fn, kernel_fn, predict_fn, pairwise_dists, crosswise_dists, batch_nn_targets, batch_targets)

Prepare a leave-one-out cross validation function as a function purely of the hyperparameters to be optimized.

This function is designed for use with MuyGPyS.optimize.chassis.optimize_from_tensors() with opt_method="bayesian", and assumes that the optimization parameters will be passed as keyword arguments.

Parameters

• loss_fn (Callable) – The loss function to be minimizes. Can be any function that accepts two numpy.ndarray objects indicating the prediction and target values, in that order.

• kernel_fn (Callable) – A function that realizes kernel tensors given a list of the free parameters.

• predict_fn (Callable) – A function that realizes MuyGPs prediction given an epsilon value. The given value is unused if epsilon is fixed.

• pairwise_dists (ndarray) – Distance tensor of floats of shape (batch_count, nn_count, nn_count) whose second two dimensions give the pairwise distances between the nearest neighbors of each batch element.

• crosswise_dists (ndarray) – Distance matrix of floats of shape (batch_count, nn_count) whose rows give the distances between each batch element and its nearest neighbors.

• batch_nn_targets (ndarray) – Tensor of floats of shape (batch_count, nn_count, response_count) containing the expected response for each nearest neighbor of each batch element.

• batch_targets (ndarray) – Matrix of floats of shape (batch_count, response_count) whose rows give the expected response for each batch element.

Return type

Callable

Returns

A Callable objective_fn as a function of only an (optim_count,) vector.

MuyGPyS.optimize.objective.mse_fn(predictions, targets)

Mean squared error function.

Computes mean squared error loss of the predicted versus known response. Treats multivariate outputs as interchangeable in terms of loss penalty.

Parameters

• predictions (ndarray) – The predicted response of shape (batch_count, response_count).

• targets (ndarray) – The expected response of shape (batch_count, response_count).

Return type

float

Returns

The mse loss of the prediction.