muygps_layer

MuyGPs PyTorch implementation

class MuyGPyS.torch.muygps_layer.MultivariateMuyGPs_layer(num_models, kernel_eps, nu_vals, length_scales, batch_indices, batch_nn_indices, batch_targets, batch_nn_targets)[source]

Multivariate MuyGPs model written as a custom PyTorch layer using nn.Module.

Implements the MuyGPs algorithm as articulated in [muyskens2021muygps]. See documentation on MuyGPs class for more detail.

The MultivariateMuyGPs_layer class only supports the Matern kernel currently. More kernels will be added to the torch module of MuyGPs in future releases.

PyTorch does not currently support the Bessel function required to compute the Matern kernel for non-special values of \(\nu\), e.g. 1/2, 3/2, 5/2, and \(\infty\). The MuyGPs layer allows the lengthscale parameter \(\rho\) to be trained (provided an initial value by the user) as well as the homoskedastic \(\varepsilon\) noise parameter.

The MuyGPs layer returns the posterior mean, posterior variance, and a vector of \(\sigma^2\) indicating the scale parameter associated with the posterior variance of each dimension of the response.

\(\sigma^2\) is the only parameter assumed to be a training target by default, and is treated differently from all other hyperparameters. All other training targets must be manually specified in the construction of a MuyGPs_layer object.

Example

>>> from MuyGPyS.torch.muygps_layer import MultivariateMuyGPs_layer
>>> num_models = 10
>>> kernel_eps = 1e-3 * torch.ones(10,)
>>> nu = 1/2 * torch.ones(10,)
>>> length_scale = 1.0 * torch.ones(10,)
>>> batch_indices = torch.arange(100,)
>>> batch_nn_indices = torch.arange(100,)
>>> batch_targets = torch.ones(100,)
>>> batch_nn_targets = torch.ones(100,)
>>> muygps_layer_object = MultivariateMuyGPs_layer(
... num_models,
... kernel_eps,
... nu,
... length_scale,
... batch_indices,
... batch_nn_indices,
... batch_targets,
... batch_nn_targets)
Parameters

  • num_models – The number of MuyGPs models to be used in the layer.

  • kernel_eps – A torch.Tensor of shape (num_models,) containing the hyperparameter corresponding to the aleatoric uncertainty in the data for each model.

  • nu – A torch.Tensor of shape (num_models,) containing the smoothness parameter in the Matern kernel for each model. Allowed to take on values 1/2, 3/2, 5/2, or \(\infty\).

  • length_scale – A torch.Tensor of shape (num_models,) containing the length scale parameter in the Matern kernel for each model.

  • batch_indices – A torch.Tensor of shape (batch_count,) containing the indices of the training data to be sampled for training.

  • batch_nn_indices – A torch.Tensor of shape (batch_count, nn_count) containing the indices of the k nearest neighbors of the batched training samples.

  • batch_targets – A torch.Tensor of shape (batch_count, response_count) containing the responses corresponding to each batched training sample.

  • batch_nn_targets – A torch.Tensor of shape (batch_count, nn_count, response_count) containing the responses corresponding to the nearest neighbors of each batched training sample.

  • kwargs – Addition parameters to be passed to the kernel, possibly including additional hyperparameter dicts and a metric keyword.

forward(x)[source]

Produce the output of a MuyGPs custom PyTorch layer.

Returns

A torch.Tensor of shape (batch_count, response_count) listing the predicted response for each of the batch elements.

class MuyGPyS.torch.muygps_layer.MuyGPs_layer(kernel_eps, nu, length_scale, batch_indices, batch_nn_indices, batch_targets, batch_nn_targets)[source]

MuyGPs model written as a custom PyTorch layer using nn.Module.

Implements the MuyGPs algorithm as articulated in [muyskens2021muygps]. See documentation on MuyGPs class for more detail.

The MuyGPs_layer class only supports the Matern kernel currently. More kernels will be added to the torch module of MuyGPs in future releases.

PyTorch does not currently support the Bessel function required to compute the Matern kernel for non-special values of \(\nu\), e.g. 1/2, 3/2, 5/2, and \(\infty\). The MuyGPs layer allows the lengthscale parameter \(\rho\) to be trained (provided an initial value by the user) as well as the homoscedastic \(\varepsilon\) noise parameter.

The MuyGPs layer returns the posterior mean, posterior variance, and a vector of \(\sigma^2\) indicating the scale parameter associated with the posterior variance of each dimension of the response.

\(\sigma^2\) is the only parameter assumed to be a training target by default, and is treated differently from all other hyperparameters. All other training targets must be manually specified in the construction of a MuyGPs_layer object.

Example

>>> from MuyGPyS.torch.muygps_layer import MuyGPs_layer
>>> kernel_eps = 1e-3
>>> nu = 1/2
>>> length_scale = 1.0
>>> batch_indices = torch.arange(100,)
>>> batch_nn_indices = torch.arange(100,)
>>> batch_targets = torch.ones(100,)
>>> batch_nn_targets = torch.ones(100,)
>>> muygps_layer_object = MuyGPs_layer(
... kernel_eps,
... nu,
... length_scale,
... batch_indices,
... batch_nn_indices,
... batch_targets,
... batch_nn_targets)
Parameters

  • kernel_eps – A hyperparameter corresponding to the aleatoric uncertainty in the data.

  • nu – A smoothness parameter allowed to take on values 1/2, 3/2, 5/2, or \(\infty\).

  • length_scale – The length scale parameter in the Matern kernel.

  • batch_indices – A torch.Tensor of shape (batch_count,) containing the indices of the training data to be sampled for training.

  • batch_nn_indices – A torch.Tensor of shape (batch_count, nn_count) containing the indices of the k nearest neighbors of the batched training samples.

  • batch_targets – A torch.Tensor of shape (batch_count, response_count) containing the responses corresponding to each batched training sample.

  • batch_nn_targets – A torch.Tensor of shape (batch_count, nn_count, response_count) containing the responses corresponding to the nearest neighbors of each batched training sample.

  • kwargs – Addition parameters to be passed to the kernel, possibly including additional hyperparameter dicts and a metric keyword.

forward(x)[source]

Produce the output of a MuyGPs custom PyTorch layer.

Returns

A torch.Tensor of shape (batch_count, response_count) listing the predicted response for each of the batch elements.

MuyGPyS.torch.muygps_layer.kernel_func(dist_tensor, nu, length_scale)[source]

Generate kernel tensors using the Matern kernel given an input distance tensor. Currently only supports the Matern kernel, but more kernels will be added in future releases.

Parameters

  • dist_matrix – A torch.Tensor distance tensor on which to evaluate the kernel.

  • nu (float) – The smoothness hyperparameter in the Matern kernel.

  • length_scale (float) – The lengthscale hyperparameter in the Matern kernel.

Return type

Tensor

Returns

A torch.Tensor containing the kernel matrix evaluated for the given input values.