MuyGPS

class MuyGPyS.gp.muygps.MuyGPS(kern='matern', eps={'val': 0.0}, **kwargs)[source]

Local Kriging Gaussian Process.

Performs approximate GP inference by locally approximating an observation’s response using its nearest neighbors. Implements the MuyGPs algorithm as articulated in [muyskens2021muygps].

Kernels accept different hyperparameter dictionaries specifying hyperparameter settings. Keys can include val and bounds. bounds must be either a len == 2 iterable container whose elements are scalars in increasing order, or the string fixed. If bounds == fixed (the default behavior), the hyperparameter value will remain fixed during optimization. val must be either a scalar (within the range of the upper and lower bounds if given) or the strings "sample" or log_sample", which will randomly sample a value within the range given by the bounds.

In addition to individual kernel hyperparamters, each MuyGPS object also possesses a homoscedastic \(\varepsilon\) noise parameter and a vector of \(\sigma^2\) indicating the scale parameter associated with the posterior variance of each dimension of the response.

\(\sigma^2\) is the only parameter assumed to be a training target by default, and is treated differently from all other hyperparameters. All other training targets must be manually specified in k_kwargs.

Example

>>> from MuyGPyS.gp.muygps import MuyGPS
>>> k_kwargs = {
...         "kern": "rbf",
...         "metric": "F2",
...         "eps": {"val": 1e-5},
...         "nu": {"val": 0.38, "bounds": (0.1, 2.5)},
...         "length_scale": {"val": 7.2},
... }
>>> muygps = MuyGPS(**k_kwarg)

MuyGPyS depends upon linear operations on specially-constructed tensors in order to efficiently estimate GP realizations. One can use (see their documentation for details) MuyGPyS.gp.distance.pairwise_distances() to construct pairwise distance tensors and MuyGPyS.gp.distance.crosswise_distances() to produce crosswise distance matrices that MuyGPS can then use to construct kernel tensors and cross-covariance matrices, respectively.

We can easily realize kernel tensors using a MuyGPS object’s kernel functor once we have computed a pairwise_dists tensor and a crosswise_dists matrix.

Example

>>> K = muygps.kernel(pairwise_dists)
>>> Kcross = muygps.kernel(crosswise_dists)
Parameters

  • kern (str) – The kernel to be used. Each kernel supports different hyperparameters that can be specified in kwargs. Currently supports only matern and rbf.

  • eps (Dict[str, Union[float, Tuple[float, float]]]) – A hyperparameter dict.

  • kwargs – Addition parameters to be passed to the kernel, possibly including additional hyperparameter dicts and a metric keyword.

fixed()[source]

Checks whether all kernel and model parameters are fixed.

This is a convenience utility to determine whether optimization is required.

Return type

bool

Returns

Returns True if all parameters are fixed, and False otherwise.

get_optim_params()[source]

Return a dictionary of references to the unfixed kernel hyperparameters.

This is a convenience function for obtaining all of the information necessary to optimize hyperparameters. It is important to note that the values of the dictionary are references to the actual hyperparameter objects underying the kernel functor - changing these references will change the kernel.

Return type

Dict[str, Hyperparameter]

Returns

A dict mapping hyperparameter names to references to their objects. Only returns hyperparameters whose bounds are not set as fixed. Returned hyperparameters can include eps, but not sigma_sq, as it is currently optimized via a separate closed-form method.

regress(K, Kcross, batch_nn_targets, variance_mode=None, apply_sigma_sq=True)[source]

Performs simultaneous regression on provided covariance, cross-covariance, and target.

Computes parallelized local solves of systems of linear equations using the last two dimensions of K along with Kcross and batch_nn_targets to predict responses in terms of the posterior mean. Also computes the posterior variance if variance_mode is set appropriately. Assumes that kernel tensor K and cross-covariance matrix Kcross are already computed and given as arguments. To implicitly construct these values from indices (useful if the kernel or distance tensors and matrices are not needed for later reference) instead use regress_from_indices().

Returns the predicted response in the form of a posterior mean for each element of the batch of observations, as computed in Equation (3.4) of [muyskens2021muygps]. For each batch element \(\mathbf{x}_i\), we compute

\[\widehat{Y}_{NN} (\mathbf{x}_i \mid X_{N_i}) = K_\theta (\mathbf{x}_i, X_{N_i}) (K_\theta (X_{N_i}, X_{N_i}) + \varepsilon I_k)^{-1} Y(X_{N_i}).\]

Here \(X_{N_i}\) is the set of nearest neighbors of \(\mathbf{x}_i\) in the training data, \(K_\theta\) is the kernel functor specified by self.kernel, \(\varepsilon I_k\) is a diagonal homoscedastic noise matrix whose diagonal is the value of the self.eps hyperparameter, and \(Y(X_{N_i})\) is the (nn_count, respones_count) matrix of responses of the nearest neighbors given by the second two dimensions of the batch_nn_targets argument.

If variance_mode == "diagonal", also return the local posterior variances of each prediction, corresponding to the diagonal elements of a covariance matrix. For each batch element \(\mathbf{x}_i\), we compute

\[Var(\widehat{Y}_{NN} (\mathbf{x}_i \mid X_{N_i})) = K_\theta (\mathbf{x}_i, \mathbf{x}_i) - K_\theta (\mathbf{x}_i, X_{N_i}) (K_\theta (X_{N_i}, X_{N_i}) + \varepsilon I_k)^{-1} K_\theta (X_{N_i}, \mathbf{x}_i).\]
Parameters

  • K (array) – A tensor of shape (batch_count, nn_count, nn_count) containing the (nn_count, nn_count -shaped kernel matrices corresponding to each of the batch elements.

  • Kcross (array) – A tensor of shape (batch_count, nn_count) containing the 1 x nn_count -shaped cross-covariance matrix corresponding to each of the batch elements.

  • batch_nn_targets (array) – A tensor of shape (batch_count, nn_count, response_count) whose last dimension lists the vector-valued responses for the nearest neighbors of each batch element.

  • variance_mode (Optional[str]) – Specifies the type of variance to return. Currently supports "diagonal" and None. If None, report no variance term.

  • apply_sigma_sq (bool) – Indicates whether to scale the posterior variance by sigma_sq. Unused if variance_mode is None or sigma_sq == "unlearned".

Return type

Union[ndarray, Tuple[ndarray, ndarray]]

Returns

  • responses – A matrix of shape (batch_count, response_count,) whose rows are the predicted response for each of the given indices.

  • diagonal_variance – A vector of shape (batch_count,) consisting of the diagonal elements of the posterior variance, or a matrix of shape (batch_count, response_count) for a multidimensional response. Only returned where variance_mode == "diagonal".

regress_from_indices(indices, nn_indices, test, train, targets, variance_mode=None, apply_sigma_sq=True, return_distances=False)[source]

Performs simultaneous regression on a list of observations.

This is similar to the old regress API in that it implicitly creates and discards the distance and kernel tensors and matrices. If these data structures are needed for later reference, instead use regress().

Parameters

  • indices (ndarray) – An integral vector of shape (batch_count,) indices of the observations to be approximated.

  • nn_indices (ndarray) – An integral matrix of shape (batch_count, nn_count) listing the nearest neighbor indices for all observations in the test batch.

  • test (ndarray) – The full testing data matrix of shape (test_count, feature_count).

  • train (ndarray) – The full training data matrix of shape (train_count, feature_count).

  • targets (ndarray) – A matrix of shape (train_count, response_count) whose rows are vector-valued responses for each training element.

  • variance_mode (Optional[str]) – Specifies the type of variance to return. Currently supports "diagonal" and None. If None, report no variance term.

  • apply_sigma_sq (bool) – Indicates whether to scale the posterior variance by sigma_sq. Unused if variance_mode is None or sigma_sq == "unlearned".

  • return_distances (bool) – If True, returns a (test_count, nn_count) matrix containing the crosswise distances between the test elements and their nearest neighbor sets and a (test_count, nn_count, nn_count) tensor containing the pairwise distances between the test data’s nearest neighbor sets.

Return type

Union[ndarray, Tuple[ndarray, ndarray], Tuple[ndarray, ndarray, ndarray], Tuple[ndarray, ndarray, ndarray, ndarray]]

Returns

  • responses – A matrix of shape (batch_count, response_count,) whose rows are the predicted response for each of the given indices.

  • diagonal_variance – A vector of shape (batch_count,) consisting of the diagonal elements of the posterior variance, or a matrix of shape (batch_count, response_count) for a multidimensional response. Only returned where variance_mode == "diagonal".

  • crosswise_dists – A matrix of shape (test_count, nn_count) whose rows list the distance of the corresponding test element to each of its nearest neighbors. Only returned if return_distances is True.

  • pairwise_dists – A tensor of shape (test_count, nn_count, nn_count,) whose latter two dimensions contain square matrices containing the pairwise distances between the nearest neighbors of the test elements. Only returned if return_distances is True.

set_eps(**eps)[source]

Reset \(\varepsilon\) value or bounds.

Uses existing value and bounds as defaults.

Parameters

eps – A hyperparameter dict.

Return type

None

sigma_sq_optim(K, nn_indices, targets)[source]

Optimize the value of the \(\sigma^2\) scale parameter for each response dimension.

We approximate \(\sigma^2\) by way of averaging over the analytic solution from each local kernel.

\[\sigma^2 = \frac{1}{n} * Y^T K^{-1} Y\]
Parameters

  • K (ndarray) – A tensor of shape (batch_count, nn_count, nn_count) containing the (nn_count, nn_count -shaped kernel matrices corresponding to each of the batch elements.

  • nn_indices (ndarray) – An integral matrix of shape (batch_count, nn_count) listing the nearest neighbor indices for all observations in the test batch.

  • targets (ndarray) – A matrix of shape (batch_count, response_count) whose rows list the vector-valued responses for all of the training targets.

Return type

ndarray

Returns

A vector of shape (response_count) listing the value of sigma^2 for each dimension.