MultivariateMuyGPS

class MuyGPyS.gp.muygps.MultivariateMuyGPS(kern, *model_args)[source]

Multivariate Local Kriging Gaussian Process.

Performs approximate GP inference by locally approximating an observation’s response using its nearest neighbors with a separate kernel allocated for each response dimension, implemented as individual MuyGPyS.gp.muygps.MuyGPS objects.

This class is similar in interface to MuyGPyS.gp.muygps.MuyGPS, but requires a list of hyperparameter dicts at initialization.

Example

>>> from MuyGPyS.gp.muygps import MultivariateMuyGPS as MMuyGPS
>>> k_kwargs1 = {
...         "eps": {"val": 1e-5},
...         "nu": {"val": 0.67, "bounds": (0.1, 2.5)},
...         "length_scale": {"val": 7.2},
... }
>>> k_kwargs2 = {
...         "eps": {"val": 1e-5},
...         "nu": {"val": 0.38, "bounds": (0.1, 2.5)},
...         "length_scale": {"val": 7.2},
... }
>>> k_args = [k_kwargs1, k_kwargs2]
>>> mmuygps = MMuyGPS("matern", *k_args)

We can realize kernel tensors for each of the models contained within a MultivariateMuyGPS object by iterating over its models member. Once we have computed a pairwise_dists tensor and a crosswise_dists matrix, it is straightforward to perform each of these realizations.

Example

>>> for model in MuyGPyS.models:
>>>         K = model.kernel(pairwise_dists)
>>>         Kcross = model.kernel(crosswise_dists)
>>>         # do something with K and Kcross...
Args
kern:

The kernel to be used. Each kernel supports different hyperparameters that can be specified in kwargs. Currently supports only matern and rbf.

model_args:

Dictionaries defining each internal MuyGPyS.gp.muygps.MuyGPS instance.

fixed()[source]

Checks whether all kernel and model parameters are fixed for each model, excluding \(\sigma^2\).

Return type

bool

Returns

Returns True if all parameters in all models are fixed, and False otherwise.

regress(pairwise_dists, crosswise_dists, batch_nn_targets, variance_mode=None, apply_sigma_sq=True)[source]

Performs simultaneous regression on provided distance tensors and the target matrix.

Computes parallelized local solves of systems of linear equations using the kernel realizations, one for each internal model, of the last two dimensions of pairwise_dists along with crosswise_dists and batch_nn_targets to predict responses in terms of the posterior mean. Also computes the posterior variance if variance_mode is set appropriately. Assumes that distance tensor pairwise_dists and crosswise distance matrix crosswise_dists are already computed and given as arguments. To implicitly construct these values from indices (useful if the distance tensors and matrices are not needed for later reference) instead use regress_from_indices().

Returns the predicted response in the form of a posterior mean for each element of the batch of observations by solving a system of linear equations induced by each kernel functor, one per response dimension, in a generalization of Equation (3.4) of [muyskens2021muygps]. For each batch element \(\mathbf{x}_i\) we compute

\[\widehat{Y}_{NN} (\mathbf{x}_i \mid X_{N_i})_{:,j} = K^{(j)}_\theta (\mathbf{x}_i, X_{N_i}) (K^{(j)}_\theta (X_{N_i}, X_{N_i}) + \varepsilon_j I_k)^{-1} Y(X_{N_i})_{:,j}.\]

Here \(X_{N_i}\) is the set of nearest neighbors of \(\mathbf{x}_i\) in the training data, \(K^{(j)}_\theta\) is the kernel functor associated with the jth internal model, corresponding to the jth response dimension, \(\varepsilon_j I_k\) is a diagonal homoscedastic noise matrix whose diagonal is the value of the self.models[j].eps hyperparameter, and \(Y(X_{N_i})_{:,j}\) is the (batch_count,) vector of the jth responses of the neartest neighbors given by a slice of the batch_nn_targets argument.

If variance_mode == "diagonal", also return the local posterior variances of each prediction, corresponding to the diagonal elements of a covariance matrix. For each batch element \(\mathbf{x}_i\), we compute

\[Var(\widehat{Y}_{NN} (\mathbf{x}_i \mid X_{N_i}))_j = K^{(j)}_\theta (\mathbf{x}_i, \mathbf{x}_i) - K^{(j)}_\theta (\mathbf{x}_i, X_{N_i}) (K^{(j)}_\theta (X_{N_i}, X_{N_i}) + \varepsilon I_k)^{-1} K^{(j)}_\theta (X_{N_i}, \mathbf{x}_i).\]
Parameters

  • pairwise_dists (ndarray) – A tensor of shape (batch_count, nn_count, nn_count) containing the (nn_count, nn_count) -shaped pairwise nearest neighbor distance matrices corresponding to each of the batch elements.

  • crosswise_dists (ndarray) – A matrix of shape (batch_count, nn_count) whose rows list the distance between each batch element element and its nearest neighbors.

  • batch_nn_targets (ndarray) – A tensor of shape (batch_count, nn_count, response_count) listing the vector-valued responses for the nearest neighbors of each batch element.

  • variance_mode (Optional[str]) – Specifies the type of variance to return. Currently supports "diagonal" and None. If None, report no variance term.

  • apply_sigma_sq (bool) – Indicates whether to scale the posterior variance by sigma_sq. Unused if variance_mode is None or sigma_sq == "unlearned".

Return type

Union[ndarray, Tuple[ndarray, ndarray]]

Returns

  • responses – A matrix of shape (batch_count, response_count,) whose rows are the predicted response for each of the given indices.

  • diagonal_variance – A vector of shape (batch_count, response_count) consisting of the diagonal elements of the posterior variance for each model. Only returned where variance_mode == "diagonal".

regress_from_indices(indices, nn_indices, test, train, targets, variance_mode=None, apply_sigma_sq=True, return_distances=False)[source]

Performs simultaneous regression on a list of observations.

Implicitly creates and discards the distance tensors and matrices. If these data structures are needed for later reference, instead use regress().

Parameters

  • indices (ndarray) – An integral vector of shape (batch_count,) indices of the observations to be approximated.

  • nn_indices (ndarray) – An integral matrix of shape (batch_count, nn_count) listing the nearest neighbor indices for all observations in the test batch.

  • test (ndarray) – The full testing data matrix of shape (test_count, feature_count).

  • train (ndarray) – The full training data matrix of shape (train_count, feature_count).

  • targets (ndarray) – A matrix of shape (train_count, response_count) whose rows are vector-valued responses for each training element.

  • variance_mode (Optional[str]) – Specifies the type of variance to return. Currently supports "diagonal" and None. If None, report no variance term.

  • apply_sigma_sq (bool) – Indicates whether to scale the posterior variance by sigma_sq. Unused if variance_mode is None or sigma_sq == "unlearned".

  • return_distances (bool) – If True, returns a (test_count, nn_count) matrix containing the crosswise distances between the test elements and their nearest neighbor sets and a (test_count, nn_count, nn_count) tensor containing the pairwise distances between the test data’s nearest neighbor sets.

Return type

Union[ndarray, Tuple[ndarray, ndarray], Tuple[ndarray, ndarray, ndarray], Tuple[ndarray, ndarray, ndarray, ndarray]]

Returns

  • responses – A matrix of shape (batch_count, response_count,) whose rows are the predicted response for each of the given indices.

  • variance – A vector of shape (batch_count,) consisting of the diagonal elements of the posterior variance. Only returned where variance_mode == "diagonal".

  • crosswise_dists – A matrix of shape (test_count, nn_count) whose rows list the distance of the corresponding test element to each of its nearest neighbors. Only returned if return_distances is True.

  • pairwise_dists – A tensor of shape (test_count, nn_count, nn_count,) whose latter two dimensions contain square matrices containing the pairwise distances between the nearest neighbors of the test elements. Only returned if return_distances is True.

sigma_sq_optim(pairwise_dists, nn_indices, targets)[source]

Optimize the value of the \(\sigma^2\) scale parameter for each response dimension.

We approximate \(\sigma^2\) by way of averaging over the analytic solution from each local kernel.

\[\sigma^2 = \frac{1}{n} * Y^T K^{-1} Y\]
Parameters

  • pairwise_dists (ndarray) – A tensor of shape (batch_count, nn_count, nn_count) containing the (nn_count, nn_count) -shaped pairwise nearest neighbor distance matrices corresponding to each of the batch elements.

  • nn_indices (ndarray) – An integral matrix of shape (batch_count, nn_count) listing the nearest neighbor indices for all observations in the testing batch.

  • targets (ndarray) – A matrix of shape (train_count, response_count) whose rows are the responses for each training element.

Return type

ndarray

Returns

A vector of shape (response_count,) listing the found value of \(\sigma^2\) for each response dimension.