MuyGPS

class MuyGPyS.gp.muygps.MuyGPS(kernel, noise=<MuyGPyS.gp.noise.homoscedastic.HomoscedasticNoise object>, scale=<MuyGPyS.gp.hyperparameter.scale.FixedScale object>, _backend_mean_fn=<function _muygps_posterior_mean>, _backend_var_fn=<function _muygps_diagonal_variance>, _backend_fast_mean_fn=<function _muygps_fast_posterior_mean>, _backend_fast_precompute_fn=<function _muygps_fast_posterior_mean_precompute>)

Local Kriging Gaussian Process.

Performs approximate GP inference by locally approximating an observation’s response using its nearest neighbors. Implements the MuyGPs algorithm as articulated in [muyskens2021muygps].

Kernels accept different hyperparameter dictionaries specifying hyperparameter settings. Keys can include val and bounds. bounds must be either a len == 2 iterable container whose elements are scalars in increasing order, or the string fixed. If bounds == fixed (the default behavior), the hyperparameter value will remain fixed during optimization. val must be either a scalar (within the range of the upper and lower bounds if given) or the strings "sample" or log_sample", which will randomly sample a value within the range given by the bounds.

In addition to individual kernel hyperparamters, each MuyGPS object also possesses a noise model, possibly with parameters, and a vector of \(\sigma^2\) indicating the scale parameter associated with the posterior variance of each dimension of the response.

Example

>>> from MuyGPyS.gp import MuyGPS
>>> muygps = MuyGPS(
...    kernel=Matern(
...        smoothness=Parameter(0.38, (0.1, 2.5)),
...        deformation=Isotropy(
...            metric=l2,
...            length_scale=Parameter(0.2),
...        ),
...    ),
...    noise=HomoscedasticNoise(1e-5),
...    scale=AnalyticScale(),
... )

MuyGPyS depends upon linear operations on specially-constructed tensors in order to efficiently estimate GP realizations. One can use (see their documentation for details) MuyGPyS.gp.tensors.pairwise_tensor() to construct pairwise difference tensors and MuyGPyS.gp.tensors.crosswise_tensor() to produce crosswise diff tensors that MuyGPS can then use to construct kernel tensors and cross-covariance matrices, respectively.

We can easily realize kernel tensors using a MuyGPS object’s kernel functor once we have computed a pairwise_diffs tensor and a crosswise_diffs matrix.

Example

>>> Kin = muygps.kernel(pairwise_diffs)
>>> Kcross = muygps.kernel(crosswise_diffs)

Parameters:

• kernel (KernelFn) – The kernel to be used. Defines \(Kin_\theta(\cdot, \cdot)\) as referenced in MuyGPS functions.

• noise (NoiseFn) – A noise model. Defines \(\varepsilon\) as referenced in MuyGPS functions.

• scale (ScaleFn) – A variance scale parameter. Defines \(\sigma^2\) as referenced in MuyGPS functions.

fast_coefficients(Kin, train_nn_targets_fast)

Produces coefficient matrix for the fast posterior mean given in Equation (8) of [dunton2022fast].

Given observation set \(X\) with responses \(Y\), noise prior set \(\varepsilon\), and kernel function \(Kin_\theta(\cdot, \cdot)\), computes the following for each observation element \(\mathbf{x}_i\) with nearest neighbors index set \(N^*_i\), containing i and the indices of the nn_count - 1 nearest neighbors of \(\mathbf{x}_i\):

\[C_i = \left ( Kin_\theta (X_{N_i}, X_{N_i} \right ) + \varepsilon_{N_i})^{-1} Y(X_{N_i}).\]

Parameters:

• Kin (ndarray) – A tensor of shape (batch_count,) + in_shape + in_shape containing the pairwise, possibly multivariate covariance among the neighborhood for each batch element.

• Kcross – A tensor of shape (batch_count,) + out_shape listing the (possibly multivariate) cross-covariance between each batch element and its nearest neighbors.

Return type:

ndarray

Returns:

A matrix \(C\) of shape (train_count, nn_count) whose rows are the precomputed coefficients for fast posterior mean inference.

fast_posterior_mean(Kcross, coeffs_tensor)

Performs fast posterior mean inference using provided cross-covariance and precomputed coefficient matrix.

Assumes that cross-covariance matrix Kcross is already computed and given as an argument.

Returns the predicted response in the form of a posterior mean for each element of the batch of observations, as computed in Equation (9) of [dunton2022fast]. Given the coefficients \(C\) created by fast_coefficients() and Equation (8) of [dunton2022fast], observation set \(X\), noise prior set \(\varepsilon\), and kernel function \(Kin_\theta(\cdot, \cdot)\), computes the following for each test point \(\mathbf{z}\) and index set \(N^*_i\) containing the union of the index \(i\) of the nearest neighbor \(\mathbf{x}_i\) of \(\mathbf{z}\) and the nn_count - 1 nearest neighbors of \(\mathbf{x}_i\):

\[\widehat{Y} \left ( \mathbf{z} \mid X \right ) = \sigma^2 Kin_\theta(\mathbf{z}, X_{N^*_i}) C_i.\]

Parameters:

• Kcross (ndarray) – A tensor of shape (batch_count,) + out_shape listing the (possibly multivariate) cross-covariance between each batch element and its nearest neighbors.

• coeffs_tensor (ndarray) – A tensor of shape (batch_count,) + out_shape listing the precomputed coefficients for each batch element.

Return type:

ndarray

Returns:

A matrix of shape (batch_count,) [+ (response_count,)] listing the predicted response for each batch element.

fixed()

Checks whether all kernel and model parameters are fixed.

This is a convenience utility to determine whether optimization is required.

Return type:

bool

Returns:

Returns True if all parameters are fixed, and False otherwise.

get_opt_mean_fn()

Return a posterior mean function for use in optimization.

Assumes that optimization parameter literals will be passed as keyword arguments.

Return type:

Callable

Returns:

A function implementing the posterior mean, where noise is either fixed or takes updating values during optimization. The function expects keyword arguments corresponding to current hyperparameter values for unfixed parameters.

get_opt_params()

Return lists of unfixed hyperparameter names, values, and bounds.

Return type:

Tuple[List[str], ndarray, ndarray]

Returns:

names:

A list of unfixed hyperparameter names.

params:

A list of unfixed hyperparameter values.

bounds:

A list of unfixed hyperparameter bound tuples.

get_opt_var_fn()

Return a posterior variance function for use in optimization.

Assumes that optimization parameter literals will be passed as keyword arguments.

Return type:

Callable

Returns:

A function implementing posterior variance, where noise is either fixed or takes updating values during optimization. The function expects keyword arguments corresponding to current hyperparameter values for unfixed parameters.

make_predict_tensors(batch_indices, batch_nn_indices, test_features, train_features, train_targets, **kwargs)

Create the metric and target tensors for prediction using the model’s deformation.

Creates the crosswise_tensor, pairwise_tensor and batch_nn_targets tensors required by posterior_mean() and posterior_variance().

Parameters:

• batch_indices (ndarray) – A vector of integers of shape (batch_count,) identifying the training batch of observations to be approximated.

• batch_nn_indices (ndarray) – A matrix of integers of shape (batch_count, nn_count) listing the nearest neighbor indices for all observations in the batch.

• test_features (Optional[ndarray]) – The full floating point testing data matrix of shape (test_count, feature_count).

• train_features (ndarray) – The full floating point training data matrix of shape (train_count, ...).

• train_targets (ndarray) – A matrix of shape (train_count, ...) whose rows are vector-valued responses for each training element.

Return type:

Tuple[ndarray, ndarray, ndarray]

Returns:

• crosswise_tensor – A tensor of shape (batch_count, nn_count) [+ (feature_count,)] containing the crosswise distances or feature-dimension-wise differences (extra feature_count dimension) between the batch elements and each of their nearest neighbors. Shape depends on the deformation.

• pairwise_tensor – A tensor of shape (batch_count, nn_count, nn_count) [+ (feature_count,)] containing the pairwise distances or feature-dimension-wise differences (extra feature_count dimension) between all pairs of nearest neighbors for each batch element. Shape depends on the deformation.

• batch_nn_targets – Tensor of floats of shape (batch_count, nn_count) [+ (response_count,)] containing the expected (possibly multivariate) response for each nearest neighbor of each batch element.

make_train_tensors(batch_indices, batch_nn_indices, train_features, train_targets, **kwargs)

Create the metric and target tensors needed for training.

Similar to make_predict_tensors() but returns the additional batch_targets matrix, which is only defined for a batch of training data.

Parameters:

• batch_indices (ndarray) – A vector of integers of shape (batch_count,) identifying the training batch of observations to be approximated.

• batch_nn_indices (ndarray) – A matrix of integers of shape (batch_count, nn_count) listing the nearest neighbor indices for all observations in the batch.

• train_features (ndarray) – The full floating point training data matrix of shape (train_count, ...).

• train_targets (ndarray) – A matrix of shape (train_count, ...) whose rows are vector-valued responses for each training element.

Return type:

Tuple[ndarray, ndarray, ndarray, ndarray]

Returns:

• crosswise_tensor – A tensor of shape (batch_count, nn_count) [+ (feature_count,)] containing the crosswise distances or feature-dimension-wise differences (extra feature_count dimension) between the batch elements and each of their nearest neighbors. Shape depends on the deformation.

• pairwise_tensor – A tensor of shape (batch_count, nn_count, nn_count) [+ (feature_count,)] containing the pairwise distances or feature-dimension-wise differences (extra feature_count dimension) between all pairs of nearest neighbors for each batch element. Shape depends on the deformation.

• batch_targets – Matrix of floats of shape (batch_count,) [+ (response_count,) containing the (possibly multivariate) expected response for each batch element.

• batch_nn_targets – Tensor of floats of shape (batch_count, nn_count) [+ (response_count,)] containing the expected (possibly multivariate) response for each nearest neighbor of each batch element.

optimize_scale(pairwise_diffs, nn_targets)

Optimize the value of the \(sigma^2\) scale parameter.

Uses the optimization method specified by the type of the scale parameter to optimize its value.

Parameters:

• pairwise_diffs (ndarray) – A tensor of shape (batch_count, nn_count, nn_count) [+ (feature_count,)] containing the pairwise distances or feature-dimension-wise differences (extra feature_count dimension) between all pairs of nearest neighbors for each batch element.

• nn_targets (ndarray) – Tensor of floats of shape (batch_count, nn_count) [+ (response_count,)] containing the expected (possibly multivariate) response for each nearest neighbor of each batch element.

Returns:

A reference to this model with a freshly-optimized scale parameter.

posterior_mean(Kin, Kcross, batch_nn_targets)

Returns the posterior mean from the provided covariance, cross-covariance, and target tensors.

Computes parallelized local solves of systems of linear equations using the last two dimensions of Kin along with Kcross and batch_nn_targets to predict responses in terms of the posterior mean. Assumes that kernel tensor Kin and cross-covariance matrix Kcross are already computed and given as arguments.

Returns the predicted response in the form of a posterior mean for each element of the batch of observations, as computed in Equation (3.4) of [muyskens2021muygps]. Given observation set \(X\) with responses \(Y\), noise prior set \(\varepsilon\), and kernel function \(Kin_\theta(\cdot, \cdot)\), computes the following for each prediction element \(\mathbf{z}_i\) with nearest neighbors index set \(N_i\):

\[\widehat{Y} (\mathbf{z}_i \mid X_{N_i}) = \sigma^2 Kin_\theta (\mathbf{z}_i, X_{N_i}) (Kin_\theta (X_{N_i}, X_{N_i}) + \varepsilon_{N_i})^{-1} Y(X_{N_i}).\]

Parameters:

• Kin (ndarray) – A tensor of shape (batch_count,) + in_shape + in_shape containing the pairwise, possibly multivariate covariance among the neighborhood for each batch element.

• Kcross (ndarray) – A tensor of shape (batch_count,) + out_shape listing the (possibly multivariate) cross-covariance between each batch element and its nearest neighbors.

• batch_nn_targets (ndarray) – A tensor of shape (batch_count,) + in_shape [+ (response_count,)] listing the (possibly multivariate) responses for the nearest neighbors of each batch element.

Return type:

ndarray

Returns:

A matrix of shape (batch_count,) [+ (response_count,)] listing the (possibly multivariate) predicted response for each batch element.

posterior_variance(Kin, Kcross)

Returns the posterior variance from the provided covariance and cross-covariance tensors.

Return the local posterior variances of each prediction, corresponding to the diagonal elements of a covariance matrix. Given observation set \(X\) with responses \(Y\), noise prior set \(\varepsilon\), and kernel function \(Kin_\theta(\cdot, \cdot)\), computes the following for each prediction element \(\mathbf{z}_i\) with nearest neighbors index set \(N_i\):

\[Var \left ( \widehat{Y} (\mathbf{z}_i \mid X_{N_i}) \right) = \sigma^2 \left ( Kin_\theta (\mathbf{z}_i, \mathbf{z}_i) - Kin_\theta (\mathbf{z}_i, X_{N_i}) \left ( Kin_\theta (X_{N_i}, X_{N_i} \right ) + \varepsilon_{N_i})^{-1} Kin_\theta (X_{N_i}, \mathbf{z}_i) \right ).\]

Parameters:

• Kin (ndarray) – A tensor of shape (batch_count,) + in_shape + in_shape containing the pairwise, possibly multivariate covariance among the neighborhood for each batch element.

• Kcross (ndarray) – A tensor of shape (batch_count,) + out_shape listing the (possibly multivariate) cross-covariance between each batch element and its nearest neighbors.

Return type:

ndarray

Returns:

A matrix of shape (batch_count,) [+ (response_count, response_count)] consisting of the (possibly blockwise) diagonal elements of the posterior variance.