MuyGPS¶
- class MuyGPyS.gp.muygps.MuyGPS(kern='matern', eps={'val': 0.0}, **kwargs)[source]¶
Local Kriging Gaussian Process.
Performs approximate GP inference by locally approximating an observation’s response using its nearest neighbors. Implements the MuyGPs algorithm as articulated in [muyskens2021muygps].
Kernels accept different hyperparameter dictionaries specifying hyperparameter settings. Keys can include
val
andbounds
.bounds
must be either a len == 2 iterable container whose elements are scalars in increasing order, or the stringfixed
. Ifbounds == fixed
(the default behavior), the hyperparameter value will remain fixed during optimization.val
must be either a scalar (within the range of the upper and lower bounds if given) or the strings"sample"
orlog_sample"
, which will randomly sample a value within the range given by the bounds.In addition to individual kernel hyperparamters, each MuyGPS object also possesses a homoscedastic \(\varepsilon\) noise parameter and a vector of \(\sigma^2\) indicating the scale parameter associated with the posterior variance of each dimension of the response.
\(\sigma^2\) is the only parameter assumed to be a training target by default, and is treated differently from all other hyperparameters. All other training targets must be manually specified in
k_kwargs
.Example
>>> from MuyGPyS.gp.muygps import MuyGPS >>> k_kwargs = { ... "kern": "rbf", ... "metric": "F2", ... "eps": {"val": 1e-5}, ... "nu": {"val": 0.38, "bounds": (0.1, 2.5)}, ... "length_scale": {"val": 7.2}, ... } >>> muygps = MuyGPS(**k_kwarg)
MuyGPyS depends upon linear operations on specially-constructed tensors in order to efficiently estimate GP realizations. One can use (see their documentation for details)
MuyGPyS.gp.distance.pairwise_distances()
to construct pairwise distance tensors andMuyGPyS.gp.distance.crosswise_distances()
to produce crosswise distance matrices thatMuyGPS
can then use to construct kernel tensors and cross-covariance matrices, respectively.We can easily realize kernel tensors using a
MuyGPS
object’skernel
functor once we have computed apairwise_dists
tensor and acrosswise_dists
matrix.Example
>>> K = muygps.kernel(pairwise_dists) >>> Kcross = muygps.kernel(crosswise_dists)
- Parameters
kern (
str
) – The kernel to be used. Each kernel supports different hyperparameters that can be specified in kwargs. Currently supports onlymatern
andrbf
.eps (
Dict
[str
,Union
[float
,Tuple
[float
,float
]]]) – A hyperparameter dict.kwargs – Addition parameters to be passed to the kernel, possibly including additional hyperparameter dicts and a metric keyword.
- fixed()[source]¶
Checks whether all kernel and model parameters are fixed.
This is a convenience utility to determine whether optimization is required.
- Return type
bool
- Returns
Returns
True
if all parameters are fixed, andFalse
otherwise.
- get_array_opt_mean_fn()[source]¶
Return a posterior mean function for use in optimization.
This function is designed for use with
MuyGPyS.optimize.chassis.optimize_from_tensors()
withopt_method="scipy"
, and assumes that the optimization parameters will be passed in an(optim_count,)
vector whereeps
is either the last element or is not included.- Return type
Callable
- Returns
A function implementing posterior mean, where
eps
is either fixed or takes updating values during optimization. The function expects a list of current hyperparameter values for unfixed parameters, which are expected to occur in a certain order matching how they are set in~MuyGPyS.gp.muygps.MuyGPS.get_optim_params()
.
- get_kwargs_opt_mean_fn()[source]¶
Return a posterior mean function for use in optimization.
This function is designed for use with
MuyGPyS.optimize.chassis.optimize_from_tensors()
withopt_method="bayesian"
, and assumes that eithereps
will be passed via a keyword argument or not at all.- Return type
Callable
- Returns
A function implementing the posterior mean, where
eps
is either fixed or takes updating values during optimization. The function expects keyword arguments corresponding to current hyperparameter values for unfixed parameters.
- get_opt_mean_fn(opt_method)[source]¶
Return a posterior mean function for use in optimization.
This function is designed for use with
MuyGPyS.optimize.chassis.optimize_from_tensors()
. Theopt_method
parameter determines the format of the returned function.- Return type
Callable
- Returns
A function implementing a posterior mean, where
eps
is either fixed or takes updating values during optimization. The format of the function depends uponopt_method
.
- get_opt_var_fn(opt_method)[source]¶
Return a posterior variance function for use in optimization.
This function is designed for use with
MuyGPyS.optimize.chassis.optimize_from_tensors()
. Theopt_method
parameter determines the format of the returned function.- Return type
Callable
- Returns
A function implementing posterior variance, where
eps
is either fixed or takes updating values during optimization. The format of the function depends uponopt_method
.
- get_optim_params()[source]¶
Return lists of unfixed hyperparameter names, values, and bounds.
- Return type
Tuple
[List
[str
],ndarray
,ndarray
]- Returns
names – A list of unfixed hyperparameter names.
params – A list of unfixed hyperparameter values.
bounds – A list of unfixed hyperparameter bound tuples.
- regress(K, Kcross, batch_nn_targets, variance_mode=None, apply_sigma_sq=True)[source]¶
Performs simultaneous regression on provided covariance, cross-covariance, and target.
Computes parallelized local solves of systems of linear equations using the last two dimensions of
K
along withKcross
andbatch_nn_targets
to predict responses in terms of the posterior mean. Also computes the posterior variance ifvariance_mode
is set appropriately. Assumes that kernel tensorK
and cross-covariance matrixKcross
are already computed and given as arguments. To implicitly construct these values from indices (useful if the kernel or distance tensors and matrices are not needed for later reference) instead useregress_from_indices()
.Returns the predicted response in the form of a posterior mean for each element of the batch of observations, as computed in Equation (3.4) of [muyskens2021muygps]. For each batch element \(\mathbf{x}_i\), we compute
\[\widehat{Y}_{NN} (\mathbf{x}_i \mid X_{N_i}) = K_\theta (\mathbf{x}_i, X_{N_i}) (K_\theta (X_{N_i}, X_{N_i}) + \varepsilon I_k)^{-1} Y(X_{N_i}).\]Here \(X_{N_i}\) is the set of nearest neighbors of \(\mathbf{x}_i\) in the training data, \(K_\theta\) is the kernel functor specified by
self.kernel
, \(\varepsilon I_k\) is a diagonal homoscedastic noise matrix whose diagonal is the value of theself.eps
hyperparameter, and \(Y(X_{N_i})\) is the(nn_count, respones_count)
matrix of responses of the nearest neighbors given by the second two dimensions of thebatch_nn_targets
argument.If
variance_mode == "diagonal"
, also return the local posterior variances of each prediction, corresponding to the diagonal elements of a covariance matrix. For each batch element \(\mathbf{x}_i\), we compute\[Var(\widehat{Y}_{NN} (\mathbf{x}_i \mid X_{N_i})) = K_\theta (\mathbf{x}_i, \mathbf{x}_i) - K_\theta (\mathbf{x}_i, X_{N_i}) (K_\theta (X_{N_i}, X_{N_i}) + \varepsilon I_k)^{-1} K_\theta (X_{N_i}, \mathbf{x}_i).\]- Parameters
K (
array
) – A tensor of shape(batch_count, nn_count, nn_count)
containing the(nn_count, nn_count
-shaped kernel matrices corresponding to each of the batch elements.Kcross (
array
) – A tensor of shape(batch_count, nn_count)
containing the1 x nn_count
-shaped cross-covariance matrix corresponding to each of the batch elements.batch_nn_targets (
array
) – A tensor of shape(batch_count, nn_count, response_count)
whose last dimension lists the vector-valued responses for the nearest neighbors of each batch element.variance_mode (
Optional
[str
]) – Specifies the type of variance to return. Currently supports"diagonal"
and None. If None, report no variance term.apply_sigma_sq (
bool
) – Indicates whether to scale the posterior variance bysigma_sq
. Unused ifvariance_mode is None
orsigma_sq.trained() is False
.
- Return type
Union
[ndarray
,Tuple
[ndarray
,ndarray
]]- Returns
responses – A matrix of shape
(batch_count, response_count,)
whose rows are the predicted response for each of the given indices.diagonal_variance – A vector of shape
(batch_count,)
consisting of the diagonal elements of the posterior variance, or a matrix of shape(batch_count, response_count)
for a multidimensional response. Only returned wherevariance_mode == "diagonal"
.
- regress_from_indices(indices, nn_indices, test, train, targets, variance_mode=None, apply_sigma_sq=True, return_distances=False, indices_by_rank=False)[source]¶
Performs simultaneous regression on a list of observations.
This is similar to the old regress API in that it implicitly creates and discards the distance and kernel tensors and matrices. If these data structures are needed for later reference, instead use
regress()
.- Parameters
indices (
ndarray
) – An integral vector of shape(batch_count,)
indices of the observations to be approximated.nn_indices (
ndarray
) – An integral matrix of shape(batch_count, nn_count)
listing the nearest neighbor indices for all observations in the test batch.test (
ndarray
) – The full testing data matrix of shape(test_count, feature_count)
.train (
ndarray
) – The full training data matrix of shape(train_count, feature_count)
.targets (
ndarray
) – A matrix of shape(train_count, response_count)
whose rows are vector-valued responses for each training element.variance_mode (
Optional
[str
]) – Specifies the type of variance to return. Currently supports"diagonal"
and None. If None, report no variance term.apply_sigma_sq (
bool
) – Indicates whether to scale the posterior variance bysigma_sq
. Unused ifvariance_mode is None
orsigma_sq.trained() is False
.return_distances (
bool
) – IfTrue
, returns a(test_count, nn_count)
matrix containing the crosswise distances between the test elements and their nearest neighbor sets and a(test_count, nn_count, nn_count)
tensor containing the pairwise distances between the test data’s nearest neighbor sets.indices_by_rank (
bool
) – IfTrue
, construct the tensors using local indices with no communication. Only for use in MPI mode.
- Return type
Union
[ndarray
,Tuple
[ndarray
,ndarray
],Tuple
[ndarray
,ndarray
,ndarray
],Tuple
[ndarray
,ndarray
,ndarray
,ndarray
]]- Returns
responses – A matrix of shape
(batch_count, response_count,)
whose rows are the predicted response for each of the given indices.diagonal_variance – A vector of shape
(batch_count,)
consisting of the diagonal elements of the posterior variance, or a matrix of shape(batch_count, response_count)
for a multidimensional response. Only returned wherevariance_mode == "diagonal"
.crosswise_dists – A matrix of shape
(test_count, nn_count)
whose rows list the distance of the corresponding test element to each of its nearest neighbors. Only returned ifreturn_distances is True
.pairwise_dists – A tensor of shape
(test_count, nn_count, nn_count,)
whose latter two dimensions contain square matrices containing the pairwise distances between the nearest neighbors of the test elements. Only returned ifreturn_distances is True
.