kernels

Hyperparameters and kernel functors

Defines kernel functors (inheriting MuyGPyS.gp.kernels.KernelFn) that transform crosswise distance matrices into cross-covariance matrices and pairwise distance matrices into covariance or kernel matrices.

Hyperparameters are expected to be provided in Dict form with the keys "val" and "bounds". "bounds" is either a 2-tuple indicating a lower and upper optimization bound or the string "fixed", which exempts the hyperparameter from optimization ("fixed" is the default behavior if bounds is unspecified). "val" is either a floating point value (within the range between the upper and lower bounds if specified).

See the following example to initialize an MuyGPyS.gp.kernels.Matern object. Other kernel functors are similar, but require different hyperparameters.

Example

>>> from MuyGPyS.gp.kernels import Matern
>>> kern = Matern(
...         nu = {"val": "log_sample", "bounds": (0.1, 2.5)},
...         length_scale = {"val": 7.2},
...         metric = "l2",
... }

One uses a previously computed pairwise_dists tensor (see MuyGPyS.gp.distance.pairwise_distance()) to compute a kernel tensor whose second two dimensions contain square kernel matrices. Similarly, one uses a previously computed crosswise_dists matrix (see MuyGPyS.gp.distance.crosswise_distance()) to compute a cross-covariance matrix. See the following example, which assumes that you have already constructed the distance numpy.nparrays and the kernel kern as shown above.

Example

>>> K = kern(pairwise_dists)
>>> Kcross = kern(crosswise_dists)

class MuyGPyS.gp.kernels.Hyperparameter(val, bounds)

Bases: object

A MuyGPs kernel or model Hyperparameter.

Hyperparameters are defined by a value and optimization bounds. Values must be scalar numeric types, and bounds are either a len == 2 iterable container whose elements are numeric scalars in increasing order, or the string fixed. If bounds == "fixed" (the default behavior), the hyperparameter value will remain fixed during optimization. val must remain within the range of the upper and lower bounds, if not fixed.

Parameters

• val (Union[str, float]) – A scalar within the range of the upper and lower bounds (if given). val can also be the strings "sample" or "log_sample", which will result in randomly sampling a value within the range given by the bounds.

• bounds (Union[str, Tuple[float, float]]) – Iterable container of len 2 containing lower and upper bounds (in that order), or the string "fixed".

Raises

• ValueError – Any bounds string other than "fixed" will produce an error.

• ValueError – A non-iterable non-string type for bounds will produce an error.

• ValueError – A bounds iterable of len other than 2 will produce an error.

• ValueError – Iterable bounds values of non-numeric types will produce an error.

• ValueError – A lower bound that is not less than an upper bound will produce an error.

• ValueError – val == "sample" or val == "log_sample" will produce an error if self._bounds == "fixed".

• ValueError – Any string other than "sample" or "log_sample" will produce an error.

• ValueError – A val outside of the range specified by self._bounds will produce an error.

__call__()

Value accessor.

Return type

float

Returns

The current value of the hyperparameter.

get_bounds()

Bounds accessor.

Return type

Union[str, Tuple[float, float]]

Returns

The string "fixed" or the lower and upper bound tuple.

class MuyGPyS.gp.kernels.KernelFn(**kwargs)

Bases: object

A kernel functor.

Base class for kernel functors that include a hyperparameter Dict and a call mechanism.

Parameters

kwargs – Ignored (by this base class) keyword arguments.

__call__(squared_dists)

Call self as a function.

Return type

ndarray

set_params(**kwargs)

Reset hyperparameters using hyperparameter dict(s).

Parameters

kwargs – Hyperparameter kwargs.

Return type

None

class MuyGPyS.gp.kernels.Matern(nu={}, length_scale={}, metric='l2')

Bases: MuyGPyS.gp.kernels.KernelFn

The Màtern kernel.

The Màtern kernel includes a length scale parameter \(\ell>0\) and an additional smoothness parameter \(\nu>0\). \(\nu\) is inversely proportional to the smoothness of the resulting function. The Màtern kernel also depends upon a distance function \(d(\cdot, \cdot)\). As \(\nu\rightarrow\infty\), the kernel becomes equivalent to the RBF kernel. When \(\nu = 1/2\), the Matérn kernel becomes identical to the absolute exponential kernel. Important intermediate values are \(\nu=1.5\) (once differentiable functions) and \(\nu=2.5\) (twice differentiable functions). NOTE[bwp] We currently assume that the kernel is isotropic, so \(|\ell| = 1\).

The kernel is defined by

\[k(x_i, x_j) = \frac{1}{\Gamma(\nu)2^{\nu-1}}\Bigg( \frac{\sqrt{2\nu}}{l} d(x_i , x_j ) \Bigg)^\nu K_\nu\Bigg( \frac{\sqrt{2\nu}}{l} d(x_i , x_j )\Bigg),\]

where \(K_{\nu}(\cdot)\) is a modified Bessel function and \(\Gamma(\cdot)\) is the gamma function.

Typically, \(d(\cdot,\cdot)\) is the Euclidean distance or \(\ell_2\) norm of the difference of the operands.

Parameters

• nu (Dict[str, Union[str, float, Tuple[float, float]]]) – A hyperparameter dict defining the length_scale parameter.

• length_scale (Dict[str, Union[str, float, Tuple[float, float]]]) – A hyperparameter dict defining the length_scale parameter.

• metric (Optional[str]) – The distance function to be used. Defaults to "l2".

__call__(dists)

Compute Matern kernels from distance tensor.

Takes inspiration from [scikit-learn](https://github.com/scikit-learn/scikit-learn/blob/95119c13a/sklearn/gaussian_process/kernels.py#L1529)

Parameters

squared_dists – A matrix or tensor of pairwise distances (usually squared l2 or F2) of shape (data_count, nn_count, nn_count) or (data_count, nn_count). In the tensor case, matrix diagonals along last two dimensions are expected to be 0.

Returns

A cross-covariance matrix of shape (data_count, nn_count) or a tensor of shape (data_count, nn_count, nn_count) whose last two dimensions are kernel matrices.

set_params(**kwargs)

Reset hyperparameters using hyperparameter dict(s).

Parameters

kwargs – Hyperparameter kwargs.

Return type

None

class MuyGPyS.gp.kernels.RBF(length_scale={}, metric='F2')

Bases: MuyGPyS.gp.kernels.KernelFn

The radial basis function (RBF) or squared-exponential kernel.

The RBF kernel includes a single explicit length scale parameter \(\ell>0\), and depends upon a distance function \(d(\cdot, \cdot)\). NOTE[bwp] We currently assume that the kernel is isotropic, so \(|\ell| = 1\).

The kernel is defined by

\[K(x_i, x_j) = \exp\left(- \frac{d(x_i, x_j)}{2\ell^2}\right).\]

Typically, \(d(\cdot,\cdot)\) is the squared Euclidean distance or second frequency moment of the difference of the operands.

Parameters

• length_scale (Dict[str, Union[str, float, Tuple[float, float]]]) – A hyperparameter dict defining the length_scale parameter.

• metric (Optional[str]) – The distance function to be used. Defaults to "F2".

__call__(squared_dists)

Compute RBF kernel(s) from a distance matrix or tensor.

Parameters

squared_dists (ndarray) – A matrix or tensor of pairwise distances (usually squared l2 or F2) of shape (data_count, nn_count, nn_count) or (data_count, nn_count). In the tensor case, matrix diagonals along last two dimensions are expected to be 0.

Return type

ndarray

Returns

A cross-covariance matrix of shape (data_count, nn_count) or a tensor of shape (data_count, nn_count, nn_count) whose last two dimensions are kernel matrices.

set_params(**kwargs)

Reset hyperparameters using hyperparameter dict(s).

Parameters

kwargs – Hyperparameter kwargs.

Return type

None

class MuyGPyS.gp.kernels.SigmaSq

Bases: object

A \(\sigma^2\) covariance scale parameter.

\(\sigma^2\) is a scaling parameter that one multiplies with the found diagonal variances of a MuyGPyS.gp.muygps.MuyGPS or MuyGPyS.gp.muygps.MultivariateMuyGPS regression in order to obtain the predicted posterior variance. Trained values assume a number of dimensions equal to the number of response dimensions, and correspond to scalar scaling parameters along the corresponding dimensions.

__call__()

Value accessor.

Return type

ndarray

Returns

The current value of the hyperparameter.