kernels

Hyperparameters and kernel functors

Defines kernel functors (inheriting KernelFn) that transform crosswise difference tensors into cross-covariance matrices and pairwise difference matrices into covariance or kernel tensors.

See the following example to initialize an MuyGPyS.gp.kernels.Matern object. Other kernel functors are similar, but require different hyperparameters.

Example

>>> from MuyGPyS.gp.kernels import Matern
>>> kern = Matern(
...     smoothness=Parameter("log_sample", (0.1, 2.5)),
...     deformation=Isotropy(
...         metric=l2,
...         length_scale=Parameter(1.0),
...     ),
... )

One uses a previously computed pairwise_diffs tensor (see MuyGPyS.gp.tensor.pairwise_tensor()) to compute a kernel tensor whose second two dimensions contain square kernel matrices. Similarly, one uses a previously computed crosswise_diffs matrix (see MuyGPyS.gp.tensor.crosswise_diffs()) to compute a cross-covariance matrix. See the following example, which assumes that you have already constructed the difference numpy.ndarrays and the kernel kern as shown above.

Example

>>> Kin = kern(pairwise_diffs)
>>> Kcross = kern(crosswise_diffs)

class MuyGPyS.gp.kernels.kernel_fn.KernelFn(deformation)

Bases: object

A kernel functor.

Base class for kernel functors that include a hyperparameter Dict and a call mechanism.

Parameters:

kwargs – Ignored (by this base class) keyword arguments.

__call__(diffs, **kwargs)

Call self as a function.

Return type:

ndarray

get_opt_params()

Report lists of unfixed hyperparameter names, values, and bounds.

Return type:

Tuple[List[str], List[float], List[Tuple[float, float]]]

Returns:

names:

A list of unfixed hyperparameter names.

params:

A list of unfixed hyperparameter values.

bounds:

A list of unfixed hyperparameter bound tuples.

set_params(**kwargs)

Reset hyperparameters using hyperparameter dict(s).

Parameters:

kwargs – Hyperparameter kwargs.

Return type:

None

class MuyGPyS.gp.kernels.rbf.RBF(deformation=<MuyGPyS.gp.deformation.isotropy.Isotropy object>, _backend_fn=<function _rbf_fn>, _backend_ones=<function fix_type.<locals>.typed_fn.<locals>.fn_wrapper>, _backend_zeros=<function fix_type.<locals>.typed_fn.<locals>.fn_wrapper>, _backend_squeeze=<function squeeze>)

The radial basis function (RBF) or squared-exponential kernel.

The RBF kernel includes a parameterized scaled distance function \(d_\ell(\cdot, \cdot)\).

The kernel is defined by

\[Kin(x_i, x_j) = \exp\left(- d_\ell(x_i, x_j)\right).\]

Typically, \(d(\cdot,\cdot)\) is the squared Euclidean distance or second frequency moment of the difference of the operands.

Parameters:

deformation (DeformationFn) – The deformation functor to be used. Includes length_scale hyperparameter information via the MuyGPyS.gp.deformation module

__call__(diffs, **kwargs)

Compute RBF kernel(s) from a difference tensor.

Parameters:

diffs (ndarray) – A tensor of pairwise or crosswise distances or distances of shape (data_count, nn_count, nn_count) [+ (feature_count,)] or (data_count, nn_count) [+ (feature_count,)]. The final feature_count dimension is only required for feature-dimension-wise deformations such as Anisotropy.

Return type:

ndarray

Returns:

A cross-covariance tensor of shape (data_count,) + out_shape or a covariance tensor of shape (data_count,) + in_shape + in_shape.

get_opt_fn()

Return a kernel function with fixed parameters set.

Assumes that optimization parameter literals will be passed as keyword arguments.

Return type:

Callable

Returns:

A function implementing the kernel where all fixed parameters are set. The function expects keyword arguments corresponding to current hyperparameter values for unfixed parameters.

class MuyGPyS.gp.kernels.matern.Matern(smoothness=<MuyGPyS.gp.hyperparameter.scalar.Parameter object>, deformation=<MuyGPyS.gp.deformation.isotropy.Isotropy object>, _backend_ones=<function fix_type.<locals>.typed_fn.<locals>.fn_wrapper>, _backend_zeros=<function fix_type.<locals>.typed_fn.<locals>.fn_wrapper>, _backend_squeeze=<function squeeze>, **_backend_fns)

The Matérn kernel.

The Màtern kernel includes a parameterized deformation model \(d_\ell(\cdot, \cdot)\) and an additional smoothness parameter \(\nu>0\). \(\nu\) is proportional to the smoothness of the resulting function. As \(\nu\rightarrow\infty\), the kernel becomes equivalent to the RBF kernel. When \(\nu = 1/2\), the Matérn kernel is identical to the absolute exponential kernel. Important intermediate values are \(\nu=1.5\) (once differentiable functions) and \(\nu=2.5\) (twice differentiable functions).

The kernel is defined by

\[k(x_i, x_j) = \frac{1}{\Gamma(\nu)2^{\nu-1}}\Bigg( \frac{\sqrt{2\nu}}{l} d_\ell(x_i , x_j ) \Bigg)^\nu Kin_\nu\Bigg( \frac{\sqrt{2\nu}}{l} d(x_i , x_j )\Bigg),\]

where \(Kin_{\nu}(\cdot)\) is a modified Bessel function and \(\Gamma(\cdot)\) is the gamma function. Typically, \(d(\cdot,\cdot)\) is the Euclidean distance or \(\ell_2\) norm of the difference of the operands.

Parameters:

• smoothness (Parameter) – A parameter determining the differentiability of the function distribution.

• deformation (DeformationFn) – The deformation functor to be used. Includes length_scale hyperparameter information via the MuyGPyS.gp.deformation module.

__call__(diffs, **kwargs)

Compute Matern kernels from distance tensor.

Takes inspiration from scikit-learn’s implementation.

Parameters:

diffs – A tensor of pairwise or crosswise distances or distances of shape (data_count, nn_count, nn_count) [+ (feature_count,)] or (data_count, nn_count) [+ (feature_count,)]. The final feature_count dimension is only required for feature-dimension-wise deformations such as Anisotropy.

Returns:

A cross-covariance tensor of shape (data_count,) + out_shape or a covariance tensor of shape (data_count,) + in_shape + in_shape.

get_opt_fn()

Return a kernel function with fixed parameters set.

Assumes that optimization parameter literals will be passed as keyword arguments.

Return type:

Callable

Returns:

A function implementing the kernel where all fixed parameters are set. The function expects keyword arguments corresponding to current hyperparameter values for unfixed parameters.

get_opt_params()

Report lists of unfixed hyperparameter names, values, and bounds.

Return type:

Tuple[List[str], List[float], List[Tuple[float, float]]]

Returns:

names:

A list of unfixed hyperparameter names.

params:

A list of unfixed hyperparameter values.

bounds:

A list of unfixed hyperparameter bound tuples.