mirror of
https://github.com/SheffieldML/GPy.git
synced 2026-04-25 21:06:23 +02:00
183 lines
7.2 KiB
ReStructuredText
183 lines
7.2 KiB
ReStructuredText
********************
|
|
Creating new kernels
|
|
********************
|
|
|
|
We will see in this tutorial how to create new kernels in GPy. We will also give details on how to implement each function of the kernel and illustrate with a running example: the rational quadratic kernel.
|
|
|
|
Structure of a kernel in GPy
|
|
============================
|
|
|
|
In GPy a kernel object is made of a list of kernpart objects, which correspond to symetric positive definite functions. More precisely, the kernel should be understood as the sum of the kernparts. In order to implement a new covariance, the following steps must be followed
|
|
|
|
1. implement the new covariance as a kernpart object
|
|
2. update the constructors that allow to use the kernpart as a kern object
|
|
3. update the __init__.py file
|
|
|
|
Theses three steps are detailed below.
|
|
|
|
Implementing a kernpart object
|
|
==============================
|
|
|
|
We advise the reader to start with copy-pasting an existing kernel and to modify the new file. We will now give a description of the various functions that can be found in a kernpart object.
|
|
|
|
**Header**
|
|
|
|
The header is similar to all kernels::
|
|
|
|
from kernpart import kernpart
|
|
import numpy as np
|
|
|
|
class rational_quadratic(kernpart):
|
|
|
|
**__init__(self,D, param1, param2, ...)**
|
|
|
|
The implementation of this function in mandatory.
|
|
|
|
For all kernparts the first parameter ``D`` corresponds to the dimension of the input space, and the following parameters stand for the parameterization of the kernel.
|
|
|
|
The following attributes are compulsory: ``self.D`` (the dimension, integer), ``self.name`` (name of the kernel, string), ``self.Nparam`` (number of parameters, integer).::
|
|
|
|
def __init__(self,D,variance=1.,lengthscale=1.,power=1.):
|
|
assert D == 1, "For this kernel we assume D=1"
|
|
self.D = D
|
|
self.Nparam = 3
|
|
self.name = 'rat_quad'
|
|
self.variance = variance
|
|
self.lengthscale = lengthscale
|
|
self.power = power
|
|
|
|
**_get_params(self)**
|
|
|
|
The implementation of this function in mandatory.
|
|
|
|
This function returns a one dimensional array of length ``self.Nparam`` containing the value of the parameters.::
|
|
|
|
def _get_params(self):
|
|
return np.hstack((self.variance,self.lengthscale,self.power))
|
|
|
|
**_set_params(self,x)**
|
|
|
|
The implementation of this function in mandatory.
|
|
|
|
The input is a one dimensional array of length ``self.Nparam`` containing the value of the parameters. The function has no output but it updates the values of the attribute associated to the parameters (such as ``self.variance``, ``self.lengthscale``, ...).::
|
|
|
|
def _set_params(self,x):
|
|
self.variance = x[0]
|
|
self.lengthscale = x[1]
|
|
self.power = x[2]
|
|
|
|
**_get_param_names(self)**
|
|
|
|
The implementation of this function in mandatory.
|
|
|
|
It returns a list of strings of length ``self.Nparam`` corresponding to the parameter names.::
|
|
|
|
def _get_param_names(self):
|
|
return ['variance','lengthscale','power']
|
|
|
|
**K(self,X,X2,target)**
|
|
|
|
The implementation of this function in mandatory.
|
|
|
|
This function is used to compute the covariance matrix associated with the inputs X, X2 (np.arrays with arbitrary number of line (say :math:`n_1`, :math:`n_2`) and ``self.D`` columns). This function does not returns anything but it adds the :math:`n_1 \times n_2` covariance matrix to the kernpart to the object ``target`` (a :math:`n_1 \times n_2` np.array). This trick allows to compute the covariance matrix of a kernel containing many kernparts with a limited memory use.::
|
|
|
|
def K(self,X,X2,target):
|
|
if X2 is None: X2 = X
|
|
dist2 = np.square((X-X2.T)/self.lengthscale)
|
|
target += self.variance*(1 + dist2/2.)**(-self.power)
|
|
|
|
**Kdiag(self,X,target)**
|
|
|
|
The implementation of this function in mandatory.
|
|
|
|
This function is similar to ``K`` but it computes only the values of the kernel on the diagonal. Thus, ``target`` is a 1-dimensional np.array of length :math:`n_1`.::
|
|
|
|
def Kdiag(self,X,target):
|
|
target += self.variance
|
|
|
|
|
|
**dK_dtheta(self,dL_dK,X,X2,target)**
|
|
|
|
This function is required for the optimization of the parameters.
|
|
|
|
Computes the derivative of the likelihood. As previously, the values are added to the object target which is a 1-dimensional np.array of length ``self.Nparam``. For example, if the kernel is parameterized by :math:`\sigma^2,\ \theta`, then :math:`\frac{dL}{d\sigma^2} = \frac{dL}{d K} \frac{dK}{d\sigma^2}` is added to the first element of target and :math:`\frac{dL}{d\theta} = \frac{dL}{d K} \frac{dK}{d\theta}` to the second.::
|
|
|
|
def dK_dtheta(self,dL_dK,X,X2,target):
|
|
if X2 is None: X2 = X
|
|
dist2 = np.square((X-X2.T)/self.lengthscale)
|
|
|
|
dvar = (1 + dist2/2.)**(-self.power)
|
|
dl = self.power * self.variance * dist2 * self.lengthscale**(-3) * (1 + dist2/2./self.power)**(-self.power-1)
|
|
dp = - self.variance * np.log(1 + dist2/2.) * (1 + dist2/2.)**(-self.power)
|
|
|
|
target[0] += np.sum(dvar*dL_dK)
|
|
target[1] += np.sum(dl*dL_dK)
|
|
target[2] += np.sum(dp*dL_dK)
|
|
|
|
|
|
**dKdiag_dtheta(self,dL_dKdiag,X,target)**
|
|
|
|
This function is required for BGPLVM, sparse models and uncertain inputs.
|
|
|
|
As previously, target is an ``self.Nparam`` array and :math:`\frac{dL}{d Kdiag} \frac{dKdiag}{dparam}` is added to each element.::
|
|
|
|
def dKdiag_dtheta(self,dL_dKdiag,X,target):
|
|
target[0] += np.sum(dL_dKdiag)
|
|
# here self.lengthscale and self.power have no influence on Kdiag so target[1:] are unchanged
|
|
|
|
**dK_dX(self,dL_dK,X,X2,target)**
|
|
|
|
This function is required for GPLVM, BGPLVM, sparse models and uncertain inputs.
|
|
|
|
Computes the derivative of the likelihood with respect to the inputs ``X`` (a :math:`n \times D` np.array). The result is added to target which is a :math:`n \times D` np.array.::
|
|
|
|
def dK_dX(self,dL_dK,X,X2,target):
|
|
"""derivative of the covariance matrix with respect to X."""
|
|
if X2 is None: X2 = X
|
|
dist2 = np.square((X-X2.T)/self.lengthscale)
|
|
|
|
dX = -self.variance*self.power * (X-X2.T)/self.lengthscale**2 * (1 + dist2/2./self.power)**(-self.power-1)
|
|
target += np.sum(dL_dK*dX)
|
|
|
|
**dKdiag_dX(self,dL_dKdiag,X,target)**
|
|
|
|
This function is required for BGPLVM, sparse models and uncertain inputs. As for ``dKdiag_dtheta``, :math:`\frac{dL}{d Kdiag} \frac{dKdiag}{dX}` is added to each element of target.::
|
|
|
|
def dKdiag_dX(self,dL_dKdiag,X,target):
|
|
pass
|
|
|
|
**Psi statistics**
|
|
|
|
The psi statistics and their derivatives are required for BGPLVM and GPS with uncertain inputs.
|
|
|
|
The expressions of the psi statistics are:
|
|
|
|
TODO
|
|
|
|
For the rational quadratic we have:
|
|
|
|
TODO
|
|
|
|
Update the constructor
|
|
======================
|
|
|
|
Once the required functions have been implemented as a kernpart object, the file GPy/kern/constructors.py has to be updated to allow to build a kernel based on the kernpart object.
|
|
|
|
The following line should be added in the preamble of the file::
|
|
|
|
from rational_quadratic import rational_quadratic as rational_quadratic_part
|
|
|
|
as well as the following block::
|
|
|
|
def rational_quadratic(D,variance=1., lengthscale=1., power=1.):
|
|
part = rational_quadraticpart(D,variance, lengthscale, power)
|
|
return kern(D, [part])
|
|
|
|
|
|
Update initialization
|
|
=====================
|
|
|
|
The last step is to update the list of kernels imported from constructor in GPy/kern/__init__.py.
|
|
|
|
|
|
|