apunkt/GPy
1
0
Fork 0
mirror of https://github.com/SheffieldML/GPy.git synced 2026-05-27 14:25:16 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

tutorials updated to comply with changes throught the code

Browse source
This commit is contained in:
Nicolas 2013-06-05 11:22:47 +01:00
parent 312cfebcb1
commit 5f899ed4f4
4 changed files with 41 additions and 46 deletions
Download patch file Download diff file Expand all files Collapse all files

20
doc/tuto_GP_regression.rst
View file

@ -23,9 +23,9 @@ Note that the observations Y include some noise.
The first step is to define the covariance kernel we want to use for the model. We choose here a kernel based on Gaussian kernel (i.e. rbf or square exponential)::
kernel = GPy.kern.rbf(D=1, variance=1., lengthscale=1.)
kernel = GPy.kern.rbf(input_dim=1, variance=1., lengthscale=1.)
The parameter ``D`` stands for the dimension of the input space. The parameters ``variance`` and ``lengthscale`` are optional. Note that many other kernels are implemented such as:
The parameter ``D`` stands for the dimension of the input space. The parameters ``variance`` and ``lengthscale`` are optional. Many other kernels are implemented such as:
* linear (``GPy.kern.linear``)
* exponential kernel (``GPy.kern.exponential``)
@ -50,7 +50,7 @@ gives the following output: ::
-----------------------------------------------------------------
rbf_variance | 1.0000 | | |
rbf_lengthscale | 1.0000 | | |
noise variance | 1.0000 | | |
noise_variance | 1.0000 | | |
.. figure:: Figures/tuto_GP_regression_m1.png
:align: center
@ -65,14 +65,14 @@ The default values of the kernel parameters may not be relevant for the current
There are various ways to constrain the parameters of the kernel. The most basic is to constrain all the parameters to be positive::
m.constrain_positive('')
m.ensure_default_constraints() # or similarly m.constrain_positive('')
but it is also possible to set a range on to constrain one parameter to be fixed. The parameter of ``m.constrain_positive`` is a regular expression that matches the name of the parameters to be constrained (as seen in ``print m``). For example, if we want the variance to be positive, the lengthscale to be in [1,10] and the noise variance to be fixed we can write::
m.unconstrain('') # Required to remove the previous constrains
m.constrain_positive('rbf_variance')
m.constrain_bounded('lengthscale',1.,10. )
m.constrain_fixed('noise',0.0025)
m.constrain_positive('.*rbf_variance')
m.constrain_bounded('.*lengthscale',1.,10. )
m.constrain_fixed('.*noise',0.0025)
Once the constrains have been imposed, the model can be optimized::
@ -80,7 +80,7 @@ Once the constrains have been imposed, the model can be optimized::
If we want to perform some restarts to try to improve the result of the optimization, we can use the ``optimize_restart`` function::
m.optimize_restarts(Nrestarts = 10)
m.optimize_restarts(num_restarts = 10)
Once again, we can use ``print(m)`` and ``m.plot()`` to look at the resulting model resulting model::
@ -89,7 +89,7 @@ Once again, we can use ``print(m)`` and ``m.plot()`` to look at the resulting mo
-----------------------------------------------------------------
rbf_variance | 0.8151 | (+ve) | |
rbf_lengthscale | 1.8037 | (1.0, 10.0) | |
noise variance | 0.0025 | Fixed | |
noise_variance | 0.0025 | Fixed | |
.. figure:: Figures/tuto_GP_regression_m2.png
:align: center
@ -122,9 +122,7 @@ Here is a 2 dimensional example::
m.constrain_positive('')
# optimize and plot
pb.figure()
m.optimize('tnc', max_f_eval = 1000)
m.plot()
print(m)

34
doc/tuto_creating_new_kernels.rst
View file

@ -29,18 +29,18 @@ The header is similar to all kernels: ::
class rational_quadratic(kernpart):
**__init__(self,D, param1, param2, ...)**
**__init__(self,input_dim, param1, param2, ...)**
The implementation of this function in mandatory.
For all kernparts the first parameter ``D`` corresponds to the dimension of the input space, and the following parameters stand for the parameterization of the kernel.
For all kernparts the first parameter ``input_dim`` corresponds to the dimension of the input space, and the following parameters stand for the parameterization of the kernel.
The following attributes are compulsory: ``self.D`` (the dimension, integer), ``self.name`` (name of the kernel, string), ``self.Nparam`` (number of parameters, integer). ::
The following attributes are compulsory: ``self.input_dim`` (the dimension, integer), ``self.name`` (name of the kernel, string), ``self.num_params`` (number of parameters, integer). ::
def __init__(self,D,variance=1.,lengthscale=1.,power=1.):
assert D == 1, "For this kernel we assume D=1"
self.D = D
self.Nparam = 3
def __init__(self,input_dim,variance=1.,lengthscale=1.,power=1.):
assert input_dim == 1, "For this kernel we assume input_dim=1"
self.input_dim = input_dim
self.num_params = 3
self.name = 'rat_quad'
self.variance = variance
self.lengthscale = lengthscale
@ -50,7 +50,7 @@ The following attributes are compulsory: ``self.D`` (the dimension, integer), ``
The implementation of this function in mandatory.
This function returns a one dimensional array of length ``self.Nparam`` containing the value of the parameters. ::
This function returns a one dimensional array of length ``self.num_params`` containing the value of the parameters. ::
def _get_params(self):
return np.hstack((self.variance,self.lengthscale,self.power))
@ -59,7 +59,7 @@ This function returns a one dimensional array of length ``self.Nparam`` containi
The implementation of this function in mandatory.
The input is a one dimensional array of length ``self.Nparam`` containing the value of the parameters. The function has no output but it updates the values of the attribute associated to the parameters (such as ``self.variance``, ``self.lengthscale``, ...). ::
The input is a one dimensional array of length ``self.num_params`` containing the value of the parameters. The function has no output but it updates the values of the attribute associated to the parameters (such as ``self.variance``, ``self.lengthscale``, ...). ::
def _set_params(self,x):
self.variance = x[0]
@ -70,7 +70,7 @@ The input is a one dimensional array of length ``self.Nparam`` containing the va
The implementation of this function in mandatory.
It returns a list of strings of length ``self.Nparam`` corresponding to the parameter names. ::
It returns a list of strings of length ``self.num_params`` corresponding to the parameter names. ::
def _get_param_names(self):
return ['variance','lengthscale','power']
@ -79,7 +79,7 @@ It returns a list of strings of length ``self.Nparam`` corresponding to the para
The implementation of this function in mandatory.
This function is used to compute the covariance matrix associated with the inputs X, X2 (np.arrays with arbitrary number of line (say :math:`n_1`, :math:`n_2`) and ``self.D`` columns). This function does not returns anything but it adds the :math:`n_1 \times n_2` covariance matrix to the kernpart to the object ``target`` (a :math:`n_1 \times n_2` np.array). This trick allows to compute the covariance matrix of a kernel containing many kernparts with a limited memory use. ::
This function is used to compute the covariance matrix associated with the inputs X, X2 (np.arrays with arbitrary number of line (say :math:`n_1`, :math:`n_2`) and ``self.input_dim`` columns). This function does not returns anything but it adds the :math:`n_1 \times n_2` covariance matrix to the kernpart to the object ``target`` (a :math:`n_1 \times n_2` np.array). This trick allows to compute the covariance matrix of a kernel containing many kernparts with a limited memory use. ::
def K(self,X,X2,target):
if X2 is None: X2 = X
@ -100,7 +100,7 @@ This function is similar to ``K`` but it computes only the values of the kernel
This function is required for the optimization of the parameters.
Computes the derivative of the likelihood. As previously, the values are added to the object target which is a 1-dimensional np.array of length ``self.Nparam``. For example, if the kernel is parameterized by :math:`\sigma^2,\ \theta`, then :math:`\frac{dL}{d\sigma^2} = \frac{dL}{d K} \frac{dK}{d\sigma^2}` is added to the first element of target and :math:`\frac{dL}{d\theta} = \frac{dL}{d K} \frac{dK}{d\theta}` to the second. ::
Computes the derivative of the likelihood. As previously, the values are added to the object target which is a 1-dimensional np.array of length ``self.input_dim``. For example, if the kernel is parameterized by :math:`\sigma^2,\ \theta`, then :math:`\frac{dL}{d\sigma^2} = \frac{dL}{d K} \frac{dK}{d\sigma^2}` is added to the first element of target and :math:`\frac{dL}{d\theta} = \frac{dL}{d K} \frac{dK}{d\theta}` to the second. ::
def dK_dtheta(self,dL_dK,X,X2,target):
if X2 is None: X2 = X
@ -119,7 +119,7 @@ Computes the derivative of the likelihood. As previously, the values are added t
This function is required for BGPLVM, sparse models and uncertain inputs.
As previously, target is an ``self.Nparam`` array and :math:`\frac{dL}{d Kdiag} \frac{dKdiag}{dparam}` is added to each element. ::
As previously, target is an ``self.num_params`` array and :math:`\frac{dL}{d Kdiag} \frac{dKdiag}{dparam}` is added to each element. ::
def dKdiag_dtheta(self,dL_dKdiag,X,target):
target[0] += np.sum(dL_dKdiag)
@ -129,7 +129,7 @@ As previously, target is an ``self.Nparam`` array and :math:`\frac{dL}{d Kdiag}
This function is required for GPLVM, BGPLVM, sparse models and uncertain inputs.
Computes the derivative of the likelihood with respect to the inputs ``X`` (a :math:`n \times D` np.array). The result is added to target which is a :math:`n \times D` np.array. ::
Computes the derivative of the likelihood with respect to the inputs ``X`` (a :math:`n \times d` np.array). The result is added to target which is a :math:`n \times d` np.array. ::
def dK_dX(self,dL_dK,X,X2,target):
"""derivative of the covariance matrix with respect to X."""
@ -169,9 +169,9 @@ The following line should be added in the preamble of the file::
as well as the following block ::
def rational_quadratic(D,variance=1., lengthscale=1., power=1.):
part = rational_quadraticpart(D,variance, lengthscale, power)
return kern(D, [part])
def rational_quadratic(input_dim,variance=1., lengthscale=1., power=1.):
part = rational_quadraticpart(input_dim,variance, lengthscale, power)
return kern(input_dim, [part])
Update initialization

24
doc/tuto_interacting_with_models.rst
View file

@ -18,6 +18,7 @@ All of the examples included in GPy return an instance
of a model class, and therefore they can be called in
the following way: ::
import numpy as np
import pylab as pb
pb.ion()
import GPy
@ -91,19 +92,17 @@ we can define a new array of values and change the parameters as follows: ::
If we call the function ``_get_params()`` again, we will obtain the new
parameters we have just set.
Parameters can be also set by name using the function ``_set()``. For example,
lets change the lengthscale to .5: ::
Parameters can be also set by name using dictionary notations. For example,
let's change the lengthscale to .5: ::
m.set('rbf_lengthscale',.5)
m['rbf_lengthscale'] = .5
``_set()`` function accepts regular expression as it first
input, and therefore all parameters matching that regular
expression are set to the given value. In this case rather
Here, the matching accepts a regular expression and therefore all parameters matching that regular expression are set to the given value. In this case rather
than passing as second output a single value, we can also
use a list of arrays. For example, lets change the inducing
inputs: ::
m.set('iip',np.arange(-4,0))
m['iip'] = np.arange(-5,0)
Getting the model's likelihood and gradients
===========================================
@ -129,10 +128,9 @@ we have been changing the parameters, the gradients are far from zero now.
Next we are going to show how to optimize the model setting different
restrictions on the parameters.
Once a constrain has been set on a parameter, it is not possible to
define a new constraint for it unless we explicitly remove the previous
one. The command to remove the constraints is ``unconstrain()``, and
just as the ``set()`` command, it also accepts regular expression.
Once a constrain has been set on a parameter, it is possible to remove it
with the command ``unconstrain()``, and
just as the previous matching commands, it also accepts regular expression.
In this case we will remove all the constraints: ::
m.unconstrain('')
@ -144,7 +142,7 @@ is to be positive. This is constraint is easily set
with the function ``constrain_positive()``. Regular expressions
are also accepted. ::
m.constrain_positive('var')
m.constrain_positive('.*var')
For convenience, GPy also provides a catch all function
which ensures that anything which appears to require
@ -179,7 +177,7 @@ however for the sake of the example we will tie the white noise
and the variance together. See `A kernel overview <tuto_kernel_overview.html>`_.
for a proper use of the tying capabilities.::
m.tie_params('e_var')
m.tie_params('.*e_var')
Optimizing the model
====================

9
doc/tuto_kernel_overview.rst
View file

@ -13,8 +13,8 @@ First we import the libraries we will need ::
For most kernels, the dimension is the only mandatory parameter to define a kernel object. However, it is also possible to specify the values of the parameters. For example, the three following commands are valid for defining a squared exponential kernel (ie rbf or Gaussian) ::
ker1 = GPy.kern.rbf(1) # Equivalent to ker1 = GPy.kern.rbf(D=1, variance=1., lengthscale=1.)
ker2 = GPy.kern.rbf(D=1, variance = .75, lengthscale=2.)
ker1 = GPy.kern.rbf(1) # Equivalent to ker1 = GPy.kern.rbf(input_dim=1, variance=1., lengthscale=1.)
ker2 = GPy.kern.rbf(input_dim=1, variance = .75, lengthscale=2.)
ker3 = GPy.kern.rbf(1, .5, .5)
A ``print`` and a ``plot`` functions are implemented to represent kernel objects. The commands ::
@ -144,9 +144,9 @@ When calling one of these functions, the parameters to constrain can either by s
k = k1 + k2 + k3
print k
k.constrain_positive('var')
k.constrain_positive('.*var')
k.constrain_fixed(np.array([1]),1.75)
k.tie_params('len')
k.tie_params('.*len')
k.unconstrain('white')
k.constrain_bounded('white',lower=1e-5,upper=.5)
print k
@ -212,7 +212,6 @@ Note the ties between the parameters of ``Kanova`` that reflect the links betwee
# Create GP regression model
m = GPy.models.GP_regression(X,Y,Kanova)
pb.figure(figsize=(5,5))
m.plot()
.. figure:: Figures/tuto_kern_overview_mANOVA.png