secml.ml.kernel

CKernel

class secml.ml.kernel.c_kernel.CKernel(batch_size=None)

Bases: secml.core.c_creator.CCreator

Abstract class that defines basic methods for kernels.

A kernel is a pairwise metric that compute the distance between sets of patterns.

Kernels can be considered similarity measures, i.e. s(a, b) > s(a, c) if objects a and b are considered “more similar” than objects a and c. A kernel must also be positive semi-definite.

Parameters

batch_sizeint or None, optional

Size of the batch used for kernel computation. Default None.

Attributes

class_type

Defines class type.

logger

Logger for current object.

verbose

Verbosity level of logger output.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
deepcopy(self)	Returns a deep copy of current class.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class parameters.
get_subclasses()	Get all the subclasses of the calling class.
gradient(self, x, v)	Calculates kernel gradient wrt vector ‘v’.
k(self, x[, y])	Compute kernel between x and y.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
similarity(self, x[, y])	Computes kernel.
timed([msg])	Timer decorator.

gradient(self, x, v)

Calculates kernel gradient wrt vector ‘v’.

Parameters

xCArray

First array of shape (n_x, n_features).

vCArray

Second array of shape (n_features, ) or (1, n_features).

Returns

kernel_gradientCArray

Kernel gradient of u with respect to vector v. Array of shape (n_x, n_features) if n_x > 1, else a flattened array of shape (n_features, ).

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernel import CKernelRBF

>>> array = CArray([[15,25],[45,55]])
>>> vector = CArray([2,5])
>>> print(CKernelRBF(gamma=1e-4).gradient(array, vector))
CArray([[0.002456 0.003779]
 [0.005567 0.006473]])

>>> print(CKernelRBF().gradient(vector, vector))
CArray([0. 0.])

k(self, x, y=None)

Compute kernel between x and y.

Parameters

xCArray

First array of shape (n_x, n_features).

yCArray, optional

Second array of shape (n_y, n_features). If not specified, the kernel k(x,x) is computed.

Returns

kernelCArray or scalar

Kernel between x and y. Array of shape (n_x, n_y) or scalar if both x and y are vector-like.

Notes

We use a batching strategy to optimize memory consumption during kernel computation. However, the parameter batch_size should be chosen wisely: a small cache can highly improve memory consumption but can significantly slow down the computation process.

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernel import CKernelRBF

>>> array1 = CArray([[15,25],[45,55]])
>>> array2 = CArray([[10,20],[40,50]])
>>> print(CKernelRBF().k(array1, array2))
CArray([[1.92875e-22 0.00000e+00]
 [0.00000e+00 1.92875e-22]])

>>> print(CKernelRBF().k(array1))
CArray([[1. 0.]
 [0. 1.]])

>>> vector = CArray([15,25])
>>> print(CKernelRBF().k(vector, array1))
CArray([[1. 0.]])
>>> print(CKernelRBF().k(array1, vector))
CArray([[1.]
 [0.]])
>>> print(CKernelRBF().k(vector, vector))
1.0

similarity(self, x, y=None)

Computes kernel. Wrapper of ‘k’ function.

See also

CKernel.k

Main computation interface for kernels.

CKernelChebyshevDistance

class secml.ml.kernel.c_kernel_chebyshev_distance.CKernelChebyshevDistance(gamma=1.0, batch_size=None)

Bases: secml.ml.kernel.c_kernel.CKernel

Chebyshev distances kernel.

Given matrices X and Y, this is computed as:

K(x, y) = max(|x - y|)

for each pair of rows in X and in Y.

Parameters

batch_sizeint or None, optional

Size of the batch used for kernel computation. Default None.

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernel.c_kernel_chebyshev_distance import CKernelChebyshevDistance

>>> print(CKernelChebyshevDistance().k(CArray([[1,2],[3,4]]), CArray([[5,6],[7,8]])))
CArray([[4. 6.]
 [2. 4.]])

>>> print(CKernelChebyshevDistance().k(CArray([[1,2],[3,4]])))
CArray([[0. 2.]
 [2. 0.]])

Attributes

class_type‘chebyshev-dist’

Defines class type.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
deepcopy(self)	Returns a deep copy of current class.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class parameters.
get_subclasses()	Get all the subclasses of the calling class.
gradient(self, x, v)	Calculates kernel gradient wrt vector ‘v’.
k(self, x[, y])	Compute kernel between x and y.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
similarity(self, x[, y])	Computes kernel.
timed([msg])	Timer decorator.

property gamma

Gamma parameter.

CKernelEuclidean

class secml.ml.kernel.c_kernel_euclidean.CKernelEuclidean(batch_size=None)

Bases: secml.ml.kernel.c_kernel.CKernel

Euclidean distances kernel.

Given matrices X and Y, this is computed by:

K(x, y) = sqrt(dot(x, x) - 2 * dot(x, y) + dot(y, y))

for each pair of rows in X and in Y. If parameter squared is True (default False), sqrt() operation is avoided.

Parameters

batch_sizeint or None, optional

Size of the batch used for kernel computation. Default None.

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernel.c_kernel_euclidean import CKernelEuclidean

>>> print(CKernelEuclidean().k(CArray([[1,2],[3,4]]), CArray([[10,20],[30,40]])))
CArray([[20.124612 47.801674]
 [17.464249 45.      ]])

>>> print(CKernelEuclidean().k(CArray([[1,2],[3,4]])))
CArray([[0.       2.828427]
 [2.828427 0.      ]])

Attributes

class_type‘euclidean’

Defines class type.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
deepcopy(self)	Returns a deep copy of current class.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class parameters.
get_subclasses()	Get all the subclasses of the calling class.
gradient(self, x, v[, squared])	Calculates Euclidean distances kernel gradient wrt vector ‘v’.
k(self, x[, y])	Compute kernel between x and y.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
similarity(self, x[, y])	Computes kernel.
timed([msg])	Timer decorator.

gradient(self, x, v, squared=False)

Calculates Euclidean distances kernel gradient wrt vector ‘v’.

The gradient of Euclidean distances kernel is given by:

dK(x,v)/dv = - (x - v) / k(x,v)   if squared = False (default)
dK(x,v)/dv = - 2 * (x - v)        if squared = True

Parameters

xCArray

First array of shape (n_x, n_features).

vCArray

Second array of shape (n_features, ) or (1, n_features).

squaredbool, optional

If True, return squared Euclidean distances. Default False

Returns

kernel_gradientCArray

Kernel gradient of x with respect to vector v. Array of shape (n_x, n_features) if n_x > 1, else a flattened array of shape (n_features, ).

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernel.c_kernel_euclidean import CKernelEuclidean

>>> array = CArray([[15,25],[45,55]])
>>> vector = CArray([2,5])
>>> print(CKernelEuclidean().gradient(array, vector))
CArray([[-0.544988 -0.838444]
 [-0.652039 -0.758185]])

>>> print(CKernelEuclidean().gradient(array, vector, squared=True))
CArray([[ -26  -40]
 [ -86 -100]])

>>> print(CKernelEuclidean().gradient(vector, vector))
CArray([0. 0.])

CKernelHamming

class secml.ml.kernel.c_kernel_hamming.CKernelHamming(batch_size=None)

Bases: secml.ml.kernel.c_kernel.CKernel

Hamming distance kernel.

Parameters

batch_sizeint or None, optional

Size of the batch used for kernel computation. Default None.

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernel.c_kernel_hamming import CKernelHamming

>>> print(CKernelHamming().k(CArray([[1,2],[3,4]]), CArray([[10,20],[30,40]])))
CArray([[1. 1.]
 [1. 1.]])

>>> print(CKernelHamming().k(CArray([[1,2],[3,4]])))
CArray([[0. 1.]
 [1. 0.]])

Attributes

class_type‘hamming’

Defines class type.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
deepcopy(self)	Returns a deep copy of current class.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class parameters.
get_subclasses()	Get all the subclasses of the calling class.
gradient(self, x, v)	Calculates kernel gradient wrt vector ‘v’.
k(self, x[, y])	Compute kernel between x and y.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
similarity(self, x[, y])	Computes kernel.
timed([msg])	Timer decorator.

CKernelHistIntersect

class secml.ml.kernel.c_kernel_histintersect.CKernelHistIntersect(batch_size=None)

Bases: secml.ml.kernel.c_kernel.CKernel

Histogram Intersection Kernel.

Given matrices X and Y, this is computed by:

K(x, y) = sum^n_i ( min(x[i], y[i]) )

for each pair of rows in X and in Y.

Parameters

batch_sizeint or None, optional

Size of the batch used for kernel computation. Default None.

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernel.c_kernel_histintersect import CKernelHistIntersect

>>> print(CKernelHistIntersect().k(CArray([[1,2],[3,4]]), CArray([[10,20],[30,40]])))
CArray([[3. 3.]
 [7. 7.]])

>>> print(CKernelHistIntersect().k(CArray([[1,2],[3,4]])))
CArray([[3. 3.]
 [3. 7.]])

Attributes

class_type‘hist-intersect’

Defines class type.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
deepcopy(self)	Returns a deep copy of current class.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class parameters.
get_subclasses()	Get all the subclasses of the calling class.
gradient(self, x, v)	Calculates kernel gradient wrt vector ‘v’.
k(self, x[, y])	Compute kernel between x and y.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
similarity(self, x[, y])	Computes kernel.
timed([msg])	Timer decorator.

CKernelLaplacian

class secml.ml.kernel.c_kernel_laplacian.CKernelLaplacian(gamma=1.0, batch_size=None)

Bases: secml.ml.kernel.c_kernel.CKernel

Laplacian Kernel.

Given matrices X and Y, this is computed by:

K(x, y) = exp(-gamma |x-y|)

for each pair of rows in X and in Y.

Parameters

gammafloat

Default is 1.0.

batch_sizeint or None, optional

Size of the batch used for kernel computation. Default None.

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernel.c_kernel_laplacian import CKernelLaplacian

>>> print(CKernelLaplacian(gamma=0.01).k(CArray([[1,2],[3,4]]), CArray([[10,0],[0,40]])))
CArray([[0.895834 0.677057]
 [0.895834 0.677057]])

>>> print(CKernelLaplacian().k(CArray([[1,2],[3,4]])))
CArray([[1.       0.018316]
 [0.018316 1.      ]])

Attributes

class_type‘laplacian’

Defines class type.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
deepcopy(self)	Returns a deep copy of current class.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class parameters.
get_subclasses()	Get all the subclasses of the calling class.
gradient(self, x, v)	Calculates laplacian kernel gradient wrt vector ‘v’.
k(self, x[, y])	Compute kernel between x and y.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
similarity(self, x[, y])	Computes kernel.
timed([msg])	Timer decorator.

property gamma

Gamma parameter.

gradient(self, x, v)

Calculates laplacian kernel gradient wrt vector ‘v’.

The gradient of laplacian kernel is given by:

dK(x,v)/dv =  gamma * k(x,v) * sign(x - v)

Parameters

xCArray

First array of shape (n_x, n_features).

vCArray

Second array of shape (n_features, ) or (1, n_features).

Returns

kernel_gradientCArray

Kernel gradient of x with respect to vector v. Array of shape (n_x, n_features) if n_x > 1, else a flattened array of shape (n_features, ).

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernel.c_kernel_laplacian import CKernelLaplacian

>>> array = CArray([[15,0], [0,55]])
>>> vector = CArray([2,5])
>>> print(CKernelLaplacian(gamma=0.01).gradient(array, vector))
CArray([[ 0.008353 -0.008353]
 [-0.005945  0.005945]])

>>> print(CKernelLaplacian().gradient(vector, vector))
CArray([0. 0.])

CKernelLinear

class secml.ml.kernel.c_kernel_linear.CKernelLinear(batch_size=None)

Bases: secml.ml.kernel.c_kernel.CKernel

Linear kernel.

Given matrices X and Y, this is computed by:

K(x, y) = x * y^T

for each pair of rows in X and in Y.

Parameters

batch_sizeint or None, optional

Size of the batch used for kernel computation. Default None.

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernel.c_kernel_linear import CKernelLinear

>>> print(CKernelLinear().k(CArray([[1,2],[3,4]]), CArray([[10,20],[30,40]])))
CArray([[ 50. 110.]
 [110. 250.]])

>>> print(CKernelLinear().k(CArray([[1,2],[3,4]])))
CArray([[ 5. 11.]
 [11. 25.]])

Attributes

class_type‘linear’

Defines class type.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
deepcopy(self)	Returns a deep copy of current class.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class parameters.
get_subclasses()	Get all the subclasses of the calling class.
gradient(self, x, v)	Calculates kernel gradient wrt vector ‘v’.
k(self, x[, y])	Compute kernel between x and y.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
similarity(self, x[, y])	Computes kernel.
timed([msg])	Timer decorator.

CKernelPoly

class secml.ml.kernel.c_kernel_poly.CKernelPoly(degree=2, gamma=1.0, coef0=1.0, batch_size=None)

Bases: secml.ml.kernel.c_kernel.CKernel

Polynomial kernel.

Given matrices X and Y, this is computed by:

K(x, y) = (coef0 + gamma * <x, y>)^degree

for each pair of rows in X and in Y.

Parameters

degreeint, optional

Kernel degree. Default 2.

gammafloat, optional

Free parameter to be used for balancing. Default 1.0.

coef0float, optional

Free parameter used for trading off the influence of higher-order versus lower-order terms in the kernel. Default 1.0.

batch_sizeint or None, optional

Size of the batch used for kernel computation. Default None.

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernel.c_kernel_poly import CKernelPoly

>>> print(CKernelPoly(degree=3, gamma=0.001, coef0=2).k(CArray([[1,2],[3,4]]), CArray([[10,20],[30,40]])))
CArray([[ 8.615125  9.393931]
 [ 9.393931 11.390625]])

>>> print(CKernelPoly().k(CArray([[1,2],[3,4]])))
CArray([[ 36. 144.]
 [144. 676.]])

Attributes

class_type‘poly’

Defines class type.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
deepcopy(self)	Returns a deep copy of current class.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class parameters.
get_subclasses()	Get all the subclasses of the calling class.
gradient(self, x, v)	Calculates kernel gradient wrt vector ‘v’.
k(self, x[, y])	Compute kernel between x and y.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
similarity(self, x[, y])	Computes kernel.
timed([msg])	Timer decorator.

property coef0

Coef0 parameter.

property degree

Degree parameter.

property gamma

Gamma parameter.

CKernelRBF

class secml.ml.kernel.c_kernel_rbf.CKernelRBF(gamma=1.0, batch_size=None)

Bases: secml.ml.kernel.c_kernel.CKernel

Radial basis function (RBF) kernel.

Given matrices X and Y, this is computed by:

K(x, y) = exp(-gamma ||x-y||^2)

for each pair of rows in X and in Y.

Parameters

gammafloat

Default is 1.0. Equals to -0.5 * sigma^-2 in the standard formulation of rbf kernel, it is a free parameter to be used for balancing.

batch_sizeint or None, optional

Size of the batch used for kernel computation. Default None.

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernel.c_kernel_rbf import CKernelRBF

>>> print(CKernelRBF(gamma=0.001).k(CArray([[1,2],[3,4]]), CArray([[10,20],[30,40]])))
CArray([[0.666977 0.101774]
 [0.737123 0.131994]])

>>> print(CKernelRBF().k(CArray([[1,2],[3,4]])))
CArray([[1.000000e+00 3.354626e-04]
 [3.354626e-04 1.000000e+00]])

Attributes

class_type‘rbf’

Defines class type.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
deepcopy(self)	Returns a deep copy of current class.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class parameters.
get_subclasses()	Get all the subclasses of the calling class.
gradient(self, x, v)	Calculates RBF kernel gradient wrt vector ‘v’.
k(self, x[, y])	Compute kernel between x and y.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
similarity(self, x[, y])	Computes kernel.
timed([msg])	Timer decorator.

property gamma

Gamma parameter.

gradient(self, x, v)

Calculates RBF kernel gradient wrt vector ‘v’.

The gradient of RBF kernel is given by:

dK(x,v)/dv = 2 * gamma * k(x,v) * (x - v)

Parameters

xCArray or array_like

First array of shape (n_x, n_features).

vCArray or array_like

Second array of shape (n_features, ) or (1, n_features).

Returns

kernel_gradientCArray

Kernel gradient of x with respect to vector v. Array of shape (n_x, n_features) if n_x > 1, else a flattened array of shape (n_features, ).

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernel.c_kernel_rbf import CKernelRBF

>>> array = CArray([[15,25],[45,55]])
>>> vector = CArray([2,5])
>>> print(CKernelRBF(gamma=1e-4).gradient(array, vector))
CArray([[0.002456 0.003779]
 [0.005567 0.006473]])

>>> print(CKernelRBF().gradient(vector, vector))
CArray([0. 0.])