secml.ml.kernels

CKernel

class secml.ml.kernels.c_kernel.CKernel[source]

Bases: secml.ml.c_module.CModule

Abstract class that defines basic methods for kernels.

A kernel is a pairwise metric that compute the distance between sets of patterns.

Kernels can be considered similarity measures, i.e. s(a, b) > s(a, c) if objects a and b are considered “more similar” than objects a and c. A kernel must be positive semi-definite (PSD), even though non-PSD kernels can also be used to train classifiers (e.g., SVMs, but losing convexity).

Attributes
class_type

Defines class type.

logger

Logger for current object.

preprocess

Inner preprocessor (if any).

rv

Reference vectors with respect to compute the kernel.

verbose

Verbosity level of logger output.

Methods

backward(self[, w])

Returns the preprocessor gradient wrt data.

copy(self)

Returns a shallow copy of current class.

create([class_item])

This method creates an instance of a class with given type.

deepcopy(self)

Returns a deep copy of current class.

forward(self, x[, caching])

Compute kernel between x and cached rv.

get_class_from_type(class_type)

Return the class associated with input type.

get_params(self)

Returns the dictionary of class parameters.

get_state(self)

Returns the object state dictionary.

get_subclasses()

Get all the subclasses of the calling class.

gradient(self, x[, w])

Compute gradient at x by doing a backward pass.

k(self, x[, rv])

Compute kernel between x and rv.

list_class_types()

This method lists all types of available subclasses of calling one.

load(path)

Loads object from file.

load_state(self, path)

Sets the object state from file.

save(self, path)

Save class object to file.

save_state(self, path)

Store the object state to file.

set(self, param_name, param_value[, copy])

Set a parameter of the class.

set_params(self, params_dict[, copy])

Set all parameters passed as a dictionary {key: value}.

set_state(self, state_dict[, copy])

Sets the object state using input dictionary.

similarity(self, x[, rv])

Deprecated since version 0.12.

timed([msg])

Timer decorator.

forward(self, x, caching=True)[source]

Compute kernel between x and cached rv. If rv is None, it is set to x and the kernel is computed between x and itself.

Parameters
xCArray

Array of shape (n_x, n_features).

caching: bool

True if preprocessed input should be cached for backward pass.

Returns
kernelCArray

Kernel between x and rv. Array of shape (n_x, n_rv).

k(self, x, rv=None)[source]

Compute kernel between x and rv.

Parameters
xCArray

First array of shape (n_x, n_features).

rvCArray, optional

Second array of shape (n_rv, n_features). If not specified, it is set to x and the kernel k(x,x) is computed.

Returns
kernelCArray or scalar

Kernel between x and rv. Array of shape (n_x, n_rv) or scalar if both x and y are vector-like.

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernels import CKernelRBF
>>> array1 = CArray([[15,25],[45,55]])
>>> array2 = CArray([[10,20],[40,50]])
>>> print(CKernelRBF().k(array1, array2))
CArray([[1.92875e-22 0.00000e+00]
 [0.00000e+00 1.92875e-22]])
>>> print(CKernelRBF().k(array1))
CArray([[1. 0.]
 [0. 1.]])
>>> vector = CArray([15,25])
>>> print(CKernelRBF().k(vector, array1))
CArray([[1. 0.]])
>>> print(CKernelRBF().k(array1, vector))
CArray([[1.]
 [0.]])
>>> print(CKernelRBF().k(vector, vector))
CArray([[1.]])
property rv

Reference vectors with respect to compute the kernel.

similarity(self, x, rv=None)[source]

Deprecated since version 0.12: use .k instead.

Computes kernel. Wrapper of ‘k’ function.

See also

CKernel.k

Main computation interface for kernels.

CKernelChebyshevDistance

class secml.ml.kernels.c_kernel_chebyshev_distance.CKernelChebyshevDistance[source]

Bases: secml.ml.kernels.c_kernel.CKernel

Chebyshev distance kernel.

Given matrices X and RV, this is computed as:

K(x, rv) = max(|x - rv|)

for each pair of rows in X and in RV.

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernels import CKernelChebyshevDistance
>>> x = CArray([[1,2],[3,4]])
>>> v = CArray([[5,6],[7,8]])
>>> print(CKernelChebyshevDistance().k(x,v))
CArray([[-4. -6.]
 [-2. -4.]])
>>> print(CKernelChebyshevDistance().k(x))
CArray([[-0. -2.]
 [-2. -0.]])
Attributes
class_type‘chebyshev-dist’

Defines class type.

Methods

backward(self[, w])

Returns the preprocessor gradient wrt data.

copy(self)

Returns a shallow copy of current class.

create([class_item])

This method creates an instance of a class with given type.

deepcopy(self)

Returns a deep copy of current class.

forward(self, x[, caching])

Compute kernel between x and cached rv.

get_class_from_type(class_type)

Return the class associated with input type.

get_params(self)

Returns the dictionary of class parameters.

get_state(self)

Returns the object state dictionary.

get_subclasses()

Get all the subclasses of the calling class.

gradient(self, x[, w])

Compute gradient at x by doing a backward pass.

k(self, x[, rv])

Compute kernel between x and rv.

list_class_types()

This method lists all types of available subclasses of calling one.

load(path)

Loads object from file.

load_state(self, path)

Sets the object state from file.

save(self, path)

Save class object to file.

save_state(self, path)

Store the object state to file.

set(self, param_name, param_value[, copy])

Set a parameter of the class.

set_params(self, params_dict[, copy])

Set all parameters passed as a dictionary {key: value}.

set_state(self, state_dict[, copy])

Sets the object state using input dictionary.

similarity(self, x[, rv])

Deprecated since version 0.12.

timed([msg])

Timer decorator.

CKernelEuclidean

class secml.ml.kernels.c_kernel_euclidean.CKernelEuclidean(squared=False)[source]

Bases: secml.ml.kernels.c_kernel.CKernel

Euclidean distance kernel.

Given matrices X and RV, this is computed as the negative Euclidean dist.:

K(x, rv) = -sqrt(dot(x, x) - 2 * dot(x, rv) + dot(rv, rv))

for each pair of rows in X and in RV. If parameter squared is True (default False), sqrt() operation is avoided.

Parameters
squaredbool, optional

If True, return squared Euclidean distances. Default False.

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernels.c_kernel_euclidean import CKernelEuclidean
>>> print(CKernelEuclidean().k(CArray([[1,2],[3,4]]), CArray([[10,20],[30,40]])))
CArray([[-20.124612 -47.801674]
 [-17.464249 -45.      ]])
>>> print(CKernelEuclidean().k(CArray([[1,2],[3,4]])))
CArray([[0.       -2.828427]
 [-2.828427 0.      ]])
Attributes
class_type‘euclidean’

Defines class type.

Methods

backward(self[, w])

Returns the preprocessor gradient wrt data.

copy(self)

Returns a shallow copy of current class.

create([class_item])

This method creates an instance of a class with given type.

deepcopy(self)

Returns a deep copy of current class.

forward(self, x[, caching])

Compute kernel between x and cached rv.

get_class_from_type(class_type)

Return the class associated with input type.

get_params(self)

Returns the dictionary of class parameters.

get_state(self)

Returns the object state dictionary.

get_subclasses()

Get all the subclasses of the calling class.

gradient(self, x[, w])

Compute gradient at x by doing a forward and a backward pass.

k(self, x[, rv])

Compute kernel between x and rv.

list_class_types()

This method lists all types of available subclasses of calling one.

load(path)

Loads object from file.

load_state(self, path)

Sets the object state from file.

save(self, path)

Save class object to file.

save_state(self, path)

Store the object state to file.

set(self, param_name, param_value[, copy])

Set a parameter of the class.

set_params(self, params_dict[, copy])

Set all parameters passed as a dictionary {key: value}.

set_state(self, state_dict[, copy])

Sets the object state using input dictionary.

similarity(self, x[, rv])

Deprecated since version 0.12.

timed([msg])

Timer decorator.

gradient(self, x, w=None)[source]

Compute gradient at x by doing a forward and a backward pass.

The gradient is pre-multiplied by w.

property rv_norm_squared

Pre-computed dot-products of vectors in rv (e.g., (rv**2).sum(axis=1)).

property squared

If True, squared Euclidean distances are computed.

property x_norm_squared

Pre-computed dot-products of vectors in x (e.g., (x**2).sum(axis=1)).

CKernelHistIntersect

class secml.ml.kernels.c_kernel_histintersect.CKernelHistIntersect[source]

Bases: secml.ml.kernels.c_kernel.CKernel

Histogram Intersection Kernel.

Given matrices X and RV, this is computed by:

K(x, rv) = sum_i ( min(x[i], rv[i]) )

for each pair of rows in X and in RV.

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernels.c_kernel_histintersect import CKernelHistIntersect
>>> print(CKernelHistIntersect().k(CArray([[1,2],[3,4]]), CArray([[10,20],[30,40]])))
CArray([[3. 3.]
 [7. 7.]])
>>> print(CKernelHistIntersect().k(CArray([[1,2],[3,4]])))
CArray([[3. 3.]
 [3. 7.]])
Attributes
class_type‘hist-intersect’

Defines class type.

Methods

backward(self[, w])

Returns the preprocessor gradient wrt data.

copy(self)

Returns a shallow copy of current class.

create([class_item])

This method creates an instance of a class with given type.

deepcopy(self)

Returns a deep copy of current class.

forward(self, x[, caching])

Compute kernel between x and cached rv.

get_class_from_type(class_type)

Return the class associated with input type.

get_params(self)

Returns the dictionary of class parameters.

get_state(self)

Returns the object state dictionary.

get_subclasses()

Get all the subclasses of the calling class.

gradient(self, x[, w])

Compute gradient at x by doing a backward pass.

k(self, x[, rv])

Compute kernel between x and rv.

list_class_types()

This method lists all types of available subclasses of calling one.

load(path)

Loads object from file.

load_state(self, path)

Sets the object state from file.

save(self, path)

Save class object to file.

save_state(self, path)

Store the object state to file.

set(self, param_name, param_value[, copy])

Set a parameter of the class.

set_params(self, params_dict[, copy])

Set all parameters passed as a dictionary {key: value}.

set_state(self, state_dict[, copy])

Sets the object state using input dictionary.

similarity(self, x[, rv])

Deprecated since version 0.12.

timed([msg])

Timer decorator.

CKernelLaplacian

class secml.ml.kernels.c_kernel_laplacian.CKernelLaplacian(gamma=1.0)[source]

Bases: secml.ml.kernels.c_kernel.CKernel

Laplacian Kernel.

Given matrices X and RV, this is computed by:

K(x, rv) = exp(-gamma |x-rv|)

for each pair of rows in X and in RV.

Parameters
gammafloat

Default is 1.0.

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernels.c_kernel_laplacian import CKernelLaplacian
>>> print(CKernelLaplacian(gamma=0.01).k(CArray([[1,2],[3,4]]), CArray([[10,0],[0,40]])))
CArray([[0.895834 0.677057]
 [0.895834 0.677057]])
>>> print(CKernelLaplacian().k(CArray([[1,2],[3,4]])))
CArray([[1.       0.018316]
 [0.018316 1.      ]])
Attributes
class_type‘laplacian’

Defines class type.

Methods

backward(self[, w])

Returns the preprocessor gradient wrt data.

copy(self)

Returns a shallow copy of current class.

create([class_item])

This method creates an instance of a class with given type.

deepcopy(self)

Returns a deep copy of current class.

forward(self, x[, caching])

Compute kernel between x and cached rv.

get_class_from_type(class_type)

Return the class associated with input type.

get_params(self)

Returns the dictionary of class parameters.

get_state(self)

Returns the object state dictionary.

get_subclasses()

Get all the subclasses of the calling class.

gradient(self, x[, w])

Compute gradient at x by doing a forward and a backward pass.

k(self, x[, rv])

Compute kernel between x and rv.

list_class_types()

This method lists all types of available subclasses of calling one.

load(path)

Loads object from file.

load_state(self, path)

Sets the object state from file.

save(self, path)

Save class object to file.

save_state(self, path)

Store the object state to file.

set(self, param_name, param_value[, copy])

Set a parameter of the class.

set_params(self, params_dict[, copy])

Set all parameters passed as a dictionary {key: value}.

set_state(self, state_dict[, copy])

Sets the object state using input dictionary.

similarity(self, x[, rv])

Deprecated since version 0.12.

timed([msg])

Timer decorator.

property gamma

Gamma parameter.

gradient(self, x, w=None)[source]

Compute gradient at x by doing a forward and a backward pass.

The gradient is pre-multiplied by w.

CKernelLinear

class secml.ml.kernels.c_kernel_linear.CKernelLinear[source]

Bases: secml.ml.kernels.c_kernel.CKernel

Linear kernel.

Given matrices X and RV, this is computed by:

K(x, rv) = x * rv^T

for each pair of rows in X and in RV.

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernels.c_kernel_linear import CKernelLinear
>>> print(CKernelLinear().k(CArray([[1,2],[3,4]]), CArray([[10,20],[30,40]])))
CArray([[ 50. 110.]
 [110. 250.]])
>>> print(CKernelLinear().k(CArray([[1,2],[3,4]])))
CArray([[ 5. 11.]
 [11. 25.]])
Attributes
class_type‘linear’

Defines class type.

Methods

backward(self[, w])

Returns the preprocessor gradient wrt data.

copy(self)

Returns a shallow copy of current class.

create([class_item])

This method creates an instance of a class with given type.

deepcopy(self)

Returns a deep copy of current class.

forward(self, x[, caching])

Compute kernel between x and cached rv.

get_class_from_type(class_type)

Return the class associated with input type.

get_params(self)

Returns the dictionary of class parameters.

get_state(self)

Returns the object state dictionary.

get_subclasses()

Get all the subclasses of the calling class.

gradient(self, x[, w])

Compute gradient at x by doing a backward pass.

k(self, x[, rv])

Compute kernel between x and rv.

list_class_types()

This method lists all types of available subclasses of calling one.

load(path)

Loads object from file.

load_state(self, path)

Sets the object state from file.

save(self, path)

Save class object to file.

save_state(self, path)

Store the object state to file.

set(self, param_name, param_value[, copy])

Set a parameter of the class.

set_params(self, params_dict[, copy])

Set all parameters passed as a dictionary {key: value}.

set_state(self, state_dict[, copy])

Sets the object state using input dictionary.

similarity(self, x[, rv])

Deprecated since version 0.12.

timed([msg])

Timer decorator.

CKernelPoly

class secml.ml.kernels.c_kernel_poly.CKernelPoly(degree=2, gamma=1.0, coef0=1.0)[source]

Bases: secml.ml.kernels.c_kernel.CKernel

Polynomial kernel.

Given matrices X and RV, this is computed by:

K(x, rv) = (coef0 + gamma * <x, rv>)^degree

for each pair of rows in X and in RV.

Parameters
degreeint, optional

Kernel degree. Default 2.

gammafloat, optional

Free parameter to be used for balancing. Default 1.0.

coef0float, optional

Free parameter used for trading off the influence of higher-order versus lower-order terms in the kernel. Default 1.0.

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernels.c_kernel_poly import CKernelPoly
>>> print(CKernelPoly(degree=3, gamma=0.001, coef0=2).k(CArray([[1,2],[3,4]]), CArray([[10,20],[30,40]])))
CArray([[ 8.615125  9.393931]
 [ 9.393931 11.390625]])
>>> print(CKernelPoly().k(CArray([[1,2],[3,4]])))
CArray([[ 36. 144.]
 [144. 676.]])
Attributes
class_type‘poly’

Defines class type.

Methods

backward(self[, w])

Returns the preprocessor gradient wrt data.

copy(self)

Returns a shallow copy of current class.

create([class_item])

This method creates an instance of a class with given type.

deepcopy(self)

Returns a deep copy of current class.

forward(self, x[, caching])

Compute kernel between x and cached rv.

get_class_from_type(class_type)

Return the class associated with input type.

get_params(self)

Returns the dictionary of class parameters.

get_state(self)

Returns the object state dictionary.

get_subclasses()

Get all the subclasses of the calling class.

gradient(self, x[, w])

Compute gradient at x by doing a backward pass.

k(self, x[, rv])

Compute kernel between x and rv.

list_class_types()

This method lists all types of available subclasses of calling one.

load(path)

Loads object from file.

load_state(self, path)

Sets the object state from file.

save(self, path)

Save class object to file.

save_state(self, path)

Store the object state to file.

set(self, param_name, param_value[, copy])

Set a parameter of the class.

set_params(self, params_dict[, copy])

Set all parameters passed as a dictionary {key: value}.

set_state(self, state_dict[, copy])

Sets the object state using input dictionary.

similarity(self, x[, rv])

Deprecated since version 0.12.

timed([msg])

Timer decorator.

property coef0

Coef0 parameter.

property degree

Degree parameter.

property gamma

Gamma parameter.

CKernelRBF

class secml.ml.kernels.c_kernel_rbf.CKernelRBF(gamma=1.0)[source]

Bases: secml.ml.kernels.c_kernel.CKernel

Radial basis function (RBF) kernel.

Given matrices X and RV, this is computed by:

K(x, rv) = exp(-gamma ||x-rv||^2)

for each pair of rows in X and in RV.

Parameters
gammafloat

Default is 1.0. Equals to -0.5 * sigma^-2 in the standard formulation of rbf kernel, it is a free parameter to be used for balancing.

Examples

>>> from secml.array import CArray
>>> from secml.ml.kernels.c_kernel_rbf import CKernelRBF
>>> print(CKernelRBF(gamma=0.001).k(CArray([[1,2],[3,4]]), CArray([[10,20],[30,40]])))
CArray([[0.666977 0.101774]
 [0.737123 0.131994]])
>>> print(CKernelRBF().k(CArray([[1,2],[3,4]])))
CArray([[1.000000e+00 3.354626e-04]
 [3.354626e-04 1.000000e+00]])
Attributes
class_type‘rbf’

Defines class type.

Methods

backward(self[, w])

Returns the preprocessor gradient wrt data.

copy(self)

Returns a shallow copy of current class.

create([class_item])

This method creates an instance of a class with given type.

deepcopy(self)

Returns a deep copy of current class.

forward(self, x[, caching])

Compute kernel between x and cached rv.

get_class_from_type(class_type)

Return the class associated with input type.

get_params(self)

Returns the dictionary of class parameters.

get_state(self)

Returns the object state dictionary.

get_subclasses()

Get all the subclasses of the calling class.

gradient(self, x[, w])

Compute gradient at x by doing a forward and a backward pass.

k(self, x[, rv])

Compute kernel between x and rv.

list_class_types()

This method lists all types of available subclasses of calling one.

load(path)

Loads object from file.

load_state(self, path)

Sets the object state from file.

save(self, path)

Save class object to file.

save_state(self, path)

Store the object state to file.

set(self, param_name, param_value[, copy])

Set a parameter of the class.

set_params(self, params_dict[, copy])

Set all parameters passed as a dictionary {key: value}.

set_state(self, state_dict[, copy])

Sets the object state using input dictionary.

similarity(self, x[, rv])

Deprecated since version 0.12.

timed([msg])

Timer decorator.

property gamma

Gamma parameter.

gradient(self, x, w=None)[source]

Compute gradient at x by doing a forward and a backward pass.

The gradient is pre-multiplied by w.