secml.ml.classifiers

• secml.ml.classifiers.multiclass
  • CClassifierMulticlass
  • CClassifierMulticlassOVA
• secml.ml.classifiers.reject
  • CClassifierReject
  • CClassifierRejectThreshold
• secml.ml.classifiers.loss
  • CLoss
  • CLossCrossEntropy
  • CLossEpsilonInsensitive
  • CLossHinge
  • CLossLogistic
  • CLossSquare
  • CSoftmax
• secml.ml.classifiers.regularizer
  • CRegularizer
  • CRegularizerElasticNet
  • CRegularizerL1
  • CRegularizerL2

CClassifier

class secml.ml.classifiers.c_classifier.CClassifier(preprocess=None)

Bases: secml.core.c_creator.CCreator

Abstract class that defines basic methods for Classifiers.

A classifier assign a label (class) to new patterns using the informations learned from training set.

This interface implements a set of generic methods for training and classification that can be used for every algorithms. However, all of them can be reimplemented if specific routines are needed.

Parameters

preprocessCPreProcess or str or None, optional

Features preprocess to be applied to input data. Can be a CPreProcess subclass or a string with the type of the desired preprocessor. If None, input data is used as is.

Attributes

class_type

Defines class type.

classes

Return the list of classes on which training has been performed.

logger

Logger for current object.

n_classes

Number of classes of training dataset.

n_features

Number of features (before preprocessing).

verbose

Verbosity level of logger output.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
decision_function(self, x[, y])	Computes the decision function for each pattern in x.
deepcopy(self)	Returns a deep copy of current class.
estimate_parameters(self, dataset, …[, …])	Estimate parameter that give better result respect a chose metric.
fit(self, dataset[, n_jobs])	Trains the classifier.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class parameters.
get_subclasses()	Get all the subclasses of the calling class.
is_fitted(self)	Return True if the classifier is trained (fitted).
is_linear(self)	True for linear classifiers, False otherwise.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
predict(self, x[, return_decision_function])	Perform classification of each pattern in x.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
timed([msg])	Timer decorator.

property classes

Return the list of classes on which training has been performed.

decision_function(self, x, y=None)

Computes the decision function for each pattern in x.

If a preprocess has been specified, input is normalized before computing the decision function.

Note

The actual decision function should be implemented inside _decision_function method.

Parameters

xCArray

Array with new patterns to classify, 2-Dimensional of shape (n_patterns, n_features).

yint or None, optional

The label of the class wrt the function should be calculated. If None, return the output for all classes.

Returns

scoreCArray

Value of the decision function for each test pattern. Dense flat array of shape (n_samples,) if y is not None, otherwise a (n_samples, n_classes) array.

estimate_parameters(self, dataset, parameters, splitter, metric, pick='first', perf_evaluator='xval', n_jobs=1)

Estimate parameter that give better result respect a chose metric.

Parameters

datasetCDataset

Dataset to be used for evaluating parameters.

parametersdict

Dictionary with each entry as {parameter: list of values to test}. Example: {‘C’: [1, 10, 100], ‘gamma’: list(10.0 ** CArray.arange(-4, 4))}

splitterCDataSplitter or str

Object to use for splitting the dataset into train and validation. A splitter type can be passed as string, in this case all default parameters will be used. For data splitters, num_folds is set to 3 by default. See CDataSplitter docs for more informations.

metricCMetric or str

Object with the metric to use while evaluating the performance. A metric type can be passed as string, in this case all default parameters will be used. See CMetric docs for more informations.

pick{‘first’, ‘last’, ‘random’}, optional

Defines which of the best parameters set pick. Usually, ‘first’ correspond to the smallest parameters while ‘last’ correspond to the biggest. The order is consistent to the parameters dict passed as input.

perf_evaluatorCPerfEvaluator or str, optional

Performance Evaluator to use. Default ‘xval’.

n_jobsint, optional

Number of parallel workers to use for performance evaluation. Default 1. Cannot be higher than processor’s number of cores.

Returns

best_parametersdict

Dictionary of best parameters found through performance evaluation.

fit(self, dataset, n_jobs=1)

Trains the classifier.

If a preprocess has been specified, input is normalized before training.

For multiclass case see .CClassifierMulticlass.

Parameters

datasetCDataset

Training set. Must be a CDataset instance with patterns data and corresponding labels.

n_jobsint

Number of parallel workers to use for training the classifier. Default 1. Cannot be higher than processor’s number of cores.

Returns

trained_clsCClassifier

Instance of the classifier trained using input dataset.

is_fitted(self)

Return True if the classifier is trained (fitted).

Returns

bool

True or False depending on the result of the call to check_is_fitted.

is_linear(self)

True for linear classifiers, False otherwise.

property n_classes

Number of classes of training dataset.

property n_features

Number of features (before preprocessing).

predict(self, x, return_decision_function=False)

Perform classification of each pattern in x.

If preprocess has been specified, input is normalized before classification.

Parameters

xCArray

Array with new patterns to classify, 2-Dimensional of shape (n_patterns, n_features).

return_decision_functionbool, optional

Whether to return the decision_function value along with predictions. Default False.

Returns

labelsCArray

Flat dense array of shape (n_patterns,) with the label assigned to each test pattern. The classification label is the label of the class associated with the highest score.

scoresCArray, optional

Array of shape (n_patterns, n_classes) with classification score of each test pattern with respect to each training class. Will be returned only if return_decision_function is True.

CClassifierLinear

class secml.ml.classifiers.c_classifier_linear.CClassifierLinear(preprocess=None)

Bases: secml.ml.classifiers.c_classifier.CClassifier

Abstract class that defines basic methods for linear classifiers.

A linear classifier assign a label (class) to new patterns computing the inner product between the patterns and a vector of weights for each training set feature.

This interface implements a set of generic methods for training and classification that can be used for every linear model.

Parameters

preprocessCPreProcess or str or None, optional

Features preprocess to be applied to input data. Can be a CPreProcess subclass or a string with the type of the desired preprocessor. If None, input data is used as is.

Attributes

b

Bias calculated from training data.

class_type

Defines class type.

classes

Return the list of classes on which training has been performed.

logger

Logger for current object.

n_classes

Number of classes of training dataset.

n_features

Number of features (before preprocessing).

verbose

Verbosity level of logger output.

w

Vector with each feature’s weight (dense or sparse).

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
decision_function(self, x[, y])	Computes the decision function for each pattern in x.
deepcopy(self)	Returns a deep copy of current class.
estimate_parameters(self, dataset, …[, …])	Estimate parameter that give better result respect a chose metric.
fit(self, dataset[, n_jobs])	Trains the linear classifier.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class parameters.
get_subclasses()	Get all the subclasses of the calling class.
is_fitted(self)	Return True if the classifier is trained (fitted).
is_linear(self)	Return True as the classifier is linear.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
predict(self, x[, return_decision_function])	Perform classification of each pattern in x.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
timed([msg])	Timer decorator.

property b

Bias calculated from training data.

fit(self, dataset, n_jobs=1)

Trains the linear classifier.

If a preprocess has been specified, input is normalized before training.

Training on 2nd class is avoided to speed up classification.

Parameters

datasetCDataset

Binary (2-classes) training set. Must be a CDataset instance with patterns data and corresponding labels.

n_jobsint

Number of parallel workers to use for training the classifier. Default 1. Cannot be higher than processor’s number of cores.

Returns

trained_clsCClassifier

Instance of the classifier trained using input dataset.

is_linear(self)

Return True as the classifier is linear.

property w

Vector with each feature’s weight (dense or sparse).

CClassifierSkLearn

class secml.ml.classifiers.c_classifier_sklearn.CClassifierSkLearn(sklearn_model, preprocess=None)

Bases: secml.ml.classifiers.c_classifier.CClassifier

Generic wrapper for SkLearn classifiers.

Attributes

class_type

Defines class type.

classes

Return the list of classes on which training has been performed.

logger

Logger for current object.

n_classes

Number of classes of training dataset.

n_features

Number of features (before preprocessing).

verbose

Verbosity level of logger output.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
decision_function(self, x[, y])	Computes the decision function for each pattern in x.
deepcopy(self)	Returns a deep copy of current class.
estimate_parameters(self, dataset, …[, …])	Estimate parameter that give better result respect a chose metric.
fit(self, dataset[, n_jobs])	Trains the classifier.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class and SkLearn model parameters.
get_subclasses()	Get all the subclasses of the calling class.
is_fitted(self)	Return True if the classifier is trained (fitted).
is_linear(self)	True for linear classifiers, False otherwise.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
predict(self, x[, return_decision_function])	Perform classification of each pattern in x.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
timed([msg])	Timer decorator.

get_params(self)

Returns the dictionary of class and SkLearn model parameters.

A parameter is a PUBLIC or READ/WRITE attribute.

CClassifierDecisionTree

class secml.ml.classifiers.c_classifier_decision_tree.CClassifierDecisionTree(criterion='gini', splitter='best', max_depth=None, min_samples_split=2, preprocess=None)

Bases: secml.ml.classifiers.c_classifier_sklearn.CClassifierSkLearn

Decision Tree Classifier.

Parameters

preprocessCPreProcess or str or None, optional

Features preprocess to be applied to input data. Can be a CPreProcess subclass or a string with the type of the desired preprocessor. If None, input data is used as is.

Attributes

class_type‘dec-tree’

Defines class type.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
decision_function(self, x[, y])	Computes the decision function for each pattern in x.
deepcopy(self)	Returns a deep copy of current class.
estimate_parameters(self, dataset, …[, …])	Estimate parameter that give better result respect a chose metric.
fit(self, dataset[, n_jobs])	Trains the classifier.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class and SkLearn model parameters.
get_subclasses()	Get all the subclasses of the calling class.
is_fitted(self)	Return True if the classifier is trained (fitted).
is_linear(self)	True for linear classifiers, False otherwise.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
predict(self, x[, return_decision_function])	Perform classification of each pattern in x.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
timed([msg])	Timer decorator.

CClassifierKDE

class secml.ml.classifiers.c_classifier_kde.CClassifierKDE(kernel=None, preprocess=None)

Bases: secml.ml.classifiers.c_classifier.CClassifier, secml.ml.classifiers.gradients.mixin_classifier_gradient_kde.CClassifierGradientKDEMixin

Kernel Density Estimator

Parameters

kernelNone or CKernel subclass, optional

Instance of a CKernel subclass to be used for computing similarity between patterns. If None (default), a linear SVM will be created.

preprocessCPreProcess or str or None, optional

Features preprocess to be applied to input data. Can be a CPreProcess subclass or a string with the type of the desired preprocessor. If None, input data is used as is.

See also

CKernel

Pairwise kernels and metrics.

Attributes

class_type‘kde’

Defines class type.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
decision_function(self, x[, y])	Computes the decision function for each pattern in x.
deepcopy(self)	Returns a deep copy of current class.
estimate_parameters(self, dataset, …[, …])	Estimate parameter that give better result respect a chose metric.
fit(self, dataset[, n_jobs])	Trains the classifier.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class parameters.
get_subclasses()	Get all the subclasses of the calling class.
grad_f_params(self, x, y)	Derivative of the decision function w.r.t.
grad_f_x(self, x, y, \*\*kwargs)	Derivative of the classifier decision function w.r.t.
grad_loss_params(self, x, y[, loss])	Derivative of a given loss w.r.t.
grad_tr_params(self, x, y)	Derivative of the classifier training objective function w.r.t.
hessian_tr_params(self, x, y)	Hessian of the training objective w.r.t.
is_fitted(self)	Return True if the classifier is trained (fitted).
is_kernel_linear(self)	Return True if the kernel is None or linear.
is_linear(self)	Return True if the classifier is linear.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
predict(self, x[, return_decision_function])	Perform classification of each pattern in x.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
timed([msg])	Timer decorator.

is_kernel_linear(self)

Return True if the kernel is None or linear.

is_linear(self)

Return True if the classifier is linear.

property kernel

Kernel function (None if a linear classifier).

property training_samples

CClassifierKNN

class secml.ml.classifiers.c_classifier_knn.CClassifierKNN(n_neighbors=5, weights='uniform', algorithm='auto', leaf_size=30, p=2, metric='minkowski', metric_params=None, preprocess=None)

Bases: secml.ml.classifiers.c_classifier_sklearn.CClassifierSkLearn

K Neighbors Classifiers.

Parameters

preprocessCPreProcess or str or None, optional

Features preprocess to be applied to input data. Can be a CPreProcess subclass or a string with the type of the desired preprocessor. If None, input data is used as is.

Attributes

class_type‘knn’

Defines class type.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
decision_function(self, x[, y])	Computes the decision function for each pattern in x.
deepcopy(self)	Returns a deep copy of current class.
estimate_parameters(self, dataset, …[, …])	Estimate parameter that give better result respect a chose metric.
fit(self, dataset[, n_jobs])	Trains the classifier.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class and SkLearn model parameters.
get_subclasses()	Get all the subclasses of the calling class.
is_fitted(self)	Return True if the classifier is trained (fitted).
is_linear(self)	True for linear classifiers, False otherwise.
kneighbors(self, x[, num_samples])	Find the training samples nearest to x
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
predict(self, x[, return_decision_function])	Perform classification of each pattern in x.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
timed([msg])	Timer decorator.

kneighbors(self, x, num_samples=None)

Find the training samples nearest to x

Parameters

xCArray

The query point or points.

num_samples: int or None

Number of neighbors to get. if None, use n_neighbors

Returns

distCArray

Array representing the lengths to points

index_point: CArray

Indices of the nearest points in the training set

tr_dataset.X: CArray

Training samples

CClassifierLogistic

class secml.ml.classifiers.c_classifier_logistic.CClassifierLogistic(C=1.0, max_iter=100, random_seed=None, preprocess=None)

Bases: secml.ml.classifiers.c_classifier_linear.CClassifierLinear, secml.ml.classifiers.gradients.mixin_classifier_gradient_logistic.CClassifierGradientLogisticMixin

Logistic Regression (aka logit, MaxEnt) classifier.

Parameters

preprocessCPreProcess or str or None, optional

Features preprocess to be applied to input data. Can be a CPreProcess subclass or a string with the type of the desired preprocessor. If None, input data is used as is.

Attributes

C

Penalty parameter C of the error term.

b

Bias calculated from training data.

class_type

Defines class type.

classes

Return the list of classes on which training has been performed.

logger

Logger for current object.

max_iter

n_classes

Number of classes of training dataset.

n_features

Number of features (before preprocessing).

random_seed

verbose

Verbosity level of logger output.

w

Vector with each feature’s weight (dense or sparse).

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
decision_function(self, x[, y])	Computes the decision function for each pattern in x.
deepcopy(self)	Returns a deep copy of current class.
estimate_parameters(self, dataset, …[, …])	Estimate parameter that give better result respect a chose metric.
fit(self, dataset[, n_jobs])	Trains the linear classifier.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class parameters.
get_subclasses()	Get all the subclasses of the calling class.
grad_f_params(self, x[, y])	Derivative of the decision function w.r.t.
grad_f_x(self[, x, y])	Computes the gradient of the classifier’s output wrt input.
grad_loss_params(self, x, y[, loss])	Derivative of the classifier loss w.r.t.
grad_tr_params(self, x, y)	Derivative of the classifier training objective w.r.t. the classifier
hessian_tr_params(self, x, y)	Hessian of the training objective w.r.t.
is_fitted(self)	Return True if the classifier is trained (fitted).
is_linear(self)	Return True as the classifier is linear.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
predict(self, x[, return_decision_function])	Perform classification of each pattern in x.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
timed([msg])	Timer decorator.

property C

Penalty parameter C of the error term.

property max_iter

property random_seed

CClassifierMCSLinear

class secml.ml.classifiers.c_classifier_mcs_linear.CClassifierMCSLinear(classifier, num_classifiers=10, max_samples=1.0, max_features=1.0, random_state=None, preprocess=None)

Bases: secml.ml.classifiers.c_classifier_linear.CClassifierLinear, secml.ml.classifiers.gradients.mixin_classifier_gradient_linear.CClassifierGradientLinearMixin

MCS averaging a set of LINEAR classifiers.

Eventually, one yields a linear classifier itself, where w (b) is the average of the feature weights (bias) of the base classifiers.

Parameters

classifierCClassifierLinear

Instance of the linear classifier to be used in the MCS.

num_classifiersint, optional

Number of linear classifiers to fit, default 10.

max_samplesfloat, optional

Percentage of the samples to use for training, range [0, 1.0]. Default 1.0 (all the samples).

max_featuresfloat, optional

Percentage of the features to use for training, range [0, 1.0]. Default 1.0 (all the features.

random_stateint or None, optional

If int, random_state is the seed used by the random number generator. If None, no fixed seed will be set.

preprocessCPreProcess or str or None, optional

Features preprocess to be applied to input data. Can be a CPreProcess subclass or a string with the type of the desired preprocessor. If None, input data is used as is.

Attributes

class_type‘mcs-linear’

Defines class type.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
decision_function(self, x[, y])	Computes the decision function for each pattern in x.
deepcopy(self)	Returns a deep copy of current class.
estimate_parameters(self, dataset, …[, …])	Estimate parameter that give better result respect a chose metric.
fit(self, dataset[, n_jobs])	Trains the linear classifier.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class parameters.
get_subclasses()	Get all the subclasses of the calling class.
grad_f_params(self, x[, y])	Derivative of the decision function w.r.t.
grad_f_x(self[, x, y])	Computes the gradient of the classifier’s output wrt input.
grad_loss_params(self, x, y[, loss])	Derivative of the classifier loss w.r.t.
grad_tr_params(self, x, y)	Derivative of the classifier training objective w.r.t. the classifier
hessian_tr_params(self, x, y)	Hessian of the training objective w.r.t.
is_fitted(self)	Return True if the classifier is trained (fitted).
is_linear(self)	Return True as the classifier is linear.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
predict(self, x[, return_decision_function])	Perform classification of each pattern in x.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
timed([msg])	Timer decorator.

property classifier

Instance of the linear classifier used in the MCS.

property max_features

property max_samples

Percentage of the samples to use for training.

property n_classifiers

Number of linear classifiers to fit.

CClassifierNearestCentroid

class secml.ml.classifiers.c_classifier_nearest_centroid.CClassifierNearestCentroid(metric='euclidean', shrink_threshold=None, preprocess=None)

Bases: secml.ml.classifiers.c_classifier_sklearn.CClassifierSkLearn

CClassifierNearestCentroid.

Parameters

preprocessCPreProcess or str or None, optional

Features preprocess to be applied to input data. Can be a CPreProcess subclass or a string with the type of the desired preprocessor. If None, input data is used as is.

Attributes

class_type‘nrst-centroid’

Defines class type.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
decision_function(self, x[, y])	Computes the decision function for each pattern in x.
deepcopy(self)	Returns a deep copy of current class.
estimate_parameters(self, dataset, …[, …])	Estimate parameter that give better result respect a chose metric.
fit(self, dataset[, n_jobs])	Trains the classifier.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class and SkLearn model parameters.
get_subclasses()	Get all the subclasses of the calling class.
is_fitted(self)	Return True if the classifier is trained (fitted).
is_linear(self)	True for linear classifiers, False otherwise.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
predict(self, x[, return_decision_function])	Perform classification of each pattern in x.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
timed([msg])	Timer decorator.

property centroids

property metric

CClassifierRandomForest

class secml.ml.classifiers.c_classifier_random_forest.CClassifierRandomForest(n_estimators=10, criterion='gini', max_depth=None, min_samples_split=2, random_state=None, preprocess=None)

Bases: secml.ml.classifiers.c_classifier_sklearn.CClassifierSkLearn

Random Forest classifier.

Parameters

preprocessCPreProcess or str or None, optional

Features preprocess to be applied to input data. Can be a CPreProcess subclass or a string with the type of the desired preprocessor. If None, input data is used as is.

Attributes

class_type‘random-forest’

Defines class type.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
decision_function(self, x[, y])	Computes the decision function for each pattern in x.
deepcopy(self)	Returns a deep copy of current class.
estimate_parameters(self, dataset, …[, …])	Estimate parameter that give better result respect a chose metric.
fit(self, dataset[, n_jobs])	Trains the classifier.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class and SkLearn model parameters.
get_subclasses()	Get all the subclasses of the calling class.
is_fitted(self)	Return True if the classifier is trained (fitted).
is_linear(self)	True for linear classifiers, False otherwise.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
predict(self, x[, return_decision_function])	Perform classification of each pattern in x.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
timed([msg])	Timer decorator.

CClassifierRidge

class secml.ml.classifiers.c_classifier_ridge.CClassifierRidge(alpha=1.0, kernel=None, max_iter=100000.0, class_weight=None, tol=0.0001, fit_intercept=True, preprocess=None)

Bases: secml.ml.classifiers.c_classifier_linear.CClassifierLinear, secml.ml.classifiers.gradients.mixin_classifier_gradient_ridge.CClassifierGradientRidgeMixin

Ridge Classifier.

Parameters

preprocessCPreProcess or str or None, optional

Features preprocess to be applied to input data. Can be a CPreProcess subclass or a string with the type of the desired preprocessor. If None, input data is used as is.

Attributes

class_type‘ridge’

Defines class type.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
decision_function(self, x[, y])	Computes the decision function for each pattern in x.
deepcopy(self)	Returns a deep copy of current class.
estimate_parameters(self, dataset, …[, …])	Estimate parameter that give better result respect a chose metric.
fit(self, dataset[, n_jobs])	Trains the linear classifier.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class parameters.
get_subclasses()	Get all the subclasses of the calling class.
grad_f_params(self, x[, y])	Derivative of the decision function w.r.t.
grad_f_x(self[, x, y])	Computes the gradient of the classifier’s output wrt input.
grad_loss_params(self, x, y[, loss])	Derivative of the classifier loss w.r.t.
grad_tr_params(self, x, y)	Derivative of the classifier training objective w.r.t. the classifier
hessian_tr_params(self, x[, y])	Hessian of the training objective w.r.t.
is_fitted(self)	Return True if the classifier is trained (fitted).
is_kernel_linear(self)	Return True if the kernel is None or linear.
is_linear(self)	Return True if the classifier is linear.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
predict(self, x[, return_decision_function])	Perform classification of each pattern in x.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
timed([msg])	Timer decorator.

property C

Constant that multiplies the regularization term.

Equal to 1 / alpha.

property alpha

Returns the Constant that multiplies the regularization term.

property class_weight

Weight of each training class.

is_kernel_linear(self)

Return True if the kernel is None or linear.

is_linear(self)

Return True if the classifier is linear.

property kernel

Kernel function.

property n_tr_samples

Returns the number of training samples.

CClassifierSGD

class secml.ml.classifiers.c_classifier_sgd.CClassifierSGD(loss, regularizer, kernel=None, alpha=0.01, fit_intercept=True, max_iter=1000, tol=-inf, shuffle=True, learning_rate='optimal', eta0=10.0, power_t=0.5, class_weight=None, warm_start=False, average=False, random_state=None, preprocess=None)

Bases: secml.ml.classifiers.c_classifier_linear.CClassifierLinear, secml.ml.classifiers.gradients.mixin_classifier_gradient_sgd.CClassifierGradientSGDMixin

Stochastic Gradient Descent Classifier.

Parameters

preprocessCPreProcess or str or None, optional

Features preprocess to be applied to input data. Can be a CPreProcess subclass or a string with the type of the desired preprocessor. If None, input data is used as is.

Attributes

class_type‘sgd’

Defines class type.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
decision_function(self, x[, y])	Computes the decision function for each pattern in x.
deepcopy(self)	Returns a deep copy of current class.
estimate_parameters(self, dataset, …[, …])	Estimate parameter that give better result respect a chose metric.
fit(self, dataset[, n_jobs])	Trains the linear classifier.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class parameters.
get_subclasses()	Get all the subclasses of the calling class.
grad_f_params(self, x[, y])	Derivative of the decision function w.r.t.
grad_f_x(self[, x, y])	Computes the gradient of the classifier’s output wrt input.
grad_loss_params(self, x, y[, loss])	Derivative of the classifier loss w.r.t.
grad_tr_params(self, x, y)	Derivative of the classifier training objective function w.r.t.
hessian_tr_params(self, x, y)	Hessian of the training objective w.r.t.
is_fitted(self)	Return True if the classifier is trained (fitted).
is_kernel_linear(self)	Return True if the kernel is None or linear.
is_linear(self)	Return True if the classifier is linear.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
predict(self, x[, return_decision_function])	Perform classification of each pattern in x.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
timed([msg])	Timer decorator.

property C

Constant that multiplies the regularization term.

Equal to 1 / alpha.

property alpha

Returns the Constant that multiplies the regularization term.

property average

When set to True, computes the averaged SGD weights. If set to an int greater than 1, averaging will begin once the total number of samples seen reaches average. So average=10 will begin averaging after seeing 10 samples.

property class_weight

Weight of each training class.

property eta0

The initial learning rate for the invscaling learning rate. Default is 10.0 (corresponding to sqrt(1.0/sqrt(alpha)), with alpha=0.0001).

is_kernel_linear(self)

Return True if the kernel is None or linear.

is_linear(self)

Return True if the classifier is linear.

property kernel

Kernel function.

property loss

Returns the loss function used by classifier.

property n_tr_samples

Returns the number of training samples.

property power_t

The exponent for inverse scaling learning rate.

property regularizer

Returns the regularizer function used by classifier.

CClassifierSVM

class secml.ml.classifiers.c_classifier_svm.CClassifierSVM(kernel=None, C=1.0, class_weight=None, preprocess=None, grad_sampling=1.0, store_dual_vars=None)

Bases: secml.ml.classifiers.c_classifier_linear.CClassifierLinear, secml.ml.classifiers.gradients.mixin_classifier_gradient_svm.CClassifierGradientSVMMixin

Support Vector Machine (SVM) classifier.

Parameters

kernelNone or CKernel subclass, optional

Instance of a CKernel subclass to be used for computing similarity between patterns. If None (default), a linear SVM will be created.

Cfloat, optional

Penalty parameter C of the error term. Default 1.0.

class_weight{dict, ‘balanced’, None}, optional

Set the parameter C of class i to class_weight[i] * C. If not given (default), all classes are supposed to have weight one. The ‘balanced’ mode uses the values of labels to automatically adjust weights inversely proportional to class frequencies as n_samples / (n_classes * np.bincount(y)).

preprocessCPreProcess or str or None, optional

Features preprocess to be applied to input data. Can be a CPreProcess subclass or a string with the type of the desired preprocessor. If None, input data is used as is.

grad_samplingfloat

Percentage in (0.0, 1.0] of the alpha weights to be considered when computing the classifier gradient.

See also

CKernel

Pairwise kernels and metrics.

CClassifierLinear

Common interface for linear classifiers.

Notes

Current implementation relies on sklearn.svm.SVC for the training step.

Attributes

class_type‘svm’

Defines class type.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
decision_function(self, x[, y])	Computes the decision function for each pattern in x.
deepcopy(self)	Returns a deep copy of current class.
estimate_parameters(self, dataset, …[, …])	Estimate parameter that give better result respect a chose metric.
fit(self, dataset[, n_jobs])	Fit the SVM classifier.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class parameters.
get_subclasses()	Get all the subclasses of the calling class.
grad_f_params(self, x[, y])	Derivative of the decision function w.r.t.
grad_f_x(self[, x, y])	Computes the gradient of the classifier’s output wrt input.
grad_loss_params(self, x, y[, loss])	Derivative of the loss w.r.t.
grad_tr_params(self, x, y)	Derivative of the classifier training objective w.r.t.
hessian_tr_params(self[, x, y])	Hessian of the training objective w.r.t.
is_fitted(self)	Return True if the classifier is trained (fitted).
is_kernel_linear(self)	Return True if the kernel is None or linear.
is_linear(self)	Return True if the classifier is linear.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads class from pickle object.
predict(self, x[, return_decision_function])	Perform classification of each pattern in x.
save(self, path)	Save class object using pickle.
set(self, param_name, param_value[, copy])	Set a parameter that has a specific name to a specific value.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
sv_margin(self[, tol])	Margin Support Vectors.
sv_margin_idx(self[, tol])	Indices of Margin Support Vectors.
sv_margin_y(self[, tol])	Margin Support Vectors class (-1/+1).
timed([msg])	Timer decorator.

property C

Penalty parameter C of the error term.

property alpha

Signed coefficients of the SVs in the decision function.

property class_weight

Weight of each training class.

fit(self, dataset, n_jobs=1)

Fit the SVM classifier.

We use sklearn.svm.SVC for weights and Support Vectors computation. The routine will set alpha, sv, sv_idx and b parameters. For linear SVM (i.e. if kernel is None) we also store the ‘w’ flat vector with each feature’s weight.

If a preprocess has been specified, input is normalized before computing the decision function.

Parameters

datasetCDataset

Binary (2-classes) Training set. Must be a CDataset instance with patterns data and corresponding labels.

n_jobsint, optional

Number of parallel workers to use for training the classifier. Default 1. Cannot be higher than processor’s number of cores.

Returns

trained_clsCClassifierSVM

Instance of the SVM classifier trained using input dataset.

property grad_sampling

Percentage of samples for approximate gradient.

is_kernel_linear(self)

Return True if the kernel is None or linear.

is_linear(self)

Return True if the classifier is linear.

property kernel

Kernel function (None if a linear classifier).

property n_sv

Return the number of support vectors.

In the 1st and in the 2nd column is stored the number of SVs for the negative and positive class respectively.

property store_dual_vars

Controls the store of dual space variables (SVs and alphas).

By default is None and dual variables are stored only if kernel is not None. If set to True, dual variables are stored even if kernel is None (linear SVM). If kernel is not None, cannot be set to False.

property sv

Support Vectors.

property sv_idx

Indices of Support Vectors within the training dataset.

sv_margin(self, tol=1e-06)

Margin Support Vectors.

Parameters

tolfloat

Alpha value threshold for considering a Support Vector on the margin.

Returns

CArray or None

Margin support vector, 2D CArray. If no margin support vector are found, return None.

indicesCArray or None

Flat array with the indices of the margin support vectors. If no margin support vector are found, return None.

sv_margin_idx(self, tol=1e-06)

Indices of Margin Support Vectors.

Parameters

tolfloat

Alpha value threshold for considering a Support Vector on the margin.

Returns

indicesCArray

Flat array with the indices of the Margin Support Vectors.

sv_margin_y(self, tol=1e-06)

Margin Support Vectors class (-1/+1).

Parameters

tolfloat

Alpha value threshold for considering a Support Vector on the margin.

Returns

CArray

Flat CArray with the class (-1/+1) of the Margin Support Vectors.

clf_utils

secml.ml.classifiers.clf_utils.check_binary_labels(labels)

Check if input labels are binary {0, +1}.

Parameters

labelsCArray or int

Binary labels to be converted. As of PRALib convention, binary labels are {0, +1}.

Raises

ValueError

If input labels are not binary.

secml.ml.classifiers.clf_utils.convert_binary_labels(labels)

Convert input binary labels to {-1, +1}.

Parameters

labelsCArray or int

Binary labels in {0, +1} to be converted to {-1, +1}.

Returns

converted_labelsCArray or int

Binary labels converted to {-1, +1}.

Examples

>>> from secml.ml.classifiers.clf_utils import convert_binary_labels
>>> from secml.array import CArray

>>> print(convert_binary_labels(0))
-1

>>> print(convert_binary_labels(CArray([0,1,1,1,0,0])))
CArray([-1  1  1  1 -1 -1])