secml.adv.seceval

CSecEval

class secml.adv.seceval.c_sec_eval.CSecEval(attack, param_name, param_values, save_adv_ds=False)[source]

Bases: secml.core.c_creator.CCreator

This class repeat the security evaluation (where security is measured with a given metric) while the power of the attacker increase.

Parameters
attackCAttack

Class that implements an attack (e.g evasion or poisoning)

param_namestr

Name of the parameter that represents the increasingly attacker power.

param_valuesCArray

Array that contains values that param_name will assumes during the attack (this define how the attacker power increases). If the first value is not zero, zero will be added as first value

save_adv_dsbool, optional

If True, the samples at each parameter will be stored. Default False.

See also

CAttack

class that implements the attack.

Attributes
attack

Return the attack object that is used from CSecEval to perform the attack.

class_type

Defines class type.

logger

Logger for current object.

save_adv_ds

Returns

sec_eval_data

Get a sec eval data objects.

verbose

Verbosity level of logger output.

Methods

copy(self)

Returns a shallow copy of current class.

create([class_item])

This method creates an instance of a class with given type.

deepcopy(self)

Returns a deep copy of current class.

get_class_from_type(class_type)

Return the class associated with input type.

get_params(self)

Returns the dictionary of class hyperparameters.

get_state(self, **kwargs)

Returns the object state dictionary.

get_subclasses()

Get all the subclasses of the calling class.

list_class_types()

This method lists all types of available subclasses of calling one.

load(path)

Loads object from file.

load_data(self, path)

Restore Sec Eval data from file.

load_state(self, path)

Sets the object state from file.

run_sec_eval(self, dataset, **kwargs)

Performs attack while the power of the attacker (named param_name) increase.

save(self, path)

Save class object to file.

save_data(self, path)

Store Sec Eval data to file.

save_state(self, path, **kwargs)

Store the object state to file.

set(self, param_name, param_value[, copy])

Set a parameter of the class.

set_params(self, params_dict[, copy])

Set all parameters passed as a dictionary {key: value}.

set_state(self, state_dict[, copy])

Sets the object state using input dictionary.

timed([msg])

Timer decorator.

property attack

Return the attack object that is used from CSecEval to perform the attack.

load_data(self, path)[source]

Restore Sec Eval data from file.

run_sec_eval(self, dataset, **kwargs)[source]

Performs attack while the power of the attacker (named param_name) increase.

Parameters
datasetCDataset

Dataset that contain samples that will be manipulated from the attacker while his attack power increase

kwargs

Additional keyword arguments for the CAttack.run method.

property save_adv_ds
Returns
True/False: whether to store or not the manipulated attack sample dataset
save_data(self, path)[source]

Store Sec Eval data to file.

property sec_eval_data

Get a sec eval data objects. It contains the Security Evaluation Results.

Returns
sec_eval_data: CSecEvalData object

contains classifier security evaluation results

CSecEvalData

class secml.adv.seceval.c_sec_eval_data.CSecEvalData[source]

Bases: secml.core.c_creator.CCreator

This class is a container for data computed during Classifier Security Evaluation.

Attributes
class_type‘standard’

Defines class type.

Methods

copy(self)

Returns a shallow copy of current class.

create([class_item])

This method creates an instance of a class with given type.

deepcopy(self)

Returns a deep copy of current class.

get_class_from_type(class_type)

Return the class associated with input type.

get_params(self)

Returns the dictionary of class hyperparameters.

get_state(self, **kwargs)

Returns the object state dictionary.

get_subclasses()

Get all the subclasses of the calling class.

list_class_types()

This method lists all types of available subclasses of calling one.

load(path)

Load Security evaluation data from file.

load_state(self, path)

Sets the object state from file.

save(self, path)

Load Security evaluation data from file.

save_state(self, path, **kwargs)

Store the object state to file.

set(self, param_name, param_value[, copy])

Set a parameter of the class.

set_params(self, params_dict[, copy])

Set all parameters passed as a dictionary {key: value}.

set_state(self, state_dict[, copy])

Sets the object state using input dictionary.

timed([msg])

Timer decorator.

property Y

Returns the values of the dataset true labels.

property Y_pred
Returns
Y_predlist of CArray

Contain one element for each attack power value. Each element contain label assigned to all the dataset samples from the attack.

property Y_target

Returns the values of the desired predicted labels.

property adv_ds
Returns
adv_dslist of CDataset.

containing one dataset for each different parameter value.

property fobj

Return objective function values with the different attack power

classmethod load(path)[source]

Load Security evaluation data from file.

Save a python dict containing all the results.

property param_name

Returns the name of the parameter representing the attack strenght.

property param_values

Returns the values of the security-evaluation parameter.

save(self, path)[source]

Load Security evaluation data from file.

Save a python dict containing all the results.

property scores
Returns
scores: list of CArray

Contain one element for each attack power value. Each element contain score assigned by the classifier to all the dataset samples.

property time
Returns
timeCArray (n_patterns, num parameter values)

Each array row contain the times of the attack for one samples. Each row element represent a different attack power.