secml.optim.optimizers.line_search

CLineSearch

class secml.optim.optimizers.line_search.c_line_search.CLineSearch(fun, constr=None, bounds=None, eta=0.0001, max_iter=20)

Bases: secml.core.c_creator.CCreator

Abstract class that implements line-search optimization algorithms.

Line-search algorithms optimize the objective function along a given direction in the feasible domain, potentially subject to constraints. The search is normally stopped when the objective improves at a satisfying level, to keep the search fast.

Parameters

funCFunction

The function to use for the optimization.

constrCConstraintL1 or CConstraintL2 or None, optional

A distance constraint. Default None.

boundsCConstraintBox or None, optional

A box constraint. Default None.

etascalar, optional

Minimum resolution of the line-search grid. Default 1e-4.

max_iterint, optional

Maximum number of iterations of the line search. Default 20.

Attributes

class_type

Defines class type.

logger

Logger for current object.

verbose

Verbosity level of logger output.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
deepcopy(self)	Returns a deep copy of current class.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class hyperparameters.
get_state(self, **kwargs)	Returns the object state dictionary.
get_subclasses()	Get all the subclasses of the calling class.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads object from file.
load_state(self, path)	Sets the object state from file.
minimize(self, x, d, **kwargs)	Line search.
save(self, path)	Save class object to file.
save_state(self, path, **kwargs)	Store the object state to file.
set(self, param_name, param_value[, copy])	Set a parameter of the class.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
set_state(self, state_dict[, copy])	Sets the object state using input dictionary.
timed([msg])	Timer decorator.

abstract minimize(self, x, d, **kwargs)

Line search.

Parameters

xCArray

The input point.

dCArray

The descent direction along which fun(x) is minimized.

kwargsdict

Additional parameters required to evaluate fun(x, **kwargs).

CLineSearchBisect

class secml.optim.optimizers.line_search.c_line_search_bisect.CLineSearchBisect(fun, constr=None, bounds=None, eta=0.0001, eta_min=0.1, eta_max=None, max_iter=20)

Bases: secml.optim.optimizers.line_search.c_line_search.CLineSearch

Binary line search.

Parameters

funCFunction

The function to use for the optimization.

constrCConstraintL1 or CConstraintL2 or None, optional

A distance constraint. Default None.

boundsCConstraintBox or None, optional

A box constraint. Default None.

etascalar, optional

Minimum resolution of the line-search grid. Default 1e-4.

eta_minscalar or None, optional

Initial step of the line search. Gets multiplied or divided by 2 at each step until convergence. If None, will be set equal to eta. Default 0.1.

eta_maxscalar or None, optional

Maximum step of the line search. Default None.

max_iterint, optional

Maximum number of iterations of the line search. Default 20.

Attributes

class_type‘bisect’

Defines class type.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
deepcopy(self)	Returns a deep copy of current class.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class hyperparameters.
get_state(self, **kwargs)	Returns the object state dictionary.
get_subclasses()	Get all the subclasses of the calling class.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads object from file.
load_state(self, path)	Sets the object state from file.
minimize(self, x, d[, fx, tol])	Bisect line search (on discrete grid).
save(self, path)	Save class object to file.
save_state(self, path, **kwargs)	Store the object state to file.
set(self, param_name, param_value[, copy])	Set a parameter of the class.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
set_state(self, state_dict[, copy])	Sets the object state using input dictionary.
timed([msg])	Timer decorator.

property eta_max

property eta_min

minimize(self, x, d, fx=None, tol=0.0001, **kwargs)

Bisect line search (on discrete grid).

The function fun( x + a*eta*d ) with a = {0, 1, 2, … } is minimized along the descent direction d.

If fun(x) >= 0 -> step_min = step else step_max = step

If eta_max is not None, it runs a bisect line search in [x + eta_min*d, x + eta_max*d]; otherwise, it runs an exponential line search in `[x + eta*d, …, x + eta_min*d, …]

Parameters

xCArray

The input point.

dCArray

The descent direction along which fun(x) is minimized.

fxint or float or None, optional

The current value of fun(x) (if available).

tolfloat, optional

Tolerance for convergence to the local minimum.

kwargsdict

Additional parameters required to evaluate fun(x, **kwargs).

Returns

x’CArray

Point x’ = x + eta * d that approximately solves min f(x + eta*d).

fx’: int or float or None, optional

The value f(x’).

property n_iter

CLineSearchBisectProj

class secml.optim.optimizers.line_search.c_line_search_bisect_proj.CLineSearchBisectProj(fun, constr=None, bounds=None, eta=0.0001, eta_min=0.1, eta_max=None, max_iter=20)

Bases: secml.optim.optimizers.line_search.c_line_search_bisect.CLineSearchBisect

Binary line search including projections.

Parameters

funCFunction

The function to use for the optimization.

constrCConstraintL1 or CConstraintL2 or None, optional

A distance constraint. Default None.

boundsCConstraintBox or None, optional

A box constraint. Default None.

etascalar, optional

Minimum resolution of the line-search grid. Default 1e-4.

eta_minscalar or None, optional

Initial step of the line search. Gets multiplied or divided by 2 at each step until convergence. If None, will be set equal to eta. Default 0.1.

eta_maxscalar or None, optional

Maximum step of the line search. Default None.

max_iterint, optional

Maximum number of iterations of the line search. Default 20.

Attributes

class_type‘bisect-proj’

Defines class type.

Methods

copy(self)	Returns a shallow copy of current class.
create([class_item])	This method creates an instance of a class with given type.
deepcopy(self)	Returns a deep copy of current class.
get_class_from_type(class_type)	Return the class associated with input type.
get_params(self)	Returns the dictionary of class hyperparameters.
get_state(self, **kwargs)	Returns the object state dictionary.
get_subclasses()	Get all the subclasses of the calling class.
list_class_types()	This method lists all types of available subclasses of calling one.
load(path)	Loads object from file.
load_state(self, path)	Sets the object state from file.
minimize(self, x, d[, fx, tol])	Exponential line search (on discrete grid).
save(self, path)	Save class object to file.
save_state(self, path, **kwargs)	Store the object state to file.
set(self, param_name, param_value[, copy])	Set a parameter of the class.
set_params(self, params_dict[, copy])	Set all parameters passed as a dictionary {key: value}.
set_state(self, state_dict[, copy])	Sets the object state using input dictionary.
timed([msg])	Timer decorator.

minimize(self, x, d, fx=None, tol=0.0001, **kwargs)

Exponential line search (on discrete grid).

The function fun( x + a*eta*d ) with a = {0, 1, 2, … } is minimized along the descent direction d.

If fun(x) >= 0 -> step_min = step else step_max = step

If eta_max is not None, it runs a bisect line search in [x + eta_min*d, x + eta_max*d]; otherwise, it runs an exponential line search in `[x + eta*d, …, x + eta_min*d, …]

Parameters

xCArray

The input point.

dCArray

The descent direction along which fun(x) is minimized.

fxint or float or None, optional

The current value of fun(x) (if available).

tolfloat, optional

Tolerance for convergence to the local minimum.

kwargsdict

Additional parameters required to evaluate fun(x, **kwargs).

Returns

x’CArray

Point x’ = x + eta * d that approximately solves min f(x + eta*d).

fx’: int or float or None, optional

The value f(x’).