secml.testing¶
CUnitTest¶
-
class
secml.testing.c_unittest.
CUnitTest
(methodName='runTest')[source]¶ Bases:
unittest.case.TestCase
Superclass for unittests.
Provides a wrapper of unittests.TestCase in addition to:
integrated logging functionalities (see CLog)
integrated timing functionalities (see CTimer)
addition assertion methods from numpy.testing
skip, skipif, importorskip functions from pytest
- Attributes
logger
Logger for current object.
Methods
__call__
(self, *args, **kwds)Call self as a function.
addCleanup
(*args, **kwargs)Add a function, with arguments, to be called when the test is completed.
addTypeEqualityFunc
(self, typeobj, function)Add a type specific assertEqual style function to compare a type.
assertAlmostEqual
(self, first, second[, …])Fail if the two objects are unequal as determined by their difference rounded to the given number of decimal places (default 7) and comparing to zero, or by comparing that the difference between the two objects is more than the given delta.
assertCountEqual
(self, first, second[, msg])An unordered sequence comparison asserting that the same elements, regardless of order.
assertDictContainsSubset
(self, subset, …)Checks whether dictionary is a superset of subset.
assertEqual
(self, first, second[, msg])Fail if the two objects are unequal as determined by the ‘==’ operator.
assertFalse
(self, expr[, msg])Check that the expression is false.
assertGreater
(self, a, b[, msg])Just like self.assertTrue(a > b), but with a nicer default message.
assertGreaterEqual
(self, a, b[, msg])Just like self.assertTrue(a >= b), but with a nicer default message.
assertIn
(self, member, container[, msg])Just like self.assertTrue(a in b), but with a nicer default message.
assertIs
(self, expr1, expr2[, msg])Just like self.assertTrue(a is b), but with a nicer default message.
assertIsInstance
(self, obj, cls[, msg])Same as self.assertTrue(isinstance(obj, cls)), with a nicer default message.
assertIsNone
(self, obj[, msg])Same as self.assertTrue(obj is None), with a nicer default message.
assertIsNot
(self, expr1, expr2[, msg])Just like self.assertTrue(a is not b), but with a nicer default message.
assertIsNotNone
(self, obj[, msg])Included for symmetry with assertIsNone.
assertIsSubDtype
(actual, desired)Raises an AssertionError if the first dtype is NOT a typecode lower/equal in type hierarchy.
assertLess
(self, a, b[, msg])Just like self.assertTrue(a < b), but with a nicer default message.
assertLessEqual
(self, a, b[, msg])Just like self.assertTrue(a <= b), but with a nicer default message.
assertListEqual
(self, list1, list2[, msg])A list-specific equality assertion.
assertLogs
(self[, logger, level])Fail unless a log message of level level or higher is emitted on logger_name or its children.
assertMultiLineEqual
(self, first, second[, msg])Assert that two multi-line strings are equal.
assertNotAlmostEqual
(self, first, second[, …])Fail if the two objects are equal as determined by their difference rounded to the given number of decimal places (default 7) and comparing to zero, or by comparing that the difference between the two objects is less than the given delta.
assertNotEqual
(self, first, second[, msg])Fail if the two objects are equal as determined by the ‘!=’ operator.
assertNotIn
(self, member, container[, msg])Just like self.assertTrue(a not in b), but with a nicer default message.
assertNotIsInstance
(self, obj, cls[, msg])Included for symmetry with assertIsInstance.
assertNotRegex
(self, text, unexpected_regex)Fail the test if the text matches the regular expression.
assertRaises
(self, expected_exception, …)Fail unless an exception of class expected_exception is raised by the callable when invoked with specified positional and keyword arguments.
assertRaisesRegex
(self, expected_exception, …)Asserts that the message in a raised exception matches a regex.
assertRegex
(self, text, expected_regex[, msg])Fail the test unless the text matches the regular expression.
assertSequenceEqual
(self, seq1, seq2[, msg, …])An equality assertion for ordered sequences (like lists and tuples).
assertSetEqual
(self, set1, set2[, msg])A set-specific equality assertion.
assertTrue
(self, expr[, msg])Check that the expression is true.
assertTupleEqual
(self, tuple1, tuple2[, msg])A tuple-specific equality assertion.
assertWarns
(self, expected_warning, *args, …)Fail unless a warning of class warnClass is triggered by the callable when invoked with specified positional and keyword arguments.
assertWarnsRegex
(self, expected_warning, …)Asserts that the message in a triggered warning matches a regexp.
assert_allclose
(self, actual, desired[, …])Raises an AssertionError if two objects are not equal up to desired tolerance.
assert_approx_equal
(self, actual, desired[, …])Raises an AssertionError if two items are not equal up to significant digits.
assert_array_almost_equal
(self, x, y[, …])Raises an AssertionError if two objects are not equal up to desired precision.
assert_array_almost_equal_nulp
(self, x, y[, …])Compare two arrays relatively to their spacing.
assert_array_equal
(self, x, y[, err_msg, …])Raises an AssertionError if two array_like objects are not equal.
assert_array_less
(self, x, y[, err_msg, verbose])Raises an AssertionError if two array_like objects are not ordered by less than.
assert_array_max_ulp
(self, a, b[, maxulp, dtype])Check that all items of arrays differ in at most N Units in the Last Place.
debug
(self)Run the test without collecting errors in a TestResult
doCleanups
(self)Execute all cleanup functions.
fail
(self[, msg])Fail immediately, with the given message.
failureException
alias of
builtins.AssertionError
alias of
unittest.main.TestProgram
setUp
(self)Hook method for setting up the test fixture before exercising it.
Hook method for setting up class fixture before running tests in the class.
shortDescription
(self)Returns a one-line description of the test, or None if no description has been provided.
skipTest
(self, reason)Skip this test.
subTest
(self[, msg])Return a context manager that will return the enclosed block of code in a subtest identified by the optional message and keyword parameters.
tearDown
(self)Hook method for deconstructing the test fixture after testing it.
tearDownClass
()Hook method for deconstructing the class fixture after running all tests in the class.
timer
(self)Returns a CTimer to be used as context manager.
assertAlmostEquals
assertDictEqual
assertEquals
assertNotAlmostEquals
assertNotEquals
assertNotRegexpMatches
assertRaisesRegexp
assertRegexpMatches
assert_
countTestCases
defaultTestResult
failIf
failIfAlmostEqual
failIfEqual
failUnless
failUnlessAlmostEqual
failUnlessEqual
failUnlessRaises
id
importskip
run
skip
skipif
-
static
assertIsSubDtype
(actual, desired)[source]¶ Raises an AssertionError if the first dtype is NOT a typecode lower/equal in type hierarchy.
- Parameters
- actualdtype or str
dtype or string representing a typecode.
- desireddtype or str
dtype or string representing a typecode.
-
assert_allclose
(self, actual, desired, rtol=1e-06, atol=0, equal_nan=True, err_msg='', verbose=True)[source]¶ Raises an AssertionError if two objects are not equal up to desired tolerance.
The test is equivalent to
allclose(actual, desired, rtol, atol)
(note thatallclose
has different default values). It compares the difference between actual and desired toatol + rtol * abs(desired)
.New in version 1.5.0.
- Parameters
- actualarray_like
Array obtained.
- desiredarray_like
Array desired.
- rtolfloat, optional
Relative tolerance.
- atolfloat, optional
Absolute tolerance.
- equal_nanbool, optional.
If True, NaNs will compare equal.
- err_msgstr, optional
The error message to be printed in case of failure.
- verbosebool, optional
If True, the conflicting values are appended to the error message.
- Raises
- AssertionError
If actual and desired are not equal up to specified precision.
Examples
>>> x = [1e-5, 1e-3, 1e-1] >>> y = np.arccos(np.cos(x)) >>> np.testing.assert_allclose(x, y, rtol=1e-5, atol=0)
-
assert_approx_equal
(self, actual, desired, significant=6, err_msg='', verbose=True)[source]¶ Raises an AssertionError if two items are not equal up to significant digits.
Note
It is recommended to use one of assert_allclose, assert_array_almost_equal_nulp or assert_array_max_ulp instead of this function for more consistent floating point comparisons.
Given two numbers, check that they are approximately equal. Approximately equal is defined as the number of significant digits that agree.
- Parameters
- actualscalar
The object to check.
- desiredscalar
The expected object.
- significantint, optional
Desired precision, default is 7.
- err_msgstr, optional
The error message to be printed in case of failure.
- verbosebool, optional
If True, the conflicting values are appended to the error message.
- Raises
- AssertionError
If actual and desired are not equal up to specified precision.
See also
assert_allclose
Compare two array_like objects for equality with desired relative and/or absolute precision.
assert_array_almost_equal_nulp
,assert_array_max_ulp
,assert_equal
Examples
>>> np.testing.assert_approx_equal(0.12345677777777e-20, 0.1234567e-20) >>> np.testing.assert_approx_equal(0.12345670e-20, 0.12345671e-20, ... significant=8) >>> np.testing.assert_approx_equal(0.12345670e-20, 0.12345672e-20, ... significant=8) Traceback (most recent call last): ... AssertionError: Items are not equal to 8 significant digits: ACTUAL: 1.234567e-21 DESIRED: 1.2345672e-21
the evaluated condition that raises the exception is
>>> abs(0.12345670e-20/1e-21 - 0.12345672e-20/1e-21) >= 10**-(8-1) True
-
assert_array_almost_equal
(self, x, y, decimal=6, err_msg='', verbose=True)[source]¶ Raises an AssertionError if two objects are not equal up to desired precision.
Note
It is recommended to use one of assert_allclose, assert_array_almost_equal_nulp or assert_array_max_ulp instead of this function for more consistent floating point comparisons.
The test verifies identical shapes and that the elements of
actual
anddesired
satisfy.abs(desired-actual) < 1.5 * 10**(-decimal)
That is a looser test than originally documented, but agrees with what the actual implementation did up to rounding vagaries. An exception is raised at shape mismatch or conflicting values. In contrast to the standard usage in numpy, NaNs are compared like numbers, no assertion is raised if both objects have NaNs in the same positions.
- Parameters
- xarray_like
The actual object to check.
- yarray_like
The desired, expected object.
- decimalint, optional
Desired precision, default is 6.
- err_msgstr, optional
The error message to be printed in case of failure.
- verbosebool, optional
If True, the conflicting values are appended to the error message.
- Raises
- AssertionError
If actual and desired are not equal up to specified precision.
See also
assert_allclose
Compare two array_like objects for equality with desired relative and/or absolute precision.
assert_array_almost_equal_nulp
,assert_array_max_ulp
,assert_equal
Examples
the first assert does not raise an exception
>>> np.testing.assert_array_almost_equal([1.0,2.333,np.nan], ... [1.0,2.333,np.nan])
>>> np.testing.assert_array_almost_equal([1.0,2.33333,np.nan], ... [1.0,2.33339,np.nan], decimal=5) Traceback (most recent call last): ... AssertionError: Arrays are not almost equal to 5 decimals Mismatched elements: 1 / 3 (33.3%) Max absolute difference: 6.e-05 Max relative difference: 2.57136612e-05 x: array([1. , 2.33333, nan]) y: array([1. , 2.33339, nan])
>>> np.testing.assert_array_almost_equal([1.0,2.33333,np.nan], ... [1.0,2.33333, 5], decimal=5) Traceback (most recent call last): ... AssertionError: Arrays are not almost equal to 5 decimals x and y nan location mismatch: x: array([1. , 2.33333, nan]) y: array([1. , 2.33333, 5. ])
-
assert_array_almost_equal_nulp
(self, x, y, nulp=1)[source]¶ Compare two arrays relatively to their spacing.
This is a relatively robust method to compare two arrays whose amplitude is variable.
- Parameters
- x, yarray_like
Input arrays.
- nulpint, optional
The maximum number of unit in the last place for tolerance (see Notes). Default is 1.
- Returns
- None
- Raises
- AssertionError
If the spacing between x and y for one or more elements is larger than nulp.
See also
assert_array_max_ulp
Check that all items of arrays differ in at most N Units in the Last Place.
spacing
Return the distance between x and the nearest adjacent number.
Notes
An assertion is raised if the following condition is not met:
abs(x - y) <= nulps * spacing(maximum(abs(x), abs(y)))
Examples
>>> x = np.array([1., 1e-10, 1e-20]) >>> eps = np.finfo(x.dtype).eps >>> np.testing.assert_array_almost_equal_nulp(x, x*eps/2 + x)
>>> np.testing.assert_array_almost_equal_nulp(x, x*eps + x) Traceback (most recent call last): ... AssertionError: X and Y are not equal to 1 ULP (max is 2)
-
assert_array_equal
(self, x, y, err_msg='', verbose=True)[source]¶ Raises an AssertionError if two array_like objects are not equal.
Given two array_like objects, check that the shape is equal and all elements of these objects are equal (but see the Notes for the special handling of a scalar). An exception is raised at shape mismatch or conflicting values. In contrast to the standard usage in numpy, NaNs are compared like numbers, no assertion is raised if both objects have NaNs in the same positions.
The usual caution for verifying equality with floating point numbers is advised.
- Parameters
- xarray_like
The actual object to check.
- yarray_like
The desired, expected object.
- err_msgstr, optional
The error message to be printed in case of failure.
- verbosebool, optional
If True, the conflicting values are appended to the error message.
- Raises
- AssertionError
If actual and desired objects are not equal.
See also
assert_allclose
Compare two array_like objects for equality with desired relative and/or absolute precision.
assert_array_almost_equal_nulp
,assert_array_max_ulp
,assert_equal
Notes
When one of x and y is a scalar and the other is array_like, the function checks that each element of the array_like object is equal to the scalar.
Examples
The first assert does not raise an exception:
>>> np.testing.assert_array_equal([1.0,2.33333,np.nan], ... [np.exp(0),2.33333, np.nan])
Assert fails with numerical imprecision with floats:
>>> np.testing.assert_array_equal([1.0,np.pi,np.nan], ... [1, np.sqrt(np.pi)**2, np.nan]) Traceback (most recent call last): ... AssertionError: Arrays are not equal Mismatched elements: 1 / 3 (33.3%) Max absolute difference: 4.4408921e-16 Max relative difference: 1.41357986e-16 x: array([1. , 3.141593, nan]) y: array([1. , 3.141593, nan])
Use assert_allclose or one of the nulp (number of floating point values) functions for these cases instead:
>>> np.testing.assert_allclose([1.0,np.pi,np.nan], ... [1, np.sqrt(np.pi)**2, np.nan], ... rtol=1e-10, atol=0)
As mentioned in the Notes section, assert_array_equal has special handling for scalars. Here the test checks that each value in x is 3:
>>> x = np.full((2, 5), fill_value=3) >>> np.testing.assert_array_equal(x, 3)
-
assert_array_less
(self, x, y, err_msg='', verbose=True)[source]¶ Raises an AssertionError if two array_like objects are not ordered by less than.
Given two array_like objects, check that the shape is equal and all elements of the first object are strictly smaller than those of the second object. An exception is raised at shape mismatch or incorrectly ordered values. Shape mismatch does not raise if an object has zero dimension. In contrast to the standard usage in numpy, NaNs are compared, no assertion is raised if both objects have NaNs in the same positions.
- Parameters
- xarray_like
The smaller object to check.
- yarray_like
The larger object to compare.
- err_msgstring
The error message to be printed in case of failure.
- verbosebool
If True, the conflicting values are appended to the error message.
- Raises
- AssertionError
If actual and desired objects are not equal.
See also
assert_array_equal
tests objects for equality
assert_array_almost_equal
test objects for equality up to precision
Examples
>>> np.testing.assert_array_less([1.0, 1.0, np.nan], [1.1, 2.0, np.nan]) >>> np.testing.assert_array_less([1.0, 1.0, np.nan], [1, 2.0, np.nan]) Traceback (most recent call last): ... AssertionError: Arrays are not less-ordered Mismatched elements: 1 / 3 (33.3%) Max absolute difference: 1. Max relative difference: 0.5 x: array([ 1., 1., nan]) y: array([ 1., 2., nan])
>>> np.testing.assert_array_less([1.0, 4.0], 3) Traceback (most recent call last): ... AssertionError: Arrays are not less-ordered Mismatched elements: 1 / 2 (50%) Max absolute difference: 2. Max relative difference: 0.66666667 x: array([1., 4.]) y: array(3)
>>> np.testing.assert_array_less([1.0, 2.0, 3.0], [4]) Traceback (most recent call last): ... AssertionError: Arrays are not less-ordered (shapes (3,), (1,) mismatch) x: array([1., 2., 3.]) y: array([4])
-
assert_array_max_ulp
(self, a, b, maxulp=1, dtype=None)[source]¶ Check that all items of arrays differ in at most N Units in the Last Place.
- Parameters
- a, barray_like
Input arrays to be compared.
- maxulpint, optional
The maximum number of units in the last place that elements of a and b can differ. Default is 1.
- dtypedtype, optional
Data-type to convert a and b to if given. Default is None.
- Returns
- retndarray
Array containing number of representable floating point numbers between items in a and b.
- Raises
- AssertionError
If one or more elements differ by more than maxulp.
See also
assert_array_almost_equal_nulp
Compare two arrays relatively to their spacing.
Notes
For computing the ULP difference, this API does not differentiate between various representations of NAN (ULP difference between 0x7fc00000 and 0xffc00000 is zero).
Examples
>>> a = np.linspace(0., 1., 100) >>> res = np.testing.assert_array_max_ulp(a, np.arcsin(np.sin(a)))
-
importskip
¶
-
property
logger
¶ Logger for current object.
-
main
¶ alias of
unittest.main.TestProgram
-
classmethod
setUpClass
()[source]¶ Hook method for setting up class fixture before running tests in the class.
-
skip
¶
-
skipif
¶