diff options
Diffstat (limited to 'python-checking.spec')
| -rw-r--r-- | python-checking.spec | 1068 |
1 files changed, 1068 insertions, 0 deletions
diff --git a/python-checking.spec b/python-checking.spec new file mode 100644 index 0000000..a003487 --- /dev/null +++ b/python-checking.spec @@ -0,0 +1,1068 @@ +%global _empty_manifest_terminate_build 0 +Name: python-checking +Version: 0.9.1 +Release: 1 +Summary: A small library for unit-testing +License: MIT License +URL: https://github.com/kotolex/checking +Source0: https://mirrors.nju.edu.cn/pypi/web/packages/29/e1/679f236f513ac33f52e5f4bd6e36f733e9313c5a858ec1a7d40731c47472/checking-0.9.1.tar.gz +BuildArch: noarch + + +%description +Test "__main__.check_cat" [Cat from 140288585437776] SUCCESS! +``` +If you want to use a text file as a data source, you can use `DATA_FILE` helper function to skip the file handling boilerplate code: +```python +from checking import * +DATA_FILE('files/data.txt', name='provider') # Use the file located at <module folder>/files/data.txt +@test(data_provider='provider') +def try_prov(it): + print(it) + is_true(it) +``` +The helper lazy-loads specified data file line by line. +Raises FileNotFoundError if the file is not found. +Also, you can transform all the lines before feeding them into the test, +for example delete trailing newlines at the end of each line: +```python +from checking import * +DATA_FILE('files/data.txt', name='provider', map_function=str.rstrip) # Feed each line through str.rstrip() +@test(data_provider='provider') +def try_prov(it): + is_true(it) +``` +If you don't specify provider_name for the DATA_FILE helper, file_path will be used: +```python +from checking import * +DATA_FILE('data.txt') # Use text file located at the module folder. Note, that no provider_name is specified. +@test(data_provider='data.txt') # Use the specified file_name parameter for provider lookup +def try_prov(it): + is_true(it) +``` +If your test suite uses a data provider more than once, you might want to avoid the IO overhead, +if this provider fetches the data from some external source (database, file system, http request etc.). +You can use the `cached` parameter to force the provider to fetch the data only once and store it into memory. +Please, be varied of the memory consumption, because the cache persists until the whole suite is done running. +Also, be careful when using the cache when running tests in parallel. +DATA_FILE helper can use this parameter too. +```python +from checking import * +DATA_FILE('data.csv', name='csv', cached=True) # Enable caching +@test(data_provider='csv') # First provider use -- data is fetched from the file and stored into memory +def check_one(it): + not_none(it) +@test(data_provider='csv') # Second use -- no file reads, cached data is used +def check_two(it): + not_none(it) +if __name__ == '__main__': + start(0) +``` +If your provider is a simple one-liner (string, list comprehension, generator expression, etc.), +you can use the CONTAINER helper function to avoid full function definition boilerplate: +```python +from checking import * +CONTAINER([e for e in range(10)], name='range') # Provide data from a listcomps, set provider name to 'range' +@test(data_provider='range') +def try_container(it): + is_true(it in range(10)) +``` +'name' parameter is optional, 'container' is used by default, +but it's strongly recommended using a unique name: +```python +from checking import * +CONTAINER((e for e in range(10))) # Provide data from a genexps +@test(data_provider='container') +def try_container(it): + is_true(it in range(10)) +``` +**Important!** You must define DATA_FILE or CONTAINER providers at the module scope, not in the fixtures and tests. +### Test Parameters ### +You can manage the test execution mode by passing a number of parameters to the @test decorator: +**enabled** (bool) - if set to False, the test will be skipped, all other parameters are ignored. By default, set to True. +**name** (str) - the name of the test. Is bound to the decorated function name if not specified. +**description** (str) - test description. If absent, the test function docstring is used. +If both description and docstring are present, description takes precedence. +**data_provider** (str) - the name of the data provider to use with the test. +If specified, the test function must take one argument to be fed with the data from the provider. +Raises UnknownProviderName if no providers with the specified name found. +**retries** (int) - the number of times to run the failing test. If test does not fail, no more runs attempted. By default, set to 1. +**groups** (Tuple[str]) - a tuple of strings, representing the test group names a test is a part of. +All tests belong to some test group, the default group holds all tests from the current module and is named after the module. +Use this parameter to manage test execution groups. +**priority** (int) - test priority. The higher the value the later the test will be executed. +Use this parameter to fine tune test run order. By default, set to 0. +**timeout** (int) - amount of time to wait for the test to end. +If the time runs out, the thread running the test is terminated and the test is marked as "broken". +Use sparingly due to potential memory leaks. +**only_if** (Callable[None, bool]) - boolean predicate, which is evaluated before the test execution. +The test will be executed only if the predicate evaluates to True. +Use this parameter for conditional test execution e.g. run only if the OS is Linux, etc. +## Fixtures +Each test group or all test-suite can have preconditions and post-actions. For example, open DB connection before test starts and close it after that. +You can easily make it with before/after fixtures. The function that marked with before/after should be without arguments. +@before - run function before EACH test in group, by default group is current module, but you can specify it with parameter +@after - run function after EACH test in group, by default group is current module, but you can specify it with parameter. +This function will not be run if there is @before and it failed! +```python +@before(group_name='api') +def my_func(): + do_some_precondition() +@after(group_name='api') +def another_func(): + do_post_actions() +``` +@before_group - function run once before running test in group, by default group is current module, but you can specify it with parameter. +@after_group - function run once after running all test in group, by default group is current module, but you can specify it with parameter. +This function will not be run if there is @before_group and it failed, except using parameter always_run = True +```python +@before_group(name='api') +def my_func(): + do_some_precondition_for_whole_group() +@after_group(name='api', always_run =True) +def another_func(): + do_post_actions_for_whole_group() +``` +@before_suite - function runs once before any group at start of the test-suite +@after_suite - function run once after all groups, at the end of the test-suite. +This function will not be run if there is @before_suite, and it failed, except using parameter 'always_run = True' +```python +@before_suite +def my_func(): + print('start suite!') +@after_suite(always_run=True) +def another_func(): + print('will be printed, even if before_suite failed!') +``` +## Mock, Double, Stub and Spy +For testing purposes you sometimes need to fake some behaviour or to isolate your application from any other classes/libraries etc. +If you need your test to use fake object, without doing any real calls, you can use mocks: +**1. Fake one of the builtin function.** +Let say you need to test function which is using standard input() inside. +But you cannot wait for real user input during the test, so fake it with mock object. +```python +def our_weird_function_with_input_inside(): + text = input() + return text.upper() +@test +def mock_builtins_input(): + with mock_builtins('input', lambda : 'test'): # Now input() just returns 'test', it does not wait for user input. + result_text = our_weird_function_with_input_inside() + equals('TEST', result_text) +``` +More convenient way is to use mock_input or mock_print for simple and most common cases. +From code above we can test our_weird_function this way +```python +@test +def check_input(): + with mock_input(['test']): # Now input() just returns 'test', it does not wait for user input. + result_text = our_weird_function_with_input_inside() + equals('TEST', result_text) +``` +Now let's say we have simple function with print inside and need to test it: +```python +def my_print(x): + print(x) +@test +def check_print(): + with mock_print([]) as result: # now print just collects all to list result + my_print(1) + my_print('1') + equals([(1,), ('1',)], result) # checks all args are in result list +``` +and more complicated case, when our function works forever, printing all inputs, until gets 'exit': +```python +def use_both(): + while True: + word = input('text>>>') + if word == 'exit': + break + print(word) +@test +def check_print_and_input(): + # you can see inputs will get 'a','b' and 'exit' to break cycle, all args will + # be collected to result list + with mock_input(['a', 'b', 'exit']), mock_print([]) as result: + use_both() + equals([('a',), ('b',)], result) +``` +**2. Fake function of the 3-d party library** +For working with other modules and libraries in test module, you need to import this module and to mock it function. +For example, you need to test function, which is using requests.get inside, but you do not want to make real http +request. Let it mock +some_module_to_test.py +```python +import requests +def func_with_get_inside(url): + response = requests.get(url) + return response.text +``` +our_tests.py +```python +import requests # need to import it for mock! +from some_module_to_test import func_with_get_inside +@test +def mock_requests_get(): + stub = Stub(text='test') # create simple stub, with attribute text equals to 'test' + with mock(requests, 'get', lambda x: stub): # Mock real requests with stub object + equals('test', func_with_get_inside('https://yandex.ru')) # Now no real requests be performed! +``` +**3. Mock read/write to file** +If you need to mock open function, push data to read from file and gets back with write to file, you can use +mock_open context-manager +```python +def my_open(): + # We read from one file, uppercase results and write to another file + with open('my_file.txt', encoding='utf-8') as f, open('another.txt', 'wt') as f2: + f2.write(f.readline().upper()) +@test +def mock_open_both(): + # Here we specify what we must "read from file" ('test') and where we want to get all writes(result) + with mock_open(on_read_text='test') as result: + my_open() + equals(['TEST'], result) # checks we get test uppercase +``` +**4. Spy object** +Spy is the object which has all attributes of original, but spy not performed any action, +all methods return None (if not specified what to return). Therefore, spy log all actions and arguments. +It can be useful if your code has inner object, and you need to test what functions were called. +```python +def function_with_str_inside(value): + # Suppose we need to check upper was called here inside + return value.upper() +@test +def spy_for_str(): + spy = Spy('it is a string') # Spy, which is like str, but it is not str! + function_with_str_inside(spy) # Send our spy instead a str + is_true(spy.upper.was_called()) # Verify upper was called +``` +You can even specify what to return when some function of the spy will be called! +```python +def function_with_str_inside(value): + # Suppose we need to check upper was called here inside + return value.upper() +@test +def spy_with_return(): + spy = Spy('string') + spy.upper.returns('test') # Tells what to return, when upper will be call + result = function_with_str_inside(spy) + is_true(spy.upper.was_called()) + equals('test', result) # verify our spy returns 'test' +``` +Spy object can be created without original inner object and can be call itself, it can be useful when you need +some dumb object to know it was called. +```python +@test +def check_spy(): + spy = Spy() # Create "empty" spy + spy() # Call it + is_true(spy.was_called()) # Checks spy was called +``` +**5. TestDouble object** +Test-Double object is like the Spy, but it saves original object behaviour, so its methods returns +real object methods results if not specified otherwise. +```python +@test +def check_double(): + spy = TestDouble("string") # Create str double-object + equals(6, len(spy)) # Len returns 6 - the real length of original object ("string") + spy.len.returns(100) # Fake len result + equals(100, len(spy)) # Len now returns 100 +``` +**Important!** Both spy and TestDouble override **isinstance**, so they emulate type of the original object. It can be useful for +testing functions, which has isinstance check inside. For example: +```python +def function_that_checks_class(obj): + if isinstance(obj, str): # check for argument type (string) + return "OK" + return "Not OK" +@test +def isinstance_check(): + spy = Spy("fake string") # fake the real string + result = function_that_checks_class(spy) # get "OK" here, cause function thinks it's a string, not Spy + equals("OK", result) +``` +**6. Stub object** +Stub object is just a helper for testing, its purpose not to check or assert something, but to give data +and perform some simple action, when application under test need it. Unlike spy or double, Stub +is not remember calls, it just a simple replacement for some object with minimum or no logic inside. +Let's say we have a function which gets some object, take its attribute, calculates something and +return result. We wish to isolate our testing from real objects, just test important behaviour, besides +this data-object can be hard to create or complicated. +```python +from checking import * +# Our function to test, it get some object and use it attribute and method, but we just +# need to test how it works! +def function(some_object)->int: + initial_value = some_object.value + result = 2 + some_object.complicate_function()*initial_value # Some calculation we need to test + return result +@test +def check_with_stub(): + stub = Stub(value=2) # Creates stub with attribute value=2 + stub.complicate_function.returns(2) # Says, when complicate_function will be called returns 2 + equals(6, function(stub)) # Asserts 6 == 2+(2*2) +``` +Pay attention - when you look for some attribute in stub - it always has it! But it will be a wrapper to use with +expression like `stub.any_attribute.returns('test')`. +So, if you need to have some attribute (not method) on stub, you just use `stub.attr=10`, but for methods just use expression above. +### Function start() to runs test at module ### +You can execute all test at current module using function start(). For example: +```python +from checking import * +@test +def some_check(): + equals(4, 2+2) +if __name__ == '__main__': + start(3) # Here we run our test function some_check +``` +There are parameters to run your tests in different ways: +**suite_name** - name of the test-suite, to use in reports or in logs +**listener** - object of Listener class, test listener, is the way to work with test results and execution +DefaultListener is used by default. If set, then the verbose parameter is ignored (the one in the listener is used). +**verbose** is the report detail, 0 - briefly (only dots and 1 letter), 1 - detail, indicating only failed +tests, 2 - detail, indicating successful and fallen tests, 3 - detail and at the end, a list of fallen and broken ones +If verbose is not between 0 and 3, then 0 is accepted +Example (name and verbose) +```python +from checking import * +@test +def some_check(): + equals(4, 2 + 2) +@test +def some_check_two(): + equals(2, 1 + 1) +@test +def failed(): + equals(5, 2 + 2) # Will fail +@test +def broken(): + int('a') # Will be broken +if __name__ == '__main__': + start(suite_name='My Suite', verbose=0) +``` +This code will gave output (mention dots and chars!): +```text + +%package -n python3-checking +Summary: A small library for unit-testing +Provides: python-checking +BuildRequires: python3-devel +BuildRequires: python3-setuptools +BuildRequires: python3-pip +%description -n python3-checking +Test "__main__.check_cat" [Cat from 140288585437776] SUCCESS! +``` +If you want to use a text file as a data source, you can use `DATA_FILE` helper function to skip the file handling boilerplate code: +```python +from checking import * +DATA_FILE('files/data.txt', name='provider') # Use the file located at <module folder>/files/data.txt +@test(data_provider='provider') +def try_prov(it): + print(it) + is_true(it) +``` +The helper lazy-loads specified data file line by line. +Raises FileNotFoundError if the file is not found. +Also, you can transform all the lines before feeding them into the test, +for example delete trailing newlines at the end of each line: +```python +from checking import * +DATA_FILE('files/data.txt', name='provider', map_function=str.rstrip) # Feed each line through str.rstrip() +@test(data_provider='provider') +def try_prov(it): + is_true(it) +``` +If you don't specify provider_name for the DATA_FILE helper, file_path will be used: +```python +from checking import * +DATA_FILE('data.txt') # Use text file located at the module folder. Note, that no provider_name is specified. +@test(data_provider='data.txt') # Use the specified file_name parameter for provider lookup +def try_prov(it): + is_true(it) +``` +If your test suite uses a data provider more than once, you might want to avoid the IO overhead, +if this provider fetches the data from some external source (database, file system, http request etc.). +You can use the `cached` parameter to force the provider to fetch the data only once and store it into memory. +Please, be varied of the memory consumption, because the cache persists until the whole suite is done running. +Also, be careful when using the cache when running tests in parallel. +DATA_FILE helper can use this parameter too. +```python +from checking import * +DATA_FILE('data.csv', name='csv', cached=True) # Enable caching +@test(data_provider='csv') # First provider use -- data is fetched from the file and stored into memory +def check_one(it): + not_none(it) +@test(data_provider='csv') # Second use -- no file reads, cached data is used +def check_two(it): + not_none(it) +if __name__ == '__main__': + start(0) +``` +If your provider is a simple one-liner (string, list comprehension, generator expression, etc.), +you can use the CONTAINER helper function to avoid full function definition boilerplate: +```python +from checking import * +CONTAINER([e for e in range(10)], name='range') # Provide data from a listcomps, set provider name to 'range' +@test(data_provider='range') +def try_container(it): + is_true(it in range(10)) +``` +'name' parameter is optional, 'container' is used by default, +but it's strongly recommended using a unique name: +```python +from checking import * +CONTAINER((e for e in range(10))) # Provide data from a genexps +@test(data_provider='container') +def try_container(it): + is_true(it in range(10)) +``` +**Important!** You must define DATA_FILE or CONTAINER providers at the module scope, not in the fixtures and tests. +### Test Parameters ### +You can manage the test execution mode by passing a number of parameters to the @test decorator: +**enabled** (bool) - if set to False, the test will be skipped, all other parameters are ignored. By default, set to True. +**name** (str) - the name of the test. Is bound to the decorated function name if not specified. +**description** (str) - test description. If absent, the test function docstring is used. +If both description and docstring are present, description takes precedence. +**data_provider** (str) - the name of the data provider to use with the test. +If specified, the test function must take one argument to be fed with the data from the provider. +Raises UnknownProviderName if no providers with the specified name found. +**retries** (int) - the number of times to run the failing test. If test does not fail, no more runs attempted. By default, set to 1. +**groups** (Tuple[str]) - a tuple of strings, representing the test group names a test is a part of. +All tests belong to some test group, the default group holds all tests from the current module and is named after the module. +Use this parameter to manage test execution groups. +**priority** (int) - test priority. The higher the value the later the test will be executed. +Use this parameter to fine tune test run order. By default, set to 0. +**timeout** (int) - amount of time to wait for the test to end. +If the time runs out, the thread running the test is terminated and the test is marked as "broken". +Use sparingly due to potential memory leaks. +**only_if** (Callable[None, bool]) - boolean predicate, which is evaluated before the test execution. +The test will be executed only if the predicate evaluates to True. +Use this parameter for conditional test execution e.g. run only if the OS is Linux, etc. +## Fixtures +Each test group or all test-suite can have preconditions and post-actions. For example, open DB connection before test starts and close it after that. +You can easily make it with before/after fixtures. The function that marked with before/after should be without arguments. +@before - run function before EACH test in group, by default group is current module, but you can specify it with parameter +@after - run function after EACH test in group, by default group is current module, but you can specify it with parameter. +This function will not be run if there is @before and it failed! +```python +@before(group_name='api') +def my_func(): + do_some_precondition() +@after(group_name='api') +def another_func(): + do_post_actions() +``` +@before_group - function run once before running test in group, by default group is current module, but you can specify it with parameter. +@after_group - function run once after running all test in group, by default group is current module, but you can specify it with parameter. +This function will not be run if there is @before_group and it failed, except using parameter always_run = True +```python +@before_group(name='api') +def my_func(): + do_some_precondition_for_whole_group() +@after_group(name='api', always_run =True) +def another_func(): + do_post_actions_for_whole_group() +``` +@before_suite - function runs once before any group at start of the test-suite +@after_suite - function run once after all groups, at the end of the test-suite. +This function will not be run if there is @before_suite, and it failed, except using parameter 'always_run = True' +```python +@before_suite +def my_func(): + print('start suite!') +@after_suite(always_run=True) +def another_func(): + print('will be printed, even if before_suite failed!') +``` +## Mock, Double, Stub and Spy +For testing purposes you sometimes need to fake some behaviour or to isolate your application from any other classes/libraries etc. +If you need your test to use fake object, without doing any real calls, you can use mocks: +**1. Fake one of the builtin function.** +Let say you need to test function which is using standard input() inside. +But you cannot wait for real user input during the test, so fake it with mock object. +```python +def our_weird_function_with_input_inside(): + text = input() + return text.upper() +@test +def mock_builtins_input(): + with mock_builtins('input', lambda : 'test'): # Now input() just returns 'test', it does not wait for user input. + result_text = our_weird_function_with_input_inside() + equals('TEST', result_text) +``` +More convenient way is to use mock_input or mock_print for simple and most common cases. +From code above we can test our_weird_function this way +```python +@test +def check_input(): + with mock_input(['test']): # Now input() just returns 'test', it does not wait for user input. + result_text = our_weird_function_with_input_inside() + equals('TEST', result_text) +``` +Now let's say we have simple function with print inside and need to test it: +```python +def my_print(x): + print(x) +@test +def check_print(): + with mock_print([]) as result: # now print just collects all to list result + my_print(1) + my_print('1') + equals([(1,), ('1',)], result) # checks all args are in result list +``` +and more complicated case, when our function works forever, printing all inputs, until gets 'exit': +```python +def use_both(): + while True: + word = input('text>>>') + if word == 'exit': + break + print(word) +@test +def check_print_and_input(): + # you can see inputs will get 'a','b' and 'exit' to break cycle, all args will + # be collected to result list + with mock_input(['a', 'b', 'exit']), mock_print([]) as result: + use_both() + equals([('a',), ('b',)], result) +``` +**2. Fake function of the 3-d party library** +For working with other modules and libraries in test module, you need to import this module and to mock it function. +For example, you need to test function, which is using requests.get inside, but you do not want to make real http +request. Let it mock +some_module_to_test.py +```python +import requests +def func_with_get_inside(url): + response = requests.get(url) + return response.text +``` +our_tests.py +```python +import requests # need to import it for mock! +from some_module_to_test import func_with_get_inside +@test +def mock_requests_get(): + stub = Stub(text='test') # create simple stub, with attribute text equals to 'test' + with mock(requests, 'get', lambda x: stub): # Mock real requests with stub object + equals('test', func_with_get_inside('https://yandex.ru')) # Now no real requests be performed! +``` +**3. Mock read/write to file** +If you need to mock open function, push data to read from file and gets back with write to file, you can use +mock_open context-manager +```python +def my_open(): + # We read from one file, uppercase results and write to another file + with open('my_file.txt', encoding='utf-8') as f, open('another.txt', 'wt') as f2: + f2.write(f.readline().upper()) +@test +def mock_open_both(): + # Here we specify what we must "read from file" ('test') and where we want to get all writes(result) + with mock_open(on_read_text='test') as result: + my_open() + equals(['TEST'], result) # checks we get test uppercase +``` +**4. Spy object** +Spy is the object which has all attributes of original, but spy not performed any action, +all methods return None (if not specified what to return). Therefore, spy log all actions and arguments. +It can be useful if your code has inner object, and you need to test what functions were called. +```python +def function_with_str_inside(value): + # Suppose we need to check upper was called here inside + return value.upper() +@test +def spy_for_str(): + spy = Spy('it is a string') # Spy, which is like str, but it is not str! + function_with_str_inside(spy) # Send our spy instead a str + is_true(spy.upper.was_called()) # Verify upper was called +``` +You can even specify what to return when some function of the spy will be called! +```python +def function_with_str_inside(value): + # Suppose we need to check upper was called here inside + return value.upper() +@test +def spy_with_return(): + spy = Spy('string') + spy.upper.returns('test') # Tells what to return, when upper will be call + result = function_with_str_inside(spy) + is_true(spy.upper.was_called()) + equals('test', result) # verify our spy returns 'test' +``` +Spy object can be created without original inner object and can be call itself, it can be useful when you need +some dumb object to know it was called. +```python +@test +def check_spy(): + spy = Spy() # Create "empty" spy + spy() # Call it + is_true(spy.was_called()) # Checks spy was called +``` +**5. TestDouble object** +Test-Double object is like the Spy, but it saves original object behaviour, so its methods returns +real object methods results if not specified otherwise. +```python +@test +def check_double(): + spy = TestDouble("string") # Create str double-object + equals(6, len(spy)) # Len returns 6 - the real length of original object ("string") + spy.len.returns(100) # Fake len result + equals(100, len(spy)) # Len now returns 100 +``` +**Important!** Both spy and TestDouble override **isinstance**, so they emulate type of the original object. It can be useful for +testing functions, which has isinstance check inside. For example: +```python +def function_that_checks_class(obj): + if isinstance(obj, str): # check for argument type (string) + return "OK" + return "Not OK" +@test +def isinstance_check(): + spy = Spy("fake string") # fake the real string + result = function_that_checks_class(spy) # get "OK" here, cause function thinks it's a string, not Spy + equals("OK", result) +``` +**6. Stub object** +Stub object is just a helper for testing, its purpose not to check or assert something, but to give data +and perform some simple action, when application under test need it. Unlike spy or double, Stub +is not remember calls, it just a simple replacement for some object with minimum or no logic inside. +Let's say we have a function which gets some object, take its attribute, calculates something and +return result. We wish to isolate our testing from real objects, just test important behaviour, besides +this data-object can be hard to create or complicated. +```python +from checking import * +# Our function to test, it get some object and use it attribute and method, but we just +# need to test how it works! +def function(some_object)->int: + initial_value = some_object.value + result = 2 + some_object.complicate_function()*initial_value # Some calculation we need to test + return result +@test +def check_with_stub(): + stub = Stub(value=2) # Creates stub with attribute value=2 + stub.complicate_function.returns(2) # Says, when complicate_function will be called returns 2 + equals(6, function(stub)) # Asserts 6 == 2+(2*2) +``` +Pay attention - when you look for some attribute in stub - it always has it! But it will be a wrapper to use with +expression like `stub.any_attribute.returns('test')`. +So, if you need to have some attribute (not method) on stub, you just use `stub.attr=10`, but for methods just use expression above. +### Function start() to runs test at module ### +You can execute all test at current module using function start(). For example: +```python +from checking import * +@test +def some_check(): + equals(4, 2+2) +if __name__ == '__main__': + start(3) # Here we run our test function some_check +``` +There are parameters to run your tests in different ways: +**suite_name** - name of the test-suite, to use in reports or in logs +**listener** - object of Listener class, test listener, is the way to work with test results and execution +DefaultListener is used by default. If set, then the verbose parameter is ignored (the one in the listener is used). +**verbose** is the report detail, 0 - briefly (only dots and 1 letter), 1 - detail, indicating only failed +tests, 2 - detail, indicating successful and fallen tests, 3 - detail and at the end, a list of fallen and broken ones +If verbose is not between 0 and 3, then 0 is accepted +Example (name and verbose) +```python +from checking import * +@test +def some_check(): + equals(4, 2 + 2) +@test +def some_check_two(): + equals(2, 1 + 1) +@test +def failed(): + equals(5, 2 + 2) # Will fail +@test +def broken(): + int('a') # Will be broken +if __name__ == '__main__': + start(suite_name='My Suite', verbose=0) +``` +This code will gave output (mention dots and chars!): +```text + +%package help +Summary: Development documents and examples for checking +Provides: python3-checking-doc +%description help +Test "__main__.check_cat" [Cat from 140288585437776] SUCCESS! +``` +If you want to use a text file as a data source, you can use `DATA_FILE` helper function to skip the file handling boilerplate code: +```python +from checking import * +DATA_FILE('files/data.txt', name='provider') # Use the file located at <module folder>/files/data.txt +@test(data_provider='provider') +def try_prov(it): + print(it) + is_true(it) +``` +The helper lazy-loads specified data file line by line. +Raises FileNotFoundError if the file is not found. +Also, you can transform all the lines before feeding them into the test, +for example delete trailing newlines at the end of each line: +```python +from checking import * +DATA_FILE('files/data.txt', name='provider', map_function=str.rstrip) # Feed each line through str.rstrip() +@test(data_provider='provider') +def try_prov(it): + is_true(it) +``` +If you don't specify provider_name for the DATA_FILE helper, file_path will be used: +```python +from checking import * +DATA_FILE('data.txt') # Use text file located at the module folder. Note, that no provider_name is specified. +@test(data_provider='data.txt') # Use the specified file_name parameter for provider lookup +def try_prov(it): + is_true(it) +``` +If your test suite uses a data provider more than once, you might want to avoid the IO overhead, +if this provider fetches the data from some external source (database, file system, http request etc.). +You can use the `cached` parameter to force the provider to fetch the data only once and store it into memory. +Please, be varied of the memory consumption, because the cache persists until the whole suite is done running. +Also, be careful when using the cache when running tests in parallel. +DATA_FILE helper can use this parameter too. +```python +from checking import * +DATA_FILE('data.csv', name='csv', cached=True) # Enable caching +@test(data_provider='csv') # First provider use -- data is fetched from the file and stored into memory +def check_one(it): + not_none(it) +@test(data_provider='csv') # Second use -- no file reads, cached data is used +def check_two(it): + not_none(it) +if __name__ == '__main__': + start(0) +``` +If your provider is a simple one-liner (string, list comprehension, generator expression, etc.), +you can use the CONTAINER helper function to avoid full function definition boilerplate: +```python +from checking import * +CONTAINER([e for e in range(10)], name='range') # Provide data from a listcomps, set provider name to 'range' +@test(data_provider='range') +def try_container(it): + is_true(it in range(10)) +``` +'name' parameter is optional, 'container' is used by default, +but it's strongly recommended using a unique name: +```python +from checking import * +CONTAINER((e for e in range(10))) # Provide data from a genexps +@test(data_provider='container') +def try_container(it): + is_true(it in range(10)) +``` +**Important!** You must define DATA_FILE or CONTAINER providers at the module scope, not in the fixtures and tests. +### Test Parameters ### +You can manage the test execution mode by passing a number of parameters to the @test decorator: +**enabled** (bool) - if set to False, the test will be skipped, all other parameters are ignored. By default, set to True. +**name** (str) - the name of the test. Is bound to the decorated function name if not specified. +**description** (str) - test description. If absent, the test function docstring is used. +If both description and docstring are present, description takes precedence. +**data_provider** (str) - the name of the data provider to use with the test. +If specified, the test function must take one argument to be fed with the data from the provider. +Raises UnknownProviderName if no providers with the specified name found. +**retries** (int) - the number of times to run the failing test. If test does not fail, no more runs attempted. By default, set to 1. +**groups** (Tuple[str]) - a tuple of strings, representing the test group names a test is a part of. +All tests belong to some test group, the default group holds all tests from the current module and is named after the module. +Use this parameter to manage test execution groups. +**priority** (int) - test priority. The higher the value the later the test will be executed. +Use this parameter to fine tune test run order. By default, set to 0. +**timeout** (int) - amount of time to wait for the test to end. +If the time runs out, the thread running the test is terminated and the test is marked as "broken". +Use sparingly due to potential memory leaks. +**only_if** (Callable[None, bool]) - boolean predicate, which is evaluated before the test execution. +The test will be executed only if the predicate evaluates to True. +Use this parameter for conditional test execution e.g. run only if the OS is Linux, etc. +## Fixtures +Each test group or all test-suite can have preconditions and post-actions. For example, open DB connection before test starts and close it after that. +You can easily make it with before/after fixtures. The function that marked with before/after should be without arguments. +@before - run function before EACH test in group, by default group is current module, but you can specify it with parameter +@after - run function after EACH test in group, by default group is current module, but you can specify it with parameter. +This function will not be run if there is @before and it failed! +```python +@before(group_name='api') +def my_func(): + do_some_precondition() +@after(group_name='api') +def another_func(): + do_post_actions() +``` +@before_group - function run once before running test in group, by default group is current module, but you can specify it with parameter. +@after_group - function run once after running all test in group, by default group is current module, but you can specify it with parameter. +This function will not be run if there is @before_group and it failed, except using parameter always_run = True +```python +@before_group(name='api') +def my_func(): + do_some_precondition_for_whole_group() +@after_group(name='api', always_run =True) +def another_func(): + do_post_actions_for_whole_group() +``` +@before_suite - function runs once before any group at start of the test-suite +@after_suite - function run once after all groups, at the end of the test-suite. +This function will not be run if there is @before_suite, and it failed, except using parameter 'always_run = True' +```python +@before_suite +def my_func(): + print('start suite!') +@after_suite(always_run=True) +def another_func(): + print('will be printed, even if before_suite failed!') +``` +## Mock, Double, Stub and Spy +For testing purposes you sometimes need to fake some behaviour or to isolate your application from any other classes/libraries etc. +If you need your test to use fake object, without doing any real calls, you can use mocks: +**1. Fake one of the builtin function.** +Let say you need to test function which is using standard input() inside. +But you cannot wait for real user input during the test, so fake it with mock object. +```python +def our_weird_function_with_input_inside(): + text = input() + return text.upper() +@test +def mock_builtins_input(): + with mock_builtins('input', lambda : 'test'): # Now input() just returns 'test', it does not wait for user input. + result_text = our_weird_function_with_input_inside() + equals('TEST', result_text) +``` +More convenient way is to use mock_input or mock_print for simple and most common cases. +From code above we can test our_weird_function this way +```python +@test +def check_input(): + with mock_input(['test']): # Now input() just returns 'test', it does not wait for user input. + result_text = our_weird_function_with_input_inside() + equals('TEST', result_text) +``` +Now let's say we have simple function with print inside and need to test it: +```python +def my_print(x): + print(x) +@test +def check_print(): + with mock_print([]) as result: # now print just collects all to list result + my_print(1) + my_print('1') + equals([(1,), ('1',)], result) # checks all args are in result list +``` +and more complicated case, when our function works forever, printing all inputs, until gets 'exit': +```python +def use_both(): + while True: + word = input('text>>>') + if word == 'exit': + break + print(word) +@test +def check_print_and_input(): + # you can see inputs will get 'a','b' and 'exit' to break cycle, all args will + # be collected to result list + with mock_input(['a', 'b', 'exit']), mock_print([]) as result: + use_both() + equals([('a',), ('b',)], result) +``` +**2. Fake function of the 3-d party library** +For working with other modules and libraries in test module, you need to import this module and to mock it function. +For example, you need to test function, which is using requests.get inside, but you do not want to make real http +request. Let it mock +some_module_to_test.py +```python +import requests +def func_with_get_inside(url): + response = requests.get(url) + return response.text +``` +our_tests.py +```python +import requests # need to import it for mock! +from some_module_to_test import func_with_get_inside +@test +def mock_requests_get(): + stub = Stub(text='test') # create simple stub, with attribute text equals to 'test' + with mock(requests, 'get', lambda x: stub): # Mock real requests with stub object + equals('test', func_with_get_inside('https://yandex.ru')) # Now no real requests be performed! +``` +**3. Mock read/write to file** +If you need to mock open function, push data to read from file and gets back with write to file, you can use +mock_open context-manager +```python +def my_open(): + # We read from one file, uppercase results and write to another file + with open('my_file.txt', encoding='utf-8') as f, open('another.txt', 'wt') as f2: + f2.write(f.readline().upper()) +@test +def mock_open_both(): + # Here we specify what we must "read from file" ('test') and where we want to get all writes(result) + with mock_open(on_read_text='test') as result: + my_open() + equals(['TEST'], result) # checks we get test uppercase +``` +**4. Spy object** +Spy is the object which has all attributes of original, but spy not performed any action, +all methods return None (if not specified what to return). Therefore, spy log all actions and arguments. +It can be useful if your code has inner object, and you need to test what functions were called. +```python +def function_with_str_inside(value): + # Suppose we need to check upper was called here inside + return value.upper() +@test +def spy_for_str(): + spy = Spy('it is a string') # Spy, which is like str, but it is not str! + function_with_str_inside(spy) # Send our spy instead a str + is_true(spy.upper.was_called()) # Verify upper was called +``` +You can even specify what to return when some function of the spy will be called! +```python +def function_with_str_inside(value): + # Suppose we need to check upper was called here inside + return value.upper() +@test +def spy_with_return(): + spy = Spy('string') + spy.upper.returns('test') # Tells what to return, when upper will be call + result = function_with_str_inside(spy) + is_true(spy.upper.was_called()) + equals('test', result) # verify our spy returns 'test' +``` +Spy object can be created without original inner object and can be call itself, it can be useful when you need +some dumb object to know it was called. +```python +@test +def check_spy(): + spy = Spy() # Create "empty" spy + spy() # Call it + is_true(spy.was_called()) # Checks spy was called +``` +**5. TestDouble object** +Test-Double object is like the Spy, but it saves original object behaviour, so its methods returns +real object methods results if not specified otherwise. +```python +@test +def check_double(): + spy = TestDouble("string") # Create str double-object + equals(6, len(spy)) # Len returns 6 - the real length of original object ("string") + spy.len.returns(100) # Fake len result + equals(100, len(spy)) # Len now returns 100 +``` +**Important!** Both spy and TestDouble override **isinstance**, so they emulate type of the original object. It can be useful for +testing functions, which has isinstance check inside. For example: +```python +def function_that_checks_class(obj): + if isinstance(obj, str): # check for argument type (string) + return "OK" + return "Not OK" +@test +def isinstance_check(): + spy = Spy("fake string") # fake the real string + result = function_that_checks_class(spy) # get "OK" here, cause function thinks it's a string, not Spy + equals("OK", result) +``` +**6. Stub object** +Stub object is just a helper for testing, its purpose not to check or assert something, but to give data +and perform some simple action, when application under test need it. Unlike spy or double, Stub +is not remember calls, it just a simple replacement for some object with minimum or no logic inside. +Let's say we have a function which gets some object, take its attribute, calculates something and +return result. We wish to isolate our testing from real objects, just test important behaviour, besides +this data-object can be hard to create or complicated. +```python +from checking import * +# Our function to test, it get some object and use it attribute and method, but we just +# need to test how it works! +def function(some_object)->int: + initial_value = some_object.value + result = 2 + some_object.complicate_function()*initial_value # Some calculation we need to test + return result +@test +def check_with_stub(): + stub = Stub(value=2) # Creates stub with attribute value=2 + stub.complicate_function.returns(2) # Says, when complicate_function will be called returns 2 + equals(6, function(stub)) # Asserts 6 == 2+(2*2) +``` +Pay attention - when you look for some attribute in stub - it always has it! But it will be a wrapper to use with +expression like `stub.any_attribute.returns('test')`. +So, if you need to have some attribute (not method) on stub, you just use `stub.attr=10`, but for methods just use expression above. +### Function start() to runs test at module ### +You can execute all test at current module using function start(). For example: +```python +from checking import * +@test +def some_check(): + equals(4, 2+2) +if __name__ == '__main__': + start(3) # Here we run our test function some_check +``` +There are parameters to run your tests in different ways: +**suite_name** - name of the test-suite, to use in reports or in logs +**listener** - object of Listener class, test listener, is the way to work with test results and execution +DefaultListener is used by default. If set, then the verbose parameter is ignored (the one in the listener is used). +**verbose** is the report detail, 0 - briefly (only dots and 1 letter), 1 - detail, indicating only failed +tests, 2 - detail, indicating successful and fallen tests, 3 - detail and at the end, a list of fallen and broken ones +If verbose is not between 0 and 3, then 0 is accepted +Example (name and verbose) +```python +from checking import * +@test +def some_check(): + equals(4, 2 + 2) +@test +def some_check_two(): + equals(2, 1 + 1) +@test +def failed(): + equals(5, 2 + 2) # Will fail +@test +def broken(): + int('a') # Will be broken +if __name__ == '__main__': + start(suite_name='My Suite', verbose=0) +``` +This code will gave output (mention dots and chars!): +```text + +%prep +%autosetup -n checking-0.9.1 + +%build +%py3_build + +%install +%py3_install +install -d -m755 %{buildroot}/%{_pkgdocdir} +if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi +if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi +if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi +if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi +pushd %{buildroot} +if [ -d usr/lib ]; then + find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/lib64 ]; then + find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/bin ]; then + find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/sbin ]; then + find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst +fi +touch doclist.lst +if [ -d usr/share/man ]; then + find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst +fi +popd +mv %{buildroot}/filelist.lst . +mv %{buildroot}/doclist.lst . + +%files -n python3-checking -f filelist.lst +%dir %{python3_sitelib}/* + +%files help -f doclist.lst +%{_docdir}/* + +%changelog +* Thu May 18 2023 Python_Bot <Python_Bot@openeuler.org> - 0.9.1-1 +- Package Spec generated |