diff options
| -rw-r--r-- | .gitignore | 1 | ||||
| -rw-r--r-- | python-flaky-tests-detection.spec | 315 | ||||
| -rw-r--r-- | sources | 1 |
3 files changed, 317 insertions, 0 deletions
@@ -0,0 +1 @@ +/flaky-tests-detection-1.3.0.tar.gz diff --git a/python-flaky-tests-detection.spec b/python-flaky-tests-detection.spec new file mode 100644 index 0000000..22c1e3c --- /dev/null +++ b/python-flaky-tests-detection.spec @@ -0,0 +1,315 @@ +%global _empty_manifest_terminate_build 0 +Name: python-flaky-tests-detection +Version: 1.3.0 +Release: 1 +Summary: Github actions plugin to check flakiness of tests by calculating fliprates. +License: Apache License 2.0 +URL: https://github.com/F-Secure/flaky-tests-detection +Source0: https://mirrors.nju.edu.cn/pypi/web/packages/de/33/f6e1d27d107416507ba2259b290969eb5fe90650b6f5a19f5fae5147e708/flaky-tests-detection-1.3.0.tar.gz +BuildArch: noarch + +Requires: python3-pandas +Requires: python3-junitparser +Requires: python3-seaborn +Requires: python3-matplotlib +Requires: python3-pytest +Requires: python3-pytest-cov +Requires: python3-black +Requires: python3-mypy +Requires: python3-semantic-release + +%description +# Flaky tests detection + +<center><img src="https://github.com/F-Secure/flaky-tests-detection/blob/master/docs/imgs/Flaky-test-detection.png" width="120px" height="120px"/></center> + +Visualize tests whose state changes most often. During software development, it is often common that some tests start to randomly fail, +but finding those tests is a tedious and time consuming problem. Flaky tests detection solves that problem by processing historical xunit test +results and checks which tests changes state most often. Flaky tests detection is available as Github Action plugin and +[Python package](https://pypi.org/project/flaky-tests-detection/). For usage, see [example](https://github.com/F-Secure/flaky-test-ci) +at the `actions` page. + +Implementation is based on ["Modeling and ranking flaky tests at Apple"](https://dl.acm.org/doi/10.1145/3377813.3381370) by Kowalczyk, Emily & Nair, Karan & Gao, Zebao & Silberstein, Leo & Long, Teng & Memon, Atif. + +## Features + +* Prints out top test names and their latest calculation window scores (normal fliprate and exponentially weighted moving average fliprate that take previous calculation windows into account). +* Calculation grouping options: + * `n` days. + * `n` runs. +* Heatmap visualization of the scores and history. + +## Parameters + +### Data options (choose one) + +* `--test-history-csv` + * Give a path to a test history csv file which includes three fields: `timestamp`, `test_identifier` and `test_status`. +* `--junit-files` + * Give a path to a folder with `JUnit` test results. + +### Calculation options + +* `--grouping-option` + * `days` to use `n` days for fliprate calculation windows. + * `runs` to use `n` runs for fliprate calculation windows. + +* `--window-size` + * Fliprate calculation window size `n`. + +* `--window-count` + * History size for exponentially weighted moving average calculations. + +* `--top-n` + * How many top highest scoring tests to print out. +### Heatmap generation +* `--heatmap` + * Turn heatmap generation on. + * Two pictures generated: normal fliprate and exponentially weighted moving average fliprate score. + * Same parameters used as with the printed statistics. + +### Full examples + +* Precomputed `test_history.csv` with daily calulations. 1 day windows, 7 day history and 5 tests printed out. + * `--test-history-csv=example_history/test_history.csv --grouping-option=days --window-size=1 --window-count=7 --top-n=5` +* `JUnit` files with calculations per 5 runs. 15 runs history and 5 tests printed out. + * `--junit-files=example_history/junit_files --grouping-option=runs --window-size=5 --window-count=3 --top-n=5` +* Precomputed `test_history.csv` with daily calculations and heatmap generation. 1 day windows, 7 day history and 50 tests printed and generated to heatmaps. + * `--test-history-csv=example_history/test_history.csv --grouping-option=days --window-size=1 --window-count=7 --top-n=50 --heatmap` + +## Install module + +* `make install` + +## Install module and development packages + +* `make install_dev` + +## Run pytest + +* `make run_test` + +## Acknowledgement + +The package was developed by [F-Secure Corporation][f-secure] and [University of Helsinki][hy] in the scope of [IVVES project][ivves]. This work was labelled by [ITEA3][itea3] and funded by local authorities under grant agreement “ITEA-2019-18022-IVVES” + +[ivves]: http://ivves.eu/ +[itea3]: https://itea3.org/ +[f-secure]: https://www.f-secure.com/en +[hy]: https://www.helsinki.fi/en/computer-science + + +%package -n python3-flaky-tests-detection +Summary: Github actions plugin to check flakiness of tests by calculating fliprates. +Provides: python-flaky-tests-detection +BuildRequires: python3-devel +BuildRequires: python3-setuptools +BuildRequires: python3-pip +%description -n python3-flaky-tests-detection +# Flaky tests detection + +<center><img src="https://github.com/F-Secure/flaky-tests-detection/blob/master/docs/imgs/Flaky-test-detection.png" width="120px" height="120px"/></center> + +Visualize tests whose state changes most often. During software development, it is often common that some tests start to randomly fail, +but finding those tests is a tedious and time consuming problem. Flaky tests detection solves that problem by processing historical xunit test +results and checks which tests changes state most often. Flaky tests detection is available as Github Action plugin and +[Python package](https://pypi.org/project/flaky-tests-detection/). For usage, see [example](https://github.com/F-Secure/flaky-test-ci) +at the `actions` page. + +Implementation is based on ["Modeling and ranking flaky tests at Apple"](https://dl.acm.org/doi/10.1145/3377813.3381370) by Kowalczyk, Emily & Nair, Karan & Gao, Zebao & Silberstein, Leo & Long, Teng & Memon, Atif. + +## Features + +* Prints out top test names and their latest calculation window scores (normal fliprate and exponentially weighted moving average fliprate that take previous calculation windows into account). +* Calculation grouping options: + * `n` days. + * `n` runs. +* Heatmap visualization of the scores and history. + +## Parameters + +### Data options (choose one) + +* `--test-history-csv` + * Give a path to a test history csv file which includes three fields: `timestamp`, `test_identifier` and `test_status`. +* `--junit-files` + * Give a path to a folder with `JUnit` test results. + +### Calculation options + +* `--grouping-option` + * `days` to use `n` days for fliprate calculation windows. + * `runs` to use `n` runs for fliprate calculation windows. + +* `--window-size` + * Fliprate calculation window size `n`. + +* `--window-count` + * History size for exponentially weighted moving average calculations. + +* `--top-n` + * How many top highest scoring tests to print out. +### Heatmap generation +* `--heatmap` + * Turn heatmap generation on. + * Two pictures generated: normal fliprate and exponentially weighted moving average fliprate score. + * Same parameters used as with the printed statistics. + +### Full examples + +* Precomputed `test_history.csv` with daily calulations. 1 day windows, 7 day history and 5 tests printed out. + * `--test-history-csv=example_history/test_history.csv --grouping-option=days --window-size=1 --window-count=7 --top-n=5` +* `JUnit` files with calculations per 5 runs. 15 runs history and 5 tests printed out. + * `--junit-files=example_history/junit_files --grouping-option=runs --window-size=5 --window-count=3 --top-n=5` +* Precomputed `test_history.csv` with daily calculations and heatmap generation. 1 day windows, 7 day history and 50 tests printed and generated to heatmaps. + * `--test-history-csv=example_history/test_history.csv --grouping-option=days --window-size=1 --window-count=7 --top-n=50 --heatmap` + +## Install module + +* `make install` + +## Install module and development packages + +* `make install_dev` + +## Run pytest + +* `make run_test` + +## Acknowledgement + +The package was developed by [F-Secure Corporation][f-secure] and [University of Helsinki][hy] in the scope of [IVVES project][ivves]. This work was labelled by [ITEA3][itea3] and funded by local authorities under grant agreement “ITEA-2019-18022-IVVES” + +[ivves]: http://ivves.eu/ +[itea3]: https://itea3.org/ +[f-secure]: https://www.f-secure.com/en +[hy]: https://www.helsinki.fi/en/computer-science + + +%package help +Summary: Development documents and examples for flaky-tests-detection +Provides: python3-flaky-tests-detection-doc +%description help +# Flaky tests detection + +<center><img src="https://github.com/F-Secure/flaky-tests-detection/blob/master/docs/imgs/Flaky-test-detection.png" width="120px" height="120px"/></center> + +Visualize tests whose state changes most often. During software development, it is often common that some tests start to randomly fail, +but finding those tests is a tedious and time consuming problem. Flaky tests detection solves that problem by processing historical xunit test +results and checks which tests changes state most often. Flaky tests detection is available as Github Action plugin and +[Python package](https://pypi.org/project/flaky-tests-detection/). For usage, see [example](https://github.com/F-Secure/flaky-test-ci) +at the `actions` page. + +Implementation is based on ["Modeling and ranking flaky tests at Apple"](https://dl.acm.org/doi/10.1145/3377813.3381370) by Kowalczyk, Emily & Nair, Karan & Gao, Zebao & Silberstein, Leo & Long, Teng & Memon, Atif. + +## Features + +* Prints out top test names and their latest calculation window scores (normal fliprate and exponentially weighted moving average fliprate that take previous calculation windows into account). +* Calculation grouping options: + * `n` days. + * `n` runs. +* Heatmap visualization of the scores and history. + +## Parameters + +### Data options (choose one) + +* `--test-history-csv` + * Give a path to a test history csv file which includes three fields: `timestamp`, `test_identifier` and `test_status`. +* `--junit-files` + * Give a path to a folder with `JUnit` test results. + +### Calculation options + +* `--grouping-option` + * `days` to use `n` days for fliprate calculation windows. + * `runs` to use `n` runs for fliprate calculation windows. + +* `--window-size` + * Fliprate calculation window size `n`. + +* `--window-count` + * History size for exponentially weighted moving average calculations. + +* `--top-n` + * How many top highest scoring tests to print out. +### Heatmap generation +* `--heatmap` + * Turn heatmap generation on. + * Two pictures generated: normal fliprate and exponentially weighted moving average fliprate score. + * Same parameters used as with the printed statistics. + +### Full examples + +* Precomputed `test_history.csv` with daily calulations. 1 day windows, 7 day history and 5 tests printed out. + * `--test-history-csv=example_history/test_history.csv --grouping-option=days --window-size=1 --window-count=7 --top-n=5` +* `JUnit` files with calculations per 5 runs. 15 runs history and 5 tests printed out. + * `--junit-files=example_history/junit_files --grouping-option=runs --window-size=5 --window-count=3 --top-n=5` +* Precomputed `test_history.csv` with daily calculations and heatmap generation. 1 day windows, 7 day history and 50 tests printed and generated to heatmaps. + * `--test-history-csv=example_history/test_history.csv --grouping-option=days --window-size=1 --window-count=7 --top-n=50 --heatmap` + +## Install module + +* `make install` + +## Install module and development packages + +* `make install_dev` + +## Run pytest + +* `make run_test` + +## Acknowledgement + +The package was developed by [F-Secure Corporation][f-secure] and [University of Helsinki][hy] in the scope of [IVVES project][ivves]. This work was labelled by [ITEA3][itea3] and funded by local authorities under grant agreement “ITEA-2019-18022-IVVES” + +[ivves]: http://ivves.eu/ +[itea3]: https://itea3.org/ +[f-secure]: https://www.f-secure.com/en +[hy]: https://www.helsinki.fi/en/computer-science + + +%prep +%autosetup -n flaky-tests-detection-1.3.0 + +%build +%py3_build + +%install +%py3_install +install -d -m755 %{buildroot}/%{_pkgdocdir} +if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi +if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi +if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi +if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi +pushd %{buildroot} +if [ -d usr/lib ]; then + find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/lib64 ]; then + find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/bin ]; then + find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/sbin ]; then + find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst +fi +touch doclist.lst +if [ -d usr/share/man ]; then + find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst +fi +popd +mv %{buildroot}/filelist.lst . +mv %{buildroot}/doclist.lst . + +%files -n python3-flaky-tests-detection -f filelist.lst +%dir %{python3_sitelib}/* + +%files help -f doclist.lst +%{_docdir}/* + +%changelog +* Mon May 15 2023 Python_Bot <Python_Bot@openeuler.org> - 1.3.0-1 +- Package Spec generated @@ -0,0 +1 @@ +8b9fe0ee15252651dece77ebf8d0b311 flaky-tests-detection-1.3.0.tar.gz |