%global _empty_manifest_terminate_build 0 Name: python-tempeh Version: 0.1.12 Release: 1 Summary: Machine Learning Performance Testing Framework License: MIT License URL: https://github.com/microsoft/tempeh Source0: https://mirrors.nju.edu.cn/pypi/web/packages/76/1b/6cec6ab29489a619141fa0312da7157acebdf2cf9ffe37ecfd4a61365ea1/tempeh-0.1.12.tar.gz BuildArch: noarch Requires: python3-memory-profiler Requires: python3-numpy Requires: python3-pandas Requires: python3-pytest Requires: python3-requests Requires: python3-scipy Requires: python3-shap Requires: python3-scikit-learn %description [![Build Status](https://img.shields.io/azure-devops/build/responsibleai/tempeh/19/master?failed_label=bad&passed_label=good&label=GatedCheckin%3ADev)](https://dev.azure.com/responsibleai/tempeh/_build/latest?definitionId=19&branchName=master) ![MIT license](https://img.shields.io/badge/License-MIT-blue.svg) ![pypi badge](https://img.shields.io/pypi/v/tempeh?color=blue) # tempeh tempeh is a framework to **TE**st **M**achine learning **PE**rformance ex**H**austively which includes tracking memory usage and run time. This is particularly useful as a pluggable tool for your repository's performance tests. Typically, people want to run them periodically over various datasets and/or with a number of models to catch regressions with respect to run time or memory consumption. This should be as easy as ```python import pytest from time import time from tempeh.configurations import datasets, models @pytest.mark.parametrize('Dataset', datasets.values()) @pytest.mark.parametrize('Model', models.values()) def test_fit_predict_regression(Dataset, Model): dataset = Dataset() X_train, X_test = dataset.get_X() y_train, y_test = dataset.get_y() model = Model() max_execution_time = get_max_execution_time(dataset, model) if model.compatible_with_dataset(dataset): start_time = time() model.fit(X_train, y_train) model.predict(X_test) duration = time() - start_time assert duration < max_execution_time ``` ## Installation tempeh depends on various packages to provide models, including `tensorflow`, `torch`, `xgboost`, `lightgbm`. To install a release version of `tempeh` just run ```python pip install tempeh ```
Common issues - If you're using a 32-bit Python version you might need to switch to a 64-bit Python version first to successfully install tensorflow. - If the installation of `torch` fails try using the recommendation from the [pytorch website](https://pytorch.org/get-started/locally/) for stable versions without CUDA for your python version on your operating system. - If the installation of `lightgbm` or `xgboost` fails try to use a pip version less than 20.0 until their bug is resolved.
## Structure ### Datasets Datasets (located in the `datasets/` directory) encapsulate different datasets used for testing. #### To add a new one + Create a python file in the `datasets/` directory with naming convention `[name]_datasets.py` + Subclass `BasePerformanceDatasetWrapper`. The naming convention is `[dataset_name]PerformanceDatasetWrapper` + In `__init__` load the dataset and call `super().__init__(data, targets, size)` + Add the class to `__init__.py` + Make sure the class contains class variables `task`, `data_type`, `size` + Add an entry to the `datasets` dictionary in `configurations.py`. ### Models Models (`models/` directory) wrap different machine learning models. #### To add a new one + Create a python file in the `models/` directory with naming convention `[name]_model.py` + Subclass `BaseModelWrapper` and name the class `[name]ModelWrapper` + In `__init__` train the model; we expect format `__init__(self, ...)` + Models must contain `tasks` and `algorithm` + Add an entry to the `models` dictionary in `configurations.py`. ## Maintainers In alphabetical order: - [Eduardo de Leon](https://github.com/eedeleon) - [Ilya Matiach](https://github.com/imatiach-msft) - [Roman Lutz](https://github.com/romanlutz) # Contributing To contribute please check our [Contributing Guide](CONTRIBUTING.md). # Issues ## Regular (non-Security) Issues Please submit a report through [Github issues](https://github.com/microsoft/tempeh/issues). A maintainer will respond within a reasonable period of time to handle the issue as follows: - bug: triage as `bug` and provide estimated timeline based on severity - feature request: triage as `feature request` and provide estimated timeline - question or discussion: triage as `question` and respond or notify/identify a suitable expert to respond Maintainers are supposed to link duplicate issues when possible. ## Reporting Security Issues Please take a look at our guidelines for reporting [security issues](SECURITY.md). %package -n python3-tempeh Summary: Machine Learning Performance Testing Framework Provides: python-tempeh BuildRequires: python3-devel BuildRequires: python3-setuptools BuildRequires: python3-pip %description -n python3-tempeh [![Build Status](https://img.shields.io/azure-devops/build/responsibleai/tempeh/19/master?failed_label=bad&passed_label=good&label=GatedCheckin%3ADev)](https://dev.azure.com/responsibleai/tempeh/_build/latest?definitionId=19&branchName=master) ![MIT license](https://img.shields.io/badge/License-MIT-blue.svg) ![pypi badge](https://img.shields.io/pypi/v/tempeh?color=blue) # tempeh tempeh is a framework to **TE**st **M**achine learning **PE**rformance ex**H**austively which includes tracking memory usage and run time. This is particularly useful as a pluggable tool for your repository's performance tests. Typically, people want to run them periodically over various datasets and/or with a number of models to catch regressions with respect to run time or memory consumption. This should be as easy as ```python import pytest from time import time from tempeh.configurations import datasets, models @pytest.mark.parametrize('Dataset', datasets.values()) @pytest.mark.parametrize('Model', models.values()) def test_fit_predict_regression(Dataset, Model): dataset = Dataset() X_train, X_test = dataset.get_X() y_train, y_test = dataset.get_y() model = Model() max_execution_time = get_max_execution_time(dataset, model) if model.compatible_with_dataset(dataset): start_time = time() model.fit(X_train, y_train) model.predict(X_test) duration = time() - start_time assert duration < max_execution_time ``` ## Installation tempeh depends on various packages to provide models, including `tensorflow`, `torch`, `xgboost`, `lightgbm`. To install a release version of `tempeh` just run ```python pip install tempeh ```
Common issues - If you're using a 32-bit Python version you might need to switch to a 64-bit Python version first to successfully install tensorflow. - If the installation of `torch` fails try using the recommendation from the [pytorch website](https://pytorch.org/get-started/locally/) for stable versions without CUDA for your python version on your operating system. - If the installation of `lightgbm` or `xgboost` fails try to use a pip version less than 20.0 until their bug is resolved.
## Structure ### Datasets Datasets (located in the `datasets/` directory) encapsulate different datasets used for testing. #### To add a new one + Create a python file in the `datasets/` directory with naming convention `[name]_datasets.py` + Subclass `BasePerformanceDatasetWrapper`. The naming convention is `[dataset_name]PerformanceDatasetWrapper` + In `__init__` load the dataset and call `super().__init__(data, targets, size)` + Add the class to `__init__.py` + Make sure the class contains class variables `task`, `data_type`, `size` + Add an entry to the `datasets` dictionary in `configurations.py`. ### Models Models (`models/` directory) wrap different machine learning models. #### To add a new one + Create a python file in the `models/` directory with naming convention `[name]_model.py` + Subclass `BaseModelWrapper` and name the class `[name]ModelWrapper` + In `__init__` train the model; we expect format `__init__(self, ...)` + Models must contain `tasks` and `algorithm` + Add an entry to the `models` dictionary in `configurations.py`. ## Maintainers In alphabetical order: - [Eduardo de Leon](https://github.com/eedeleon) - [Ilya Matiach](https://github.com/imatiach-msft) - [Roman Lutz](https://github.com/romanlutz) # Contributing To contribute please check our [Contributing Guide](CONTRIBUTING.md). # Issues ## Regular (non-Security) Issues Please submit a report through [Github issues](https://github.com/microsoft/tempeh/issues). A maintainer will respond within a reasonable period of time to handle the issue as follows: - bug: triage as `bug` and provide estimated timeline based on severity - feature request: triage as `feature request` and provide estimated timeline - question or discussion: triage as `question` and respond or notify/identify a suitable expert to respond Maintainers are supposed to link duplicate issues when possible. ## Reporting Security Issues Please take a look at our guidelines for reporting [security issues](SECURITY.md). %package help Summary: Development documents and examples for tempeh Provides: python3-tempeh-doc %description help [![Build Status](https://img.shields.io/azure-devops/build/responsibleai/tempeh/19/master?failed_label=bad&passed_label=good&label=GatedCheckin%3ADev)](https://dev.azure.com/responsibleai/tempeh/_build/latest?definitionId=19&branchName=master) ![MIT license](https://img.shields.io/badge/License-MIT-blue.svg) ![pypi badge](https://img.shields.io/pypi/v/tempeh?color=blue) # tempeh tempeh is a framework to **TE**st **M**achine learning **PE**rformance ex**H**austively which includes tracking memory usage and run time. This is particularly useful as a pluggable tool for your repository's performance tests. Typically, people want to run them periodically over various datasets and/or with a number of models to catch regressions with respect to run time or memory consumption. This should be as easy as ```python import pytest from time import time from tempeh.configurations import datasets, models @pytest.mark.parametrize('Dataset', datasets.values()) @pytest.mark.parametrize('Model', models.values()) def test_fit_predict_regression(Dataset, Model): dataset = Dataset() X_train, X_test = dataset.get_X() y_train, y_test = dataset.get_y() model = Model() max_execution_time = get_max_execution_time(dataset, model) if model.compatible_with_dataset(dataset): start_time = time() model.fit(X_train, y_train) model.predict(X_test) duration = time() - start_time assert duration < max_execution_time ``` ## Installation tempeh depends on various packages to provide models, including `tensorflow`, `torch`, `xgboost`, `lightgbm`. To install a release version of `tempeh` just run ```python pip install tempeh ```
Common issues - If you're using a 32-bit Python version you might need to switch to a 64-bit Python version first to successfully install tensorflow. - If the installation of `torch` fails try using the recommendation from the [pytorch website](https://pytorch.org/get-started/locally/) for stable versions without CUDA for your python version on your operating system. - If the installation of `lightgbm` or `xgboost` fails try to use a pip version less than 20.0 until their bug is resolved.
## Structure ### Datasets Datasets (located in the `datasets/` directory) encapsulate different datasets used for testing. #### To add a new one + Create a python file in the `datasets/` directory with naming convention `[name]_datasets.py` + Subclass `BasePerformanceDatasetWrapper`. The naming convention is `[dataset_name]PerformanceDatasetWrapper` + In `__init__` load the dataset and call `super().__init__(data, targets, size)` + Add the class to `__init__.py` + Make sure the class contains class variables `task`, `data_type`, `size` + Add an entry to the `datasets` dictionary in `configurations.py`. ### Models Models (`models/` directory) wrap different machine learning models. #### To add a new one + Create a python file in the `models/` directory with naming convention `[name]_model.py` + Subclass `BaseModelWrapper` and name the class `[name]ModelWrapper` + In `__init__` train the model; we expect format `__init__(self, ...)` + Models must contain `tasks` and `algorithm` + Add an entry to the `models` dictionary in `configurations.py`. ## Maintainers In alphabetical order: - [Eduardo de Leon](https://github.com/eedeleon) - [Ilya Matiach](https://github.com/imatiach-msft) - [Roman Lutz](https://github.com/romanlutz) # Contributing To contribute please check our [Contributing Guide](CONTRIBUTING.md). # Issues ## Regular (non-Security) Issues Please submit a report through [Github issues](https://github.com/microsoft/tempeh/issues). A maintainer will respond within a reasonable period of time to handle the issue as follows: - bug: triage as `bug` and provide estimated timeline based on severity - feature request: triage as `feature request` and provide estimated timeline - question or discussion: triage as `question` and respond or notify/identify a suitable expert to respond Maintainers are supposed to link duplicate issues when possible. ## Reporting Security Issues Please take a look at our guidelines for reporting [security issues](SECURITY.md). %prep %autosetup -n tempeh-0.1.12 %build %py3_build %install %py3_install install -d -m755 %{buildroot}/%{_pkgdocdir} if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi pushd %{buildroot} if [ -d usr/lib ]; then find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/lib64 ]; then find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/bin ]; then find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/sbin ]; then find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst fi touch doclist.lst if [ -d usr/share/man ]; then find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst fi popd mv %{buildroot}/filelist.lst . mv %{buildroot}/doclist.lst . %files -n python3-tempeh -f filelist.lst %dir %{python3_sitelib}/* %files help -f doclist.lst %{_docdir}/* %changelog * Wed May 10 2023 Python_Bot - 0.1.12-1 - Package Spec generated