summaryrefslogtreecommitdiff
path: root/python-loky.spec
diff options
context:
space:
mode:
authorCoprDistGit <infra@openeuler.org>2023-04-11 08:00:31 +0000
committerCoprDistGit <infra@openeuler.org>2023-04-11 08:00:31 +0000
commit643bfbdf764b512506dac1066eb6c9581fde2266 (patch)
tree6ebdc94a27e0f2ef45f39fc0f8063cb77fc31aaa /python-loky.spec
parentfeb7af46600a84655103003d21d2b3bd0e20e6a2 (diff)
automatic import of python-loky
Diffstat (limited to 'python-loky.spec')
-rw-r--r--python-loky.spec514
1 files changed, 514 insertions, 0 deletions
diff --git a/python-loky.spec b/python-loky.spec
new file mode 100644
index 0000000..4a36821
--- /dev/null
+++ b/python-loky.spec
@@ -0,0 +1,514 @@
+%global _empty_manifest_terminate_build 0
+Name: python-loky
+Version: 3.3.0
+Release: 1
+Summary: A robust implementation of concurrent.futures.ProcessPoolExecutor
+License: BSD
+URL: https://github.com/joblib/loky/
+Source0: https://mirrors.nju.edu.cn/pypi/web/packages/35/0e/97520e1a9317637727fd05d00ae4d834eaf82b19ca7ef3e2dcedf4446656/loky-3.3.0.tar.gz
+BuildArch: noarch
+
+Requires: python3-cloudpickle
+
+%description
+
+<a href="https://loky.readthedocs.io">
+<img src="docs/_static/loky_logo.svg"
+alt="Loky logo" width=96/></a>
+
+
+# Reusable Process Pool Executor
+[![Build Status](https://dev.azure.com/joblib/loky/_apis/build/status/joblib.loky?branchName=master)](https://dev.azure.com/joblib/loky/_build/latest?definitionId=2&branchName=master)
+[![Documentation Status](https://readthedocs.org/projects/loky/badge/?version=latest)](https://loky.readthedocs.io/en/latest/?badge=latest)
+[![codecov](https://codecov.io/gh/joblib/loky/branch/master/graph/badge.svg)](https://codecov.io/gh/joblib/loky)
+[![DOI](https://zenodo.org/badge/48578152.svg)](https://zenodo.org/badge/latestdoi/48578152)
+
+
+### Goal
+
+The aim of this project is to provide a robust, cross-platform and
+cross-version implementation of the `ProcessPoolExecutor` class of
+`concurrent.futures`. It notably features:
+
+ * __Consistent and robust spawn behavior__: All processes are started
+ using fork + exec on POSIX systems. This ensures safer interactions with
+ third party libraries. On the contrary, `multiprocessing.Pool` uses
+ fork without exec by default, causing third party runtimes to crash
+ (e.g. OpenMP, macOS Accelerate...).
+
+ * __Reusable executor__: strategy to avoid re-spawning a complete
+ executor every time. A singleton executor instance can be reused (and
+ dynamically resized if necessary) across consecutive calls to limit
+ spawning and shutdown overhead. The worker processes can be shutdown
+ automatically after a configurable idling timeout to free system
+ resources.
+
+ * __Transparent cloudpickle integration__: to call interactively
+ defined functions and lambda expressions in parallel. It is also
+ possible to register a custom pickler implementation to handle
+ inter-process communications.
+
+ * __No need for ``if __name__ == "__main__":`` in scripts__: thanks
+ to the use of ``cloudpickle`` to call functions defined in the
+ ``__main__`` module, it is not required to protect the code calling
+ parallel functions under Windows.
+
+ * __Deadlock free implementation__: one of the major concern in
+ standard `multiprocessing` and `concurrent.futures` modules is the
+ ability of the `Pool/Executor` to handle crashes of worker
+ processes. This library intends to fix those possible deadlocks and
+ send back meaningful errors. Note that the implementation of
+ `concurrent.futures.ProcessPoolExecutor` that comes with Python 3.7+
+ is as robust as the executor from loky but the latter also works for
+ older versions of Python.
+
+
+### Installation
+
+The recommended way to install `loky` is with `pip`,
+```bash
+pip install loky
+```
+
+`loky` can also be installed from sources using
+```bash
+git clone https://github.com/joblib/loky
+cd loky
+python setup.py install
+```
+
+Note that `loky` has an optional dependency on [`psutil`][1] to allow early
+memory leak detections.
+
+### Usage
+
+The basic usage of `loky` relies on the `get_reusable_executor`, which
+internally manages a custom `ProcessPoolExecutor` object, which is reused or
+re-spawned depending on the context.
+
+```python
+import os
+from time import sleep
+from loky import get_reusable_executor
+
+
+def say_hello(k):
+ pid = os.getpid()
+ print(f"Hello from {pid} with arg {k}")
+ sleep(.01)
+ return pid
+
+
+# Create an executor with 4 worker processes, that will
+# automatically shutdown after idling for 2s
+executor = get_reusable_executor(max_workers=4, timeout=2)
+
+res = executor.submit(say_hello, 1)
+print("Got results:", res.result())
+
+results = executor.map(say_hello, range(50))
+n_workers = len(set(results))
+print("Number of used processes:", n_workers)
+assert n_workers == 4
+```
+
+For more advance usage, see our
+[documentation](https://loky.readthedocs.io/en/stable/)
+
+### Workflow to contribute
+
+To contribute to **loky**, first create an account on
+[github](http://github.com/). Once this is done, fork the
+[loky repository](http://github.com/loky/loky) to have your own repository,
+clone it using 'git clone' on the computers where you want to work. Make your
+changes in your clone, push them to your github account, test them on several
+computers, and when you are happy with them, send a pull request to the main
+repository.
+
+### Running the test suite
+
+To run the test suite, you need the `pytest` (version >= 3) and `psutil`
+modules. From the root of the project, run the test suite using:
+
+```sh
+ pip install -e .
+ pytest .
+```
+
+### Why was the project named `loky`?
+
+While developping `loky`, we had some bad experiences trying to debug deadlocks
+when using `multiprocessing.Pool` and `concurrent.futures.ProcessPoolExecutor`,
+especially when calling functions with non-picklable arguments or returned
+values at the beginning of the project. When we had to chose a name, we had
+dealt with so many deadlocks that we wanted some kind of invocation to repel
+them! Hence `loky`: a mix of a god, locks and the `y` that make it somehow
+cooler and nicer : (and also less likely to result in name conflict in google
+results ^^).
+
+Fixes to avoid those deadlocks in `concurrent.futures` were also contributed
+upstream in Python 3.7+, as a less mystical way to repel the deadlocks :D
+
+### Acknowledgement
+
+This work is supported by the Center for Data Science, funded by the IDEX
+Paris-Saclay, ANR-11-IDEX-0003-02
+
+
+[1]: https://github.com/giampaolo/psutil
+
+
+
+
+%package -n python3-loky
+Summary: A robust implementation of concurrent.futures.ProcessPoolExecutor
+Provides: python-loky
+BuildRequires: python3-devel
+BuildRequires: python3-setuptools
+BuildRequires: python3-pip
+%description -n python3-loky
+
+<a href="https://loky.readthedocs.io">
+<img src="docs/_static/loky_logo.svg"
+alt="Loky logo" width=96/></a>
+
+
+# Reusable Process Pool Executor
+[![Build Status](https://dev.azure.com/joblib/loky/_apis/build/status/joblib.loky?branchName=master)](https://dev.azure.com/joblib/loky/_build/latest?definitionId=2&branchName=master)
+[![Documentation Status](https://readthedocs.org/projects/loky/badge/?version=latest)](https://loky.readthedocs.io/en/latest/?badge=latest)
+[![codecov](https://codecov.io/gh/joblib/loky/branch/master/graph/badge.svg)](https://codecov.io/gh/joblib/loky)
+[![DOI](https://zenodo.org/badge/48578152.svg)](https://zenodo.org/badge/latestdoi/48578152)
+
+
+### Goal
+
+The aim of this project is to provide a robust, cross-platform and
+cross-version implementation of the `ProcessPoolExecutor` class of
+`concurrent.futures`. It notably features:
+
+ * __Consistent and robust spawn behavior__: All processes are started
+ using fork + exec on POSIX systems. This ensures safer interactions with
+ third party libraries. On the contrary, `multiprocessing.Pool` uses
+ fork without exec by default, causing third party runtimes to crash
+ (e.g. OpenMP, macOS Accelerate...).
+
+ * __Reusable executor__: strategy to avoid re-spawning a complete
+ executor every time. A singleton executor instance can be reused (and
+ dynamically resized if necessary) across consecutive calls to limit
+ spawning and shutdown overhead. The worker processes can be shutdown
+ automatically after a configurable idling timeout to free system
+ resources.
+
+ * __Transparent cloudpickle integration__: to call interactively
+ defined functions and lambda expressions in parallel. It is also
+ possible to register a custom pickler implementation to handle
+ inter-process communications.
+
+ * __No need for ``if __name__ == "__main__":`` in scripts__: thanks
+ to the use of ``cloudpickle`` to call functions defined in the
+ ``__main__`` module, it is not required to protect the code calling
+ parallel functions under Windows.
+
+ * __Deadlock free implementation__: one of the major concern in
+ standard `multiprocessing` and `concurrent.futures` modules is the
+ ability of the `Pool/Executor` to handle crashes of worker
+ processes. This library intends to fix those possible deadlocks and
+ send back meaningful errors. Note that the implementation of
+ `concurrent.futures.ProcessPoolExecutor` that comes with Python 3.7+
+ is as robust as the executor from loky but the latter also works for
+ older versions of Python.
+
+
+### Installation
+
+The recommended way to install `loky` is with `pip`,
+```bash
+pip install loky
+```
+
+`loky` can also be installed from sources using
+```bash
+git clone https://github.com/joblib/loky
+cd loky
+python setup.py install
+```
+
+Note that `loky` has an optional dependency on [`psutil`][1] to allow early
+memory leak detections.
+
+### Usage
+
+The basic usage of `loky` relies on the `get_reusable_executor`, which
+internally manages a custom `ProcessPoolExecutor` object, which is reused or
+re-spawned depending on the context.
+
+```python
+import os
+from time import sleep
+from loky import get_reusable_executor
+
+
+def say_hello(k):
+ pid = os.getpid()
+ print(f"Hello from {pid} with arg {k}")
+ sleep(.01)
+ return pid
+
+
+# Create an executor with 4 worker processes, that will
+# automatically shutdown after idling for 2s
+executor = get_reusable_executor(max_workers=4, timeout=2)
+
+res = executor.submit(say_hello, 1)
+print("Got results:", res.result())
+
+results = executor.map(say_hello, range(50))
+n_workers = len(set(results))
+print("Number of used processes:", n_workers)
+assert n_workers == 4
+```
+
+For more advance usage, see our
+[documentation](https://loky.readthedocs.io/en/stable/)
+
+### Workflow to contribute
+
+To contribute to **loky**, first create an account on
+[github](http://github.com/). Once this is done, fork the
+[loky repository](http://github.com/loky/loky) to have your own repository,
+clone it using 'git clone' on the computers where you want to work. Make your
+changes in your clone, push them to your github account, test them on several
+computers, and when you are happy with them, send a pull request to the main
+repository.
+
+### Running the test suite
+
+To run the test suite, you need the `pytest` (version >= 3) and `psutil`
+modules. From the root of the project, run the test suite using:
+
+```sh
+ pip install -e .
+ pytest .
+```
+
+### Why was the project named `loky`?
+
+While developping `loky`, we had some bad experiences trying to debug deadlocks
+when using `multiprocessing.Pool` and `concurrent.futures.ProcessPoolExecutor`,
+especially when calling functions with non-picklable arguments or returned
+values at the beginning of the project. When we had to chose a name, we had
+dealt with so many deadlocks that we wanted some kind of invocation to repel
+them! Hence `loky`: a mix of a god, locks and the `y` that make it somehow
+cooler and nicer : (and also less likely to result in name conflict in google
+results ^^).
+
+Fixes to avoid those deadlocks in `concurrent.futures` were also contributed
+upstream in Python 3.7+, as a less mystical way to repel the deadlocks :D
+
+### Acknowledgement
+
+This work is supported by the Center for Data Science, funded by the IDEX
+Paris-Saclay, ANR-11-IDEX-0003-02
+
+
+[1]: https://github.com/giampaolo/psutil
+
+
+
+
+%package help
+Summary: Development documents and examples for loky
+Provides: python3-loky-doc
+%description help
+
+<a href="https://loky.readthedocs.io">
+<img src="docs/_static/loky_logo.svg"
+alt="Loky logo" width=96/></a>
+
+
+# Reusable Process Pool Executor
+[![Build Status](https://dev.azure.com/joblib/loky/_apis/build/status/joblib.loky?branchName=master)](https://dev.azure.com/joblib/loky/_build/latest?definitionId=2&branchName=master)
+[![Documentation Status](https://readthedocs.org/projects/loky/badge/?version=latest)](https://loky.readthedocs.io/en/latest/?badge=latest)
+[![codecov](https://codecov.io/gh/joblib/loky/branch/master/graph/badge.svg)](https://codecov.io/gh/joblib/loky)
+[![DOI](https://zenodo.org/badge/48578152.svg)](https://zenodo.org/badge/latestdoi/48578152)
+
+
+### Goal
+
+The aim of this project is to provide a robust, cross-platform and
+cross-version implementation of the `ProcessPoolExecutor` class of
+`concurrent.futures`. It notably features:
+
+ * __Consistent and robust spawn behavior__: All processes are started
+ using fork + exec on POSIX systems. This ensures safer interactions with
+ third party libraries. On the contrary, `multiprocessing.Pool` uses
+ fork without exec by default, causing third party runtimes to crash
+ (e.g. OpenMP, macOS Accelerate...).
+
+ * __Reusable executor__: strategy to avoid re-spawning a complete
+ executor every time. A singleton executor instance can be reused (and
+ dynamically resized if necessary) across consecutive calls to limit
+ spawning and shutdown overhead. The worker processes can be shutdown
+ automatically after a configurable idling timeout to free system
+ resources.
+
+ * __Transparent cloudpickle integration__: to call interactively
+ defined functions and lambda expressions in parallel. It is also
+ possible to register a custom pickler implementation to handle
+ inter-process communications.
+
+ * __No need for ``if __name__ == "__main__":`` in scripts__: thanks
+ to the use of ``cloudpickle`` to call functions defined in the
+ ``__main__`` module, it is not required to protect the code calling
+ parallel functions under Windows.
+
+ * __Deadlock free implementation__: one of the major concern in
+ standard `multiprocessing` and `concurrent.futures` modules is the
+ ability of the `Pool/Executor` to handle crashes of worker
+ processes. This library intends to fix those possible deadlocks and
+ send back meaningful errors. Note that the implementation of
+ `concurrent.futures.ProcessPoolExecutor` that comes with Python 3.7+
+ is as robust as the executor from loky but the latter also works for
+ older versions of Python.
+
+
+### Installation
+
+The recommended way to install `loky` is with `pip`,
+```bash
+pip install loky
+```
+
+`loky` can also be installed from sources using
+```bash
+git clone https://github.com/joblib/loky
+cd loky
+python setup.py install
+```
+
+Note that `loky` has an optional dependency on [`psutil`][1] to allow early
+memory leak detections.
+
+### Usage
+
+The basic usage of `loky` relies on the `get_reusable_executor`, which
+internally manages a custom `ProcessPoolExecutor` object, which is reused or
+re-spawned depending on the context.
+
+```python
+import os
+from time import sleep
+from loky import get_reusable_executor
+
+
+def say_hello(k):
+ pid = os.getpid()
+ print(f"Hello from {pid} with arg {k}")
+ sleep(.01)
+ return pid
+
+
+# Create an executor with 4 worker processes, that will
+# automatically shutdown after idling for 2s
+executor = get_reusable_executor(max_workers=4, timeout=2)
+
+res = executor.submit(say_hello, 1)
+print("Got results:", res.result())
+
+results = executor.map(say_hello, range(50))
+n_workers = len(set(results))
+print("Number of used processes:", n_workers)
+assert n_workers == 4
+```
+
+For more advance usage, see our
+[documentation](https://loky.readthedocs.io/en/stable/)
+
+### Workflow to contribute
+
+To contribute to **loky**, first create an account on
+[github](http://github.com/). Once this is done, fork the
+[loky repository](http://github.com/loky/loky) to have your own repository,
+clone it using 'git clone' on the computers where you want to work. Make your
+changes in your clone, push them to your github account, test them on several
+computers, and when you are happy with them, send a pull request to the main
+repository.
+
+### Running the test suite
+
+To run the test suite, you need the `pytest` (version >= 3) and `psutil`
+modules. From the root of the project, run the test suite using:
+
+```sh
+ pip install -e .
+ pytest .
+```
+
+### Why was the project named `loky`?
+
+While developping `loky`, we had some bad experiences trying to debug deadlocks
+when using `multiprocessing.Pool` and `concurrent.futures.ProcessPoolExecutor`,
+especially when calling functions with non-picklable arguments or returned
+values at the beginning of the project. When we had to chose a name, we had
+dealt with so many deadlocks that we wanted some kind of invocation to repel
+them! Hence `loky`: a mix of a god, locks and the `y` that make it somehow
+cooler and nicer : (and also less likely to result in name conflict in google
+results ^^).
+
+Fixes to avoid those deadlocks in `concurrent.futures` were also contributed
+upstream in Python 3.7+, as a less mystical way to repel the deadlocks :D
+
+### Acknowledgement
+
+This work is supported by the Center for Data Science, funded by the IDEX
+Paris-Saclay, ANR-11-IDEX-0003-02
+
+
+[1]: https://github.com/giampaolo/psutil
+
+
+
+
+%prep
+%autosetup -n loky-3.3.0
+
+%build
+%py3_build
+
+%install
+%py3_install
+install -d -m755 %{buildroot}/%{_pkgdocdir}
+if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi
+if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi
+if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi
+if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi
+pushd %{buildroot}
+if [ -d usr/lib ]; then
+ find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/lib64 ]; then
+ find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/bin ]; then
+ find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/sbin ]; then
+ find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+touch doclist.lst
+if [ -d usr/share/man ]; then
+ find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst
+fi
+popd
+mv %{buildroot}/filelist.lst .
+mv %{buildroot}/doclist.lst .
+
+%files -n python3-loky -f filelist.lst
+%dir %{python3_sitelib}/*
+
+%files help -f doclist.lst
+%{_docdir}/*
+
+%changelog
+* Tue Apr 11 2023 Python_Bot <Python_Bot@openeuler.org> - 3.3.0-1
+- Package Spec generated