path: root/python-loky.spec

%global _empty_manifest_terminate_build 0
Name:		python-loky
Version:	3.3.0
Release:	1
Summary:	A robust implementation of concurrent.futures.ProcessPoolExecutor
License:	BSD
URL:		https://github.com/joblib/loky/
Source0:	https://mirrors.nju.edu.cn/pypi/web/packages/35/0e/97520e1a9317637727fd05d00ae4d834eaf82b19ca7ef3e2dcedf4446656/loky-3.3.0.tar.gz
BuildArch:	noarch

Requires:	python3-cloudpickle

%description

<a href="https://loky.readthedocs.io">
<img src="docs/_static/loky_logo.svg"
alt="Loky logo" width=96/></a>


# Reusable Process Pool Executor
[![Build Status](https://dev.azure.com/joblib/loky/_apis/build/status/joblib.loky?branchName=master)](https://dev.azure.com/joblib/loky/_build/latest?definitionId=2&branchName=master)
[![Documentation Status](https://readthedocs.org/projects/loky/badge/?version=latest)](https://loky.readthedocs.io/en/latest/?badge=latest)
[![codecov](https://codecov.io/gh/joblib/loky/branch/master/graph/badge.svg)](https://codecov.io/gh/joblib/loky)
[![DOI](https://zenodo.org/badge/48578152.svg)](https://zenodo.org/badge/latestdoi/48578152)


### Goal

The aim of this project is to provide a robust, cross-platform and
cross-version implementation of the `ProcessPoolExecutor` class of
`concurrent.futures`. It notably features:

  * __Consistent and robust spawn behavior__: All processes are started
    using fork + exec on POSIX systems. This ensures safer interactions with
    third party libraries. On the contrary, `multiprocessing.Pool` uses
    fork without exec by default, causing third party runtimes to crash
    (e.g. OpenMP, macOS Accelerate...).

  * __Reusable executor__: strategy to avoid re-spawning a complete
    executor every time. A singleton executor instance can be reused (and
    dynamically resized if necessary) across consecutive calls to limit
    spawning and shutdown overhead. The worker processes can be shutdown
    automatically after a configurable idling timeout to free system
    resources.

  * __Transparent cloudpickle integration__: to call interactively
    defined functions and lambda expressions in parallel. It is also
    possible to register a custom pickler implementation to handle
    inter-process communications.

  * __No need for ``if __name__ == "__main__":`` in scripts__: thanks
    to the use of ``cloudpickle`` to call functions defined in the
    ``__main__`` module, it is not required to protect the code calling
    parallel functions under Windows.

  * __Deadlock free implementation__: one of the major concern in
    standard `multiprocessing` and `concurrent.futures` modules is the
    ability of the `Pool/Executor` to handle crashes of worker
    processes. This library intends to fix those possible deadlocks and
    send back meaningful errors. Note that the implementation of
    `concurrent.futures.ProcessPoolExecutor` that comes with Python 3.7+
    is as robust as the executor from loky but the latter also works for
    older versions of Python.


### Installation

The recommended way to install `loky` is with `pip`,
```bash
pip install loky
```

`loky` can also be installed from sources using
```bash
git clone https://github.com/joblib/loky
cd loky
python setup.py install
```

Note that `loky` has an optional dependency on [`psutil`][1] to allow early
memory leak detections.

### Usage

The basic usage of `loky` relies on the `get_reusable_executor`, which
internally manages a custom `ProcessPoolExecutor` object, which is reused or
re-spawned depending on the context.

```python
import os
from time import sleep
from loky import get_reusable_executor


def say_hello(k):
    pid = os.getpid()
    print(f"Hello from {pid} with arg {k}")
    sleep(.01)
    return pid


# Create an executor with 4 worker processes, that will
# automatically shutdown after idling for 2s
executor = get_reusable_executor(max_workers=4, timeout=2)

res = executor.submit(say_hello, 1)
print("Got results:", res.result())

results = executor.map(say_hello, range(50))
n_workers = len(set(results))
print("Number of used processes:", n_workers)
assert n_workers == 4
```

For more advance usage, see our
[documentation](https://loky.readthedocs.io/en/stable/)

### Workflow to contribute

To contribute to **loky**, first create an account on
[github](http://github.com/). Once this is done, fork the
[loky repository](http://github.com/loky/loky) to have your own repository,
clone it using 'git clone' on the computers where you want to work. Make your
changes in your clone, push them to your github account, test them on several
computers, and when you are happy with them, send a pull request to the main
repository.

### Running the test suite

To run the test suite, you need the `pytest` (version >= 3) and `psutil`
modules. From the root of the project, run the test suite using:

```sh
    pip install -e .
    pytest .
```

### Why was the project named `loky`?

While developping `loky`, we had some bad experiences trying to debug deadlocks
when using `multiprocessing.Pool` and `concurrent.futures.ProcessPoolExecutor`,
especially when calling functions with non-picklable arguments or returned
values at the beginning of the project. When we had to chose a name, we had
dealt with so many deadlocks that we wanted some kind of invocation to repel
them! Hence `loky`: a mix of a god, locks and the `y` that make it somehow
cooler and nicer : (and also less likely to result in name conflict in google
results ^^).

Fixes to avoid those deadlocks in `concurrent.futures` were also contributed
upstream in Python 3.7+, as a less mystical way to repel the deadlocks :D

### Acknowledgement

This work is supported by the Center for Data Science, funded by the IDEX
Paris-Saclay, ANR-11-IDEX-0003-02


[1]: https://github.com/giampaolo/psutil




%package -n python3-loky
Summary:	A robust implementation of concurrent.futures.ProcessPoolExecutor
Provides:	python-loky
BuildRequires:	python3-devel
BuildRequires:	python3-setuptools
BuildRequires:	python3-pip
%description -n python3-loky

<a href="https://loky.readthedocs.io">
<img src="docs/_static/loky_logo.svg"
alt="Loky logo" width=96/></a>


# Reusable Process Pool Executor
[![Build Status](https://dev.azure.com/joblib/loky/_apis/build/status/joblib.loky?branchName=master)](https://dev.azure.com/joblib/loky/_build/latest?definitionId=2&branchName=master)
[![Documentation Status](https://readthedocs.org/projects/loky/badge/?version=latest)](https://loky.readthedocs.io/en/latest/?badge=latest)
[![codecov](https://codecov.io/gh/joblib/loky/branch/master/graph/badge.svg)](https://codecov.io/gh/joblib/loky)
[![DOI](https://zenodo.org/badge/48578152.svg)](https://zenodo.org/badge/latestdoi/48578152)


### Goal

The aim of this project is to provide a robust, cross-platform and
cross-version implementation of the `ProcessPoolExecutor` class of
`concurrent.futures`. It notably features:

  * __Consistent and robust spawn behavior__: All processes are started
    using fork + exec on POSIX systems. This ensures safer interactions with
    third party libraries. On the contrary, `multiprocessing.Pool` uses
    fork without exec by default, causing third party runtimes to crash
    (e.g. OpenMP, macOS Accelerate...).

  * __Reusable executor__: strategy to avoid re-spawning a complete
    executor every time. A singleton executor instance can be reused (and
    dynamically resized if necessary) across consecutive calls to limit
    spawning and shutdown overhead. The worker processes can be shutdown
    automatically after a configurable idling timeout to free system
    resources.

  * __Transparent cloudpickle integration__: to call interactively
    defined functions and lambda expressions in parallel. It is also
    possible to register a custom pickler implementation to handle
    inter-process communications.

  * __No need for ``if __name__ == "__main__":`` in scripts__: thanks
    to the use of ``cloudpickle`` to call functions defined in the
    ``__main__`` module, it is not required to protect the code calling
    parallel functions under Windows.

  * __Deadlock free implementation__: one of the major concern in
    standard `multiprocessing` and `concurrent.futures` modules is the
    ability of the `Pool/Executor` to handle crashes of worker
    processes. This library intends to fix those possible deadlocks and
    send back meaningful errors. Note that the implementation of
    `concurrent.futures.ProcessPoolExecutor` that comes with Python 3.7+
    is as robust as the executor from loky but the latter also works for
    older versions of Python.


### Installation

The recommended way to install `loky` is with `pip`,
```bash
pip install loky
```

`loky` can also be installed from sources using
```bash
git clone https://github.com/joblib/loky
cd loky
python setup.py install
```

Note that `loky` has an optional dependency on [`psutil`][1] to allow early
memory leak detections.

### Usage

The basic usage of `loky` relies on the `get_reusable_executor`, which
internally manages a custom `ProcessPoolExecutor` object, which is reused or
re-spawned depending on the context.

```python
import os
from time import sleep
from loky import get_reusable_executor


def say_hello(k):
    pid = os.getpid()
    print(f"Hello from {pid} with arg {k}")
    sleep(.01)
    return pid


# Create an executor with 4 worker processes, that will
# automatically shutdown after idling for 2s
executor = get_reusable_executor(max_workers=4, timeout=2)

res = executor.submit(say_hello, 1)
print("Got results:", res.result())

results = executor.map(say_hello, range(50))
n_workers = len(set(results))
print("Number of used processes:", n_workers)
assert n_workers == 4
```

For more advance usage, see our
[documentation](https://loky.readthedocs.io/en/stable/)

### Workflow to contribute

To contribute to **loky**, first create an account on
[github](http://github.com/). Once this is done, fork the
[loky repository](http://github.com/loky/loky) to have your own repository,
clone it using 'git clone' on the computers where you want to work. Make your
changes in your clone, push them to your github account, test them on several
computers, and when you are happy with them, send a pull request to the main
repository.

### Running the test suite

To run the test suite, you need the `pytest` (version >= 3) and `psutil`
modules. From the root of the project, run the test suite using:

```sh
    pip install -e .
    pytest .
```

### Why was the project named `loky`?

While developping `loky`, we had some bad experiences trying to debug deadlocks
when using `multiprocessing.Pool` and `concurrent.futures.ProcessPoolExecutor`,
especially when calling functions with non-picklable arguments or returned
values at the beginning of the project. When we had to chose a name, we had
dealt with so many deadlocks that we wanted some kind of invocation to repel
them! Hence `loky`: a mix of a god, locks and the `y` that make it somehow
cooler and nicer : (and also less likely to result in name conflict in google
results ^^).

Fixes to avoid those deadlocks in `concurrent.futures` were also contributed
upstream in Python 3.7+, as a less mystical way to repel the deadlocks :D

### Acknowledgement

This work is supported by the Center for Data Science, funded by the IDEX
Paris-Saclay, ANR-11-IDEX-0003-02


[1]: https://github.com/giampaolo/psutil




%package help
Summary:	Development documents and examples for loky
Provides:	python3-loky-doc
%description help

<a href="https://loky.readthedocs.io">
<img src="docs/_static/loky_logo.svg"
alt="Loky logo" width=96/></a>


# Reusable Process Pool Executor
[![Build Status](https://dev.azure.com/joblib/loky/_apis/build/status/joblib.loky?branchName=master)](https://dev.azure.com/joblib/loky/_build/latest?definitionId=2&branchName=master)
[![Documentation Status](https://readthedocs.org/projects/loky/badge/?version=latest)](https://loky.readthedocs.io/en/latest/?badge=latest)
[![codecov](https://codecov.io/gh/joblib/loky/branch/master/graph/badge.svg)](https://codecov.io/gh/joblib/loky)
[![DOI](https://zenodo.org/badge/48578152.svg)](https://zenodo.org/badge/latestdoi/48578152)


### Goal

The aim of this project is to provide a robust, cross-platform and
cross-version implementation of the `ProcessPoolExecutor` class of
`concurrent.futures`. It notably features:

  * __Consistent and robust spawn behavior__: All processes are started
    using fork + exec on POSIX systems. This ensures safer interactions with
    third party libraries. On the contrary, `multiprocessing.Pool` uses
    fork without exec by default, causing third party runtimes to crash
    (e.g. OpenMP, macOS Accelerate...).

  * __Reusable executor__: strategy to avoid re-spawning a complete
    executor every time. A singleton executor instance can be reused (and
    dynamically resized if necessary) across consecutive calls to limit
    spawning and shutdown overhead. The worker processes can be shutdown
    automatically after a configurable idling timeout to free system
    resources.

  * __Transparent cloudpickle integration__: to call interactively
    defined functions and lambda expressions in parallel. It is also
    possible to register a custom pickler implementation to handle
    inter-process communications.

  * __No need for ``if __name__ == "__main__":`` in scripts__: thanks
    to the use of ``cloudpickle`` to call functions defined in the
    ``__main__`` module, it is not required to protect the code calling
    parallel functions under Windows.

  * __Deadlock free implementation__: one of the major concern in
    standard `multiprocessing` and `concurrent.futures` modules is the
    ability of the `Pool/Executor` to handle crashes of worker
    processes. This library intends to fix those possible deadlocks and
    send back meaningful errors. Note that the implementation of
    `concurrent.futures.ProcessPoolExecutor` that comes with Python 3.7+
    is as robust as the executor from loky but the latter also works for
    older versions of Python.


### Installation

The recommended way to install `loky` is with `pip`,
```bash
pip install loky
```

`loky` can also be installed from sources using
```bash
git clone https://github.com/joblib/loky
cd loky
python setup.py install
```

Note that `loky` has an optional dependency on [`psutil`][1] to allow early
memory leak detections.

### Usage

The basic usage of `loky` relies on the `get_reusable_executor`, which
internally manages a custom `ProcessPoolExecutor` object, which is reused or
re-spawned depending on the context.

```python
import os
from time import sleep
from loky import get_reusable_executor


def say_hello(k):
    pid = os.getpid()
    print(f"Hello from {pid} with arg {k}")
    sleep(.01)
    return pid


# Create an executor with 4 worker processes, that will
# automatically shutdown after idling for 2s
executor = get_reusable_executor(max_workers=4, timeout=2)

res = executor.submit(say_hello, 1)
print("Got results:", res.result())

results = executor.map(say_hello, range(50))
n_workers = len(set(results))
print("Number of used processes:", n_workers)
assert n_workers == 4
```

For more advance usage, see our
[documentation](https://loky.readthedocs.io/en/stable/)

### Workflow to contribute

To contribute to **loky**, first create an account on
[github](http://github.com/). Once this is done, fork the
[loky repository](http://github.com/loky/loky) to have your own repository,
clone it using 'git clone' on the computers where you want to work. Make your
changes in your clone, push them to your github account, test them on several
computers, and when you are happy with them, send a pull request to the main
repository.

### Running the test suite

To run the test suite, you need the `pytest` (version >= 3) and `psutil`
modules. From the root of the project, run the test suite using:

```sh
    pip install -e .
    pytest .
```

### Why was the project named `loky`?

While developping `loky`, we had some bad experiences trying to debug deadlocks
when using `multiprocessing.Pool` and `concurrent.futures.ProcessPoolExecutor`,
especially when calling functions with non-picklable arguments or returned
values at the beginning of the project. When we had to chose a name, we had
dealt with so many deadlocks that we wanted some kind of invocation to repel
them! Hence `loky`: a mix of a god, locks and the `y` that make it somehow
cooler and nicer : (and also less likely to result in name conflict in google
results ^^).

Fixes to avoid those deadlocks in `concurrent.futures` were also contributed
upstream in Python 3.7+, as a less mystical way to repel the deadlocks :D

### Acknowledgement

This work is supported by the Center for Data Science, funded by the IDEX
Paris-Saclay, ANR-11-IDEX-0003-02


[1]: https://github.com/giampaolo/psutil




%prep
%autosetup -n loky-3.3.0

%build
%py3_build

%install
%py3_install
install -d -m755 %{buildroot}/%{_pkgdocdir}
if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi
if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi
if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi
if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi
pushd %{buildroot}
if [ -d usr/lib ]; then
	find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst
fi
if [ -d usr/lib64 ]; then
	find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst
fi
if [ -d usr/bin ]; then
	find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst
fi
if [ -d usr/sbin ]; then
	find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst
fi
touch doclist.lst
if [ -d usr/share/man ]; then
	find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst
fi
popd
mv %{buildroot}/filelist.lst .
mv %{buildroot}/doclist.lst .

%files -n python3-loky -f filelist.lst
%dir %{python3_sitelib}/*

%files help -f doclist.lst
%{_docdir}/*

%changelog
* Tue Apr 11 2023 Python_Bot <Python_Bot@openeuler.org> - 3.3.0-1
- Package Spec generated