diff options
Diffstat (limited to 'python-prefetch-generator.spec')
| -rw-r--r-- | python-prefetch-generator.spec | 177 |
1 files changed, 177 insertions, 0 deletions
diff --git a/python-prefetch-generator.spec b/python-prefetch-generator.spec new file mode 100644 index 0000000..0ed7d7a --- /dev/null +++ b/python-prefetch-generator.spec @@ -0,0 +1,177 @@ +%global _empty_manifest_terminate_build 0 +Name: python-prefetch-generator +Version: 1.0.3 +Release: 1 +Summary: a simple tool to compute arbitrary generator in a background thread +License: The Unlicense +URL: https://github.com/justheuristic/prefetch_generator +Source0: https://mirrors.nju.edu.cn/pypi/web/packages/28/cf/f6c148289fd87832e9f20428b7e842921efac5313cb3ac5920a424543760/prefetch_generator-1.0.3.tar.gz +BuildArch: noarch + + +%description +Based on http://stackoverflow.com/questions/7323664/python-generator-pre-fetch + +This is a single-function package that makes it possible to transform any generator into a `BackgroundGenerator` which computes any number of elements from the generator ahead, in a background thread. + +It is quite lightweight, but not entirely weightless. + +The `BackgroundGenerator` is most useful when you have a GIL releasing task which might take a long time to complete (e.g. Disk I/O, Web Requests, pure C functions, GPU processing, ...), and another task which takes a similar amount of time, but is dependent on the results of the first task (e.g. Computationally intensive processing of data loaded from disk). + +Normally these two tasks will constantly wait for one another to finish. If you make one of these tasks a `BackgroundGenerator` (see examples below), they will work in parallel, potentially saving up to 50% of execution time (definitely less in practice). + +We personally use the `BackgroundGenerator` when iterating over minibatches of data for deep learning with tensorflow and theano ( lasagne, blocks, raw, etc.). + +Quick usage example (ipython notebook) - https://github.com/justheuristic/prefetch_generator/blob/master/example.ipynb + +This package contains two objects: + - The Class `BackgroundGenerator(generator [,max_prefetch=1])` + - The decorator `@prefetch([max_prefetch=1])` + +the usage is either + +#for item in BackgroundGenerator(my_generator): +# do_stuff(item) + +or + +#@prefetch() +#def my_generator(some_param): +# while True: +# X = read_heavy_file() +# y = wget_from_cornhub() +# do_pretty_much_anything(some_param) +# yield X, y + + +More details are written in the `BackgroundGenerator` doc: +See `help(BackgroundGenerator)` + +%package -n python3-prefetch-generator +Summary: a simple tool to compute arbitrary generator in a background thread +Provides: python-prefetch-generator +BuildRequires: python3-devel +BuildRequires: python3-setuptools +BuildRequires: python3-pip +%description -n python3-prefetch-generator +Based on http://stackoverflow.com/questions/7323664/python-generator-pre-fetch + +This is a single-function package that makes it possible to transform any generator into a `BackgroundGenerator` which computes any number of elements from the generator ahead, in a background thread. + +It is quite lightweight, but not entirely weightless. + +The `BackgroundGenerator` is most useful when you have a GIL releasing task which might take a long time to complete (e.g. Disk I/O, Web Requests, pure C functions, GPU processing, ...), and another task which takes a similar amount of time, but is dependent on the results of the first task (e.g. Computationally intensive processing of data loaded from disk). + +Normally these two tasks will constantly wait for one another to finish. If you make one of these tasks a `BackgroundGenerator` (see examples below), they will work in parallel, potentially saving up to 50% of execution time (definitely less in practice). + +We personally use the `BackgroundGenerator` when iterating over minibatches of data for deep learning with tensorflow and theano ( lasagne, blocks, raw, etc.). + +Quick usage example (ipython notebook) - https://github.com/justheuristic/prefetch_generator/blob/master/example.ipynb + +This package contains two objects: + - The Class `BackgroundGenerator(generator [,max_prefetch=1])` + - The decorator `@prefetch([max_prefetch=1])` + +the usage is either + +#for item in BackgroundGenerator(my_generator): +# do_stuff(item) + +or + +#@prefetch() +#def my_generator(some_param): +# while True: +# X = read_heavy_file() +# y = wget_from_cornhub() +# do_pretty_much_anything(some_param) +# yield X, y + + +More details are written in the `BackgroundGenerator` doc: +See `help(BackgroundGenerator)` + +%package help +Summary: Development documents and examples for prefetch-generator +Provides: python3-prefetch-generator-doc +%description help +Based on http://stackoverflow.com/questions/7323664/python-generator-pre-fetch + +This is a single-function package that makes it possible to transform any generator into a `BackgroundGenerator` which computes any number of elements from the generator ahead, in a background thread. + +It is quite lightweight, but not entirely weightless. + +The `BackgroundGenerator` is most useful when you have a GIL releasing task which might take a long time to complete (e.g. Disk I/O, Web Requests, pure C functions, GPU processing, ...), and another task which takes a similar amount of time, but is dependent on the results of the first task (e.g. Computationally intensive processing of data loaded from disk). + +Normally these two tasks will constantly wait for one another to finish. If you make one of these tasks a `BackgroundGenerator` (see examples below), they will work in parallel, potentially saving up to 50% of execution time (definitely less in practice). + +We personally use the `BackgroundGenerator` when iterating over minibatches of data for deep learning with tensorflow and theano ( lasagne, blocks, raw, etc.). + +Quick usage example (ipython notebook) - https://github.com/justheuristic/prefetch_generator/blob/master/example.ipynb + +This package contains two objects: + - The Class `BackgroundGenerator(generator [,max_prefetch=1])` + - The decorator `@prefetch([max_prefetch=1])` + +the usage is either + +#for item in BackgroundGenerator(my_generator): +# do_stuff(item) + +or + +#@prefetch() +#def my_generator(some_param): +# while True: +# X = read_heavy_file() +# y = wget_from_cornhub() +# do_pretty_much_anything(some_param) +# yield X, y + + +More details are written in the `BackgroundGenerator` doc: +See `help(BackgroundGenerator)` + +%prep +%autosetup -n prefetch-generator-1.0.3 + +%build +%py3_build + +%install +%py3_install +install -d -m755 %{buildroot}/%{_pkgdocdir} +if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi +if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi +if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi +if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi +pushd %{buildroot} +if [ -d usr/lib ]; then + find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/lib64 ]; then + find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/bin ]; then + find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/sbin ]; then + find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst +fi +touch doclist.lst +if [ -d usr/share/man ]; then + find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst +fi +popd +mv %{buildroot}/filelist.lst . +mv %{buildroot}/doclist.lst . + +%files -n python3-prefetch-generator -f filelist.lst +%dir %{python3_sitelib}/* + +%files help -f doclist.lst +%{_docdir}/* + +%changelog +* Tue Apr 11 2023 Python_Bot <Python_Bot@openeuler.org> - 1.0.3-1 +- Package Spec generated |