%global _empty_manifest_terminate_build 0 Name: python-packtivity Version: 0.15.0 Release: 1 Summary: packtivity - general purpose schema + bindings for PROV activities License: MIT License URL: https://github.com/yadage/packtivity Source0: https://mirrors.aliyun.com/pypi/web/packages/20/bb/3f7b9406f81288eea20e00214416f616ea766d3861a331b46d4b2d86dae1/packtivity-0.15.0.tar.gz BuildArch: noarch Requires: python3-requests[security] Requires: python3-jsonschema Requires: python3-jsonref Requires: python3-pyyaml Requires: python3-click Requires: python3-glob2 Requires: python3-jsonpointer Requires: python3-jsonpath-rw Requires: python3-jq Requires: python3-yadage-schemas Requires: python3-mock Requires: python3-checksumdir Requires: python3-celery Requires: python3-redis Requires: python3-importlib-metadata %description # packtivity [![DOI](https://zenodo.org/badge/53696818.svg)](https://zenodo.org/badge/latestdoi/53696818) [![Coverage Status](https://coveralls.io/repos/github/diana-hep/packtivity/badge.svg)](https://coveralls.io/github/diana-hep/packtivity) [![Documentation Status](https://readthedocs.org/projects/packtivity/badge/?version=latest)](http://packtivity.readthedocs.io/en/latest/?badge=latest) [![PyPI](https://img.shields.io/pypi/v/packtivity.svg)](https://pypi.python.org/pypi/packtivity) This package aims to collect implementations of both synchronous and asynchronous execution of preserved, but parametrized scientific computational tasks that come with batteries included, i.e. with a full specification of their software dependencies. In that sense they are *packaged activities* -- packtivities. This package provides tools to validate and execute data processing tasks that are written according to the "packtivity" JSON schemas defined in https://github.com/diana-hep/yadage-schemas. Packtivities define * the software environment * parametrized process descriptions (what programs to run within these environment) and * produces human and machine readable outputs (as JSON) of the resulting data fragments. At run-time they are paired with a concrete set of parameters supplied as JSON documents and and external storage/state to actually execute these tasks. ## Packtivity in Yadage This package is used by https://github.com/lukasheinrich/yadage to execute the individual steps of yadage workflows. ## Example Packtivity spec This packtivity spec is part of a number of yadage workflow and runs the Delphes detector simulation on a HepMC file and outputs events in the LHCO and ROOT file formats. This packtivity is (stored in a public location)[https://github.com/lukasheinrich/yadage-workflows/blob/master/phenochain/delphes.yml] from which it can be later retrieved: process: process_type: 'string-interpolated-cmd' cmd: 'DelphesHepMC {delphes_card} {outputroot} {inputhepmc} && root2lhco {outputroot} {outputlhco}' publisher: publisher_type: 'frompar-pub' outputmap: lhcofile: outputlhco rootfile: outputroot environment: environment_type: 'docker-encapsulated' image: lukasheinrich/root-delphes ## Usage You can run the packtivity in a synchronous way by specifying the spec (can point to GitHub), all necessary parameters and attaching an external state (via the `--read` and `--write` flags). packtivity-run -t from-github/phenochain delphes.yml \ -p inputhepmc="$PWD/pythia/output.hepmc" \ -p outputroot="'{workdir}/output.root'" \ -p outputlhco="'{workdir}/output.lhco'" \ -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \ --read pythia --write outdir ## Asynchronous Backends In order to facilitate usage of distributed resources, a number of Asynchronous backends can be specified. Here is an example for IPython Parallel clusters packtivity-run -b ipcluster --asyncwait \ -t from-github/phenochain delphes.yml \ -p inputhepmc="$PWD/pythia/output.hepmc" \ -p outputroot="'{workdir}/output.root'" \ -p outputlhco="'{workdir}/output.lhco'" \ -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \ --read pythia --write outdir You can replacing the `--asyncwait` with `--async` flag in order to get a JSONable proxy representation with which to later on check on the job status. By default the proxy information is written to `proxy.json` (customizable via the `-x` flag): packtivity-run -b celery --async \ -t from-github/phenochain delphes.yml \ -p inputhepmc="$PWD/pythia/output.hepmc" \ -p outputroot="'{workdir}/output.root'" \ -p outputlhco="'{workdir}/output.lhco'" \ -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \ --read pythia --write outdir And at a later point in time you can check via: packtivity-checkproxy proxy.json ## External Backends Users can implement their own backends to handle the JSON documents describing the packtivities. It can be enabled by using the `fromenv` backend and setting an environment variable specifying the module holding the backend and proxy classes. The format of the environment variable is `module:backendclass:proxyclass`. E.g.: export PACKTIVITY_ASYNCBACKEND="externalbackend:ExternalBackend:ExternalProxy" %package -n python3-packtivity Summary: packtivity - general purpose schema + bindings for PROV activities Provides: python-packtivity BuildRequires: python3-devel BuildRequires: python3-setuptools BuildRequires: python3-pip %description -n python3-packtivity # packtivity [![DOI](https://zenodo.org/badge/53696818.svg)](https://zenodo.org/badge/latestdoi/53696818) [![Coverage Status](https://coveralls.io/repos/github/diana-hep/packtivity/badge.svg)](https://coveralls.io/github/diana-hep/packtivity) [![Documentation Status](https://readthedocs.org/projects/packtivity/badge/?version=latest)](http://packtivity.readthedocs.io/en/latest/?badge=latest) [![PyPI](https://img.shields.io/pypi/v/packtivity.svg)](https://pypi.python.org/pypi/packtivity) This package aims to collect implementations of both synchronous and asynchronous execution of preserved, but parametrized scientific computational tasks that come with batteries included, i.e. with a full specification of their software dependencies. In that sense they are *packaged activities* -- packtivities. This package provides tools to validate and execute data processing tasks that are written according to the "packtivity" JSON schemas defined in https://github.com/diana-hep/yadage-schemas. Packtivities define * the software environment * parametrized process descriptions (what programs to run within these environment) and * produces human and machine readable outputs (as JSON) of the resulting data fragments. At run-time they are paired with a concrete set of parameters supplied as JSON documents and and external storage/state to actually execute these tasks. ## Packtivity in Yadage This package is used by https://github.com/lukasheinrich/yadage to execute the individual steps of yadage workflows. ## Example Packtivity spec This packtivity spec is part of a number of yadage workflow and runs the Delphes detector simulation on a HepMC file and outputs events in the LHCO and ROOT file formats. This packtivity is (stored in a public location)[https://github.com/lukasheinrich/yadage-workflows/blob/master/phenochain/delphes.yml] from which it can be later retrieved: process: process_type: 'string-interpolated-cmd' cmd: 'DelphesHepMC {delphes_card} {outputroot} {inputhepmc} && root2lhco {outputroot} {outputlhco}' publisher: publisher_type: 'frompar-pub' outputmap: lhcofile: outputlhco rootfile: outputroot environment: environment_type: 'docker-encapsulated' image: lukasheinrich/root-delphes ## Usage You can run the packtivity in a synchronous way by specifying the spec (can point to GitHub), all necessary parameters and attaching an external state (via the `--read` and `--write` flags). packtivity-run -t from-github/phenochain delphes.yml \ -p inputhepmc="$PWD/pythia/output.hepmc" \ -p outputroot="'{workdir}/output.root'" \ -p outputlhco="'{workdir}/output.lhco'" \ -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \ --read pythia --write outdir ## Asynchronous Backends In order to facilitate usage of distributed resources, a number of Asynchronous backends can be specified. Here is an example for IPython Parallel clusters packtivity-run -b ipcluster --asyncwait \ -t from-github/phenochain delphes.yml \ -p inputhepmc="$PWD/pythia/output.hepmc" \ -p outputroot="'{workdir}/output.root'" \ -p outputlhco="'{workdir}/output.lhco'" \ -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \ --read pythia --write outdir You can replacing the `--asyncwait` with `--async` flag in order to get a JSONable proxy representation with which to later on check on the job status. By default the proxy information is written to `proxy.json` (customizable via the `-x` flag): packtivity-run -b celery --async \ -t from-github/phenochain delphes.yml \ -p inputhepmc="$PWD/pythia/output.hepmc" \ -p outputroot="'{workdir}/output.root'" \ -p outputlhco="'{workdir}/output.lhco'" \ -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \ --read pythia --write outdir And at a later point in time you can check via: packtivity-checkproxy proxy.json ## External Backends Users can implement their own backends to handle the JSON documents describing the packtivities. It can be enabled by using the `fromenv` backend and setting an environment variable specifying the module holding the backend and proxy classes. The format of the environment variable is `module:backendclass:proxyclass`. E.g.: export PACKTIVITY_ASYNCBACKEND="externalbackend:ExternalBackend:ExternalProxy" %package help Summary: Development documents and examples for packtivity Provides: python3-packtivity-doc %description help # packtivity [![DOI](https://zenodo.org/badge/53696818.svg)](https://zenodo.org/badge/latestdoi/53696818) [![Coverage Status](https://coveralls.io/repos/github/diana-hep/packtivity/badge.svg)](https://coveralls.io/github/diana-hep/packtivity) [![Documentation Status](https://readthedocs.org/projects/packtivity/badge/?version=latest)](http://packtivity.readthedocs.io/en/latest/?badge=latest) [![PyPI](https://img.shields.io/pypi/v/packtivity.svg)](https://pypi.python.org/pypi/packtivity) This package aims to collect implementations of both synchronous and asynchronous execution of preserved, but parametrized scientific computational tasks that come with batteries included, i.e. with a full specification of their software dependencies. In that sense they are *packaged activities* -- packtivities. This package provides tools to validate and execute data processing tasks that are written according to the "packtivity" JSON schemas defined in https://github.com/diana-hep/yadage-schemas. Packtivities define * the software environment * parametrized process descriptions (what programs to run within these environment) and * produces human and machine readable outputs (as JSON) of the resulting data fragments. At run-time they are paired with a concrete set of parameters supplied as JSON documents and and external storage/state to actually execute these tasks. ## Packtivity in Yadage This package is used by https://github.com/lukasheinrich/yadage to execute the individual steps of yadage workflows. ## Example Packtivity spec This packtivity spec is part of a number of yadage workflow and runs the Delphes detector simulation on a HepMC file and outputs events in the LHCO and ROOT file formats. This packtivity is (stored in a public location)[https://github.com/lukasheinrich/yadage-workflows/blob/master/phenochain/delphes.yml] from which it can be later retrieved: process: process_type: 'string-interpolated-cmd' cmd: 'DelphesHepMC {delphes_card} {outputroot} {inputhepmc} && root2lhco {outputroot} {outputlhco}' publisher: publisher_type: 'frompar-pub' outputmap: lhcofile: outputlhco rootfile: outputroot environment: environment_type: 'docker-encapsulated' image: lukasheinrich/root-delphes ## Usage You can run the packtivity in a synchronous way by specifying the spec (can point to GitHub), all necessary parameters and attaching an external state (via the `--read` and `--write` flags). packtivity-run -t from-github/phenochain delphes.yml \ -p inputhepmc="$PWD/pythia/output.hepmc" \ -p outputroot="'{workdir}/output.root'" \ -p outputlhco="'{workdir}/output.lhco'" \ -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \ --read pythia --write outdir ## Asynchronous Backends In order to facilitate usage of distributed resources, a number of Asynchronous backends can be specified. Here is an example for IPython Parallel clusters packtivity-run -b ipcluster --asyncwait \ -t from-github/phenochain delphes.yml \ -p inputhepmc="$PWD/pythia/output.hepmc" \ -p outputroot="'{workdir}/output.root'" \ -p outputlhco="'{workdir}/output.lhco'" \ -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \ --read pythia --write outdir You can replacing the `--asyncwait` with `--async` flag in order to get a JSONable proxy representation with which to later on check on the job status. By default the proxy information is written to `proxy.json` (customizable via the `-x` flag): packtivity-run -b celery --async \ -t from-github/phenochain delphes.yml \ -p inputhepmc="$PWD/pythia/output.hepmc" \ -p outputroot="'{workdir}/output.root'" \ -p outputlhco="'{workdir}/output.lhco'" \ -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \ --read pythia --write outdir And at a later point in time you can check via: packtivity-checkproxy proxy.json ## External Backends Users can implement their own backends to handle the JSON documents describing the packtivities. It can be enabled by using the `fromenv` backend and setting an environment variable specifying the module holding the backend and proxy classes. The format of the environment variable is `module:backendclass:proxyclass`. E.g.: export PACKTIVITY_ASYNCBACKEND="externalbackend:ExternalBackend:ExternalProxy" %prep %autosetup -n packtivity-0.15.0 %build %py3_build %install %py3_install install -d -m755 %{buildroot}/%{_pkgdocdir} if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi pushd %{buildroot} if [ -d usr/lib ]; then find usr/lib -type f -printf "\"/%h/%f\"\n" >> filelist.lst fi if [ -d usr/lib64 ]; then find usr/lib64 -type f -printf "\"/%h/%f\"\n" >> filelist.lst fi if [ -d usr/bin ]; then find usr/bin -type f -printf "\"/%h/%f\"\n" >> filelist.lst fi if [ -d usr/sbin ]; then find usr/sbin -type f -printf "\"/%h/%f\"\n" >> filelist.lst fi touch doclist.lst if [ -d usr/share/man ]; then find usr/share/man -type f -printf "\"/%h/%f.gz\"\n" >> doclist.lst fi popd mv %{buildroot}/filelist.lst . mv %{buildroot}/doclist.lst . %files -n python3-packtivity -f filelist.lst %dir %{python3_sitelib}/* %files help -f doclist.lst %{_docdir}/* %changelog * Thu Jun 08 2023 Python_Bot - 0.15.0-1 - Package Spec generated