diff options
author | CoprDistGit <infra@openeuler.org> | 2023-05-10 10:02:17 +0000 |
---|---|---|
committer | CoprDistGit <infra@openeuler.org> | 2023-05-10 10:02:17 +0000 |
commit | f7ed37f3a49a53295e984bb25adab701ca7694ce (patch) | |
tree | 1afddd151bae86f69dbf44fcc4aefae446e8d15a | |
parent | d154d9e10dc08b3732682ec1e9c297ca88903ce7 (diff) |
automatic import of python-packtivity
-rw-r--r-- | .gitignore | 1 | ||||
-rw-r--r-- | python-packtivity.spec | 339 | ||||
-rw-r--r-- | sources | 1 |
3 files changed, 341 insertions, 0 deletions
@@ -0,0 +1 @@ +/packtivity-0.15.0.tar.gz diff --git a/python-packtivity.spec b/python-packtivity.spec new file mode 100644 index 0000000..105c66e --- /dev/null +++ b/python-packtivity.spec @@ -0,0 +1,339 @@ +%global _empty_manifest_terminate_build 0 +Name: python-packtivity +Version: 0.15.0 +Release: 1 +Summary: packtivity - general purpose schema + bindings for PROV activities +License: MIT License +URL: https://github.com/yadage/packtivity +Source0: https://mirrors.nju.edu.cn/pypi/web/packages/20/bb/3f7b9406f81288eea20e00214416f616ea766d3861a331b46d4b2d86dae1/packtivity-0.15.0.tar.gz +BuildArch: noarch + +Requires: python3-requests[security] +Requires: python3-jsonschema +Requires: python3-jsonref +Requires: python3-pyyaml +Requires: python3-click +Requires: python3-glob2 +Requires: python3-jsonpointer +Requires: python3-jsonpath-rw +Requires: python3-jq +Requires: python3-yadage-schemas +Requires: python3-mock +Requires: python3-checksumdir +Requires: python3-celery +Requires: python3-redis +Requires: python3-importlib-metadata + +%description +# packtivity + +[](https://zenodo.org/badge/latestdoi/53696818) +[](https://coveralls.io/github/diana-hep/packtivity) +[](http://packtivity.readthedocs.io/en/latest/?badge=latest) +[](https://pypi.python.org/pypi/packtivity) + +This package aims to collect implementations of both synchronous and asynchronous execution of preserved, but parametrized scientific computational tasks that come with batteries included, i.e. with a full specification of their software dependencies. In that sense they are *packaged activities* -- packtivities. + +This package provides tools to validate and execute data processing tasks that are written according to the "packtivity" JSON schemas defined in https://github.com/diana-hep/yadage-schemas. + +Packtivities define + +* the software environment +* parametrized process descriptions (what programs to run within these environment) and +* produces human and machine readable outputs (as JSON) of the resulting data fragments. + +At run-time they are paired with a concrete set of parameters supplied as JSON documents and and external storage/state to actually execute these tasks. + +## Packtivity in Yadage + +This package is used by https://github.com/lukasheinrich/yadage to execute the individual steps of yadage workflows. + +## Example Packtivity spec + +This packtivity spec is part of a number of yadage workflow and runs the Delphes detector simulation on a HepMC file and outputs events in the LHCO and ROOT file formats. This packtivity is (stored in a public location)[https://github.com/lukasheinrich/yadage-workflows/blob/master/phenochain/delphes.yml] from which it can be later retrieved: + + process: + process_type: 'string-interpolated-cmd' + cmd: 'DelphesHepMC {delphes_card} {outputroot} {inputhepmc} && root2lhco {outputroot} {outputlhco}' + publisher: + publisher_type: 'frompar-pub' + outputmap: + lhcofile: outputlhco + rootfile: outputroot + environment: + environment_type: 'docker-encapsulated' + image: lukasheinrich/root-delphes + +## Usage + +You can run the packtivity in a synchronous way by specifying the spec (can point to GitHub), all necessary parameters and attaching an external state (via the `--read` and `--write` flags). + + packtivity-run -t from-github/phenochain delphes.yml \ + -p inputhepmc="$PWD/pythia/output.hepmc" \ + -p outputroot="'{workdir}/output.root'" \ + -p outputlhco="'{workdir}/output.lhco'" \ + -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \ + --read pythia --write outdir + +## Asynchronous Backends + +In order to facilitate usage of distributed resources, a number of Asynchronous +backends can be specified. Here is an example for IPython Parallel clusters + + packtivity-run -b ipcluster --asyncwait \ + -t from-github/phenochain delphes.yml \ + -p inputhepmc="$PWD/pythia/output.hepmc" \ + -p outputroot="'{workdir}/output.root'" \ + -p outputlhco="'{workdir}/output.lhco'" \ + -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \ + --read pythia --write outdir + +You can replacing the `--asyncwait` with `--async` flag in order to get a JSONable proxy representation with which to later on check on the job status. By default the proxy information is written to `proxy.json` (customizable via the `-x` flag): + + packtivity-run -b celery --async \ + -t from-github/phenochain delphes.yml \ + -p inputhepmc="$PWD/pythia/output.hepmc" \ + -p outputroot="'{workdir}/output.root'" \ + -p outputlhco="'{workdir}/output.lhco'" \ + -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \ + --read pythia --write outdir + +And at a later point in time you can check via: + + packtivity-checkproxy proxy.json + +## External Backends + +Users can implement their own backends to handle the JSON documents describing the packtivities. It can be enabled +by using the `fromenv` backend and setting an environment variable specifying the module holding the backend and proxy +classes. The format of the environment variable is `module:backendclass:proxyclass`. E.g.: + + export PACKTIVITY_ASYNCBACKEND="externalbackend:ExternalBackend:ExternalProxy" + + +%package -n python3-packtivity +Summary: packtivity - general purpose schema + bindings for PROV activities +Provides: python-packtivity +BuildRequires: python3-devel +BuildRequires: python3-setuptools +BuildRequires: python3-pip +%description -n python3-packtivity +# packtivity + +[](https://zenodo.org/badge/latestdoi/53696818) +[](https://coveralls.io/github/diana-hep/packtivity) +[](http://packtivity.readthedocs.io/en/latest/?badge=latest) +[](https://pypi.python.org/pypi/packtivity) + +This package aims to collect implementations of both synchronous and asynchronous execution of preserved, but parametrized scientific computational tasks that come with batteries included, i.e. with a full specification of their software dependencies. In that sense they are *packaged activities* -- packtivities. + +This package provides tools to validate and execute data processing tasks that are written according to the "packtivity" JSON schemas defined in https://github.com/diana-hep/yadage-schemas. + +Packtivities define + +* the software environment +* parametrized process descriptions (what programs to run within these environment) and +* produces human and machine readable outputs (as JSON) of the resulting data fragments. + +At run-time they are paired with a concrete set of parameters supplied as JSON documents and and external storage/state to actually execute these tasks. + +## Packtivity in Yadage + +This package is used by https://github.com/lukasheinrich/yadage to execute the individual steps of yadage workflows. + +## Example Packtivity spec + +This packtivity spec is part of a number of yadage workflow and runs the Delphes detector simulation on a HepMC file and outputs events in the LHCO and ROOT file formats. This packtivity is (stored in a public location)[https://github.com/lukasheinrich/yadage-workflows/blob/master/phenochain/delphes.yml] from which it can be later retrieved: + + process: + process_type: 'string-interpolated-cmd' + cmd: 'DelphesHepMC {delphes_card} {outputroot} {inputhepmc} && root2lhco {outputroot} {outputlhco}' + publisher: + publisher_type: 'frompar-pub' + outputmap: + lhcofile: outputlhco + rootfile: outputroot + environment: + environment_type: 'docker-encapsulated' + image: lukasheinrich/root-delphes + +## Usage + +You can run the packtivity in a synchronous way by specifying the spec (can point to GitHub), all necessary parameters and attaching an external state (via the `--read` and `--write` flags). + + packtivity-run -t from-github/phenochain delphes.yml \ + -p inputhepmc="$PWD/pythia/output.hepmc" \ + -p outputroot="'{workdir}/output.root'" \ + -p outputlhco="'{workdir}/output.lhco'" \ + -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \ + --read pythia --write outdir + +## Asynchronous Backends + +In order to facilitate usage of distributed resources, a number of Asynchronous +backends can be specified. Here is an example for IPython Parallel clusters + + packtivity-run -b ipcluster --asyncwait \ + -t from-github/phenochain delphes.yml \ + -p inputhepmc="$PWD/pythia/output.hepmc" \ + -p outputroot="'{workdir}/output.root'" \ + -p outputlhco="'{workdir}/output.lhco'" \ + -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \ + --read pythia --write outdir + +You can replacing the `--asyncwait` with `--async` flag in order to get a JSONable proxy representation with which to later on check on the job status. By default the proxy information is written to `proxy.json` (customizable via the `-x` flag): + + packtivity-run -b celery --async \ + -t from-github/phenochain delphes.yml \ + -p inputhepmc="$PWD/pythia/output.hepmc" \ + -p outputroot="'{workdir}/output.root'" \ + -p outputlhco="'{workdir}/output.lhco'" \ + -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \ + --read pythia --write outdir + +And at a later point in time you can check via: + + packtivity-checkproxy proxy.json + +## External Backends + +Users can implement their own backends to handle the JSON documents describing the packtivities. It can be enabled +by using the `fromenv` backend and setting an environment variable specifying the module holding the backend and proxy +classes. The format of the environment variable is `module:backendclass:proxyclass`. E.g.: + + export PACKTIVITY_ASYNCBACKEND="externalbackend:ExternalBackend:ExternalProxy" + + +%package help +Summary: Development documents and examples for packtivity +Provides: python3-packtivity-doc +%description help +# packtivity + +[](https://zenodo.org/badge/latestdoi/53696818) +[](https://coveralls.io/github/diana-hep/packtivity) +[](http://packtivity.readthedocs.io/en/latest/?badge=latest) +[](https://pypi.python.org/pypi/packtivity) + +This package aims to collect implementations of both synchronous and asynchronous execution of preserved, but parametrized scientific computational tasks that come with batteries included, i.e. with a full specification of their software dependencies. In that sense they are *packaged activities* -- packtivities. + +This package provides tools to validate and execute data processing tasks that are written according to the "packtivity" JSON schemas defined in https://github.com/diana-hep/yadage-schemas. + +Packtivities define + +* the software environment +* parametrized process descriptions (what programs to run within these environment) and +* produces human and machine readable outputs (as JSON) of the resulting data fragments. + +At run-time they are paired with a concrete set of parameters supplied as JSON documents and and external storage/state to actually execute these tasks. + +## Packtivity in Yadage + +This package is used by https://github.com/lukasheinrich/yadage to execute the individual steps of yadage workflows. + +## Example Packtivity spec + +This packtivity spec is part of a number of yadage workflow and runs the Delphes detector simulation on a HepMC file and outputs events in the LHCO and ROOT file formats. This packtivity is (stored in a public location)[https://github.com/lukasheinrich/yadage-workflows/blob/master/phenochain/delphes.yml] from which it can be later retrieved: + + process: + process_type: 'string-interpolated-cmd' + cmd: 'DelphesHepMC {delphes_card} {outputroot} {inputhepmc} && root2lhco {outputroot} {outputlhco}' + publisher: + publisher_type: 'frompar-pub' + outputmap: + lhcofile: outputlhco + rootfile: outputroot + environment: + environment_type: 'docker-encapsulated' + image: lukasheinrich/root-delphes + +## Usage + +You can run the packtivity in a synchronous way by specifying the spec (can point to GitHub), all necessary parameters and attaching an external state (via the `--read` and `--write` flags). + + packtivity-run -t from-github/phenochain delphes.yml \ + -p inputhepmc="$PWD/pythia/output.hepmc" \ + -p outputroot="'{workdir}/output.root'" \ + -p outputlhco="'{workdir}/output.lhco'" \ + -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \ + --read pythia --write outdir + +## Asynchronous Backends + +In order to facilitate usage of distributed resources, a number of Asynchronous +backends can be specified. Here is an example for IPython Parallel clusters + + packtivity-run -b ipcluster --asyncwait \ + -t from-github/phenochain delphes.yml \ + -p inputhepmc="$PWD/pythia/output.hepmc" \ + -p outputroot="'{workdir}/output.root'" \ + -p outputlhco="'{workdir}/output.lhco'" \ + -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \ + --read pythia --write outdir + +You can replacing the `--asyncwait` with `--async` flag in order to get a JSONable proxy representation with which to later on check on the job status. By default the proxy information is written to `proxy.json` (customizable via the `-x` flag): + + packtivity-run -b celery --async \ + -t from-github/phenochain delphes.yml \ + -p inputhepmc="$PWD/pythia/output.hepmc" \ + -p outputroot="'{workdir}/output.root'" \ + -p outputlhco="'{workdir}/output.lhco'" \ + -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \ + --read pythia --write outdir + +And at a later point in time you can check via: + + packtivity-checkproxy proxy.json + +## External Backends + +Users can implement their own backends to handle the JSON documents describing the packtivities. It can be enabled +by using the `fromenv` backend and setting an environment variable specifying the module holding the backend and proxy +classes. The format of the environment variable is `module:backendclass:proxyclass`. E.g.: + + export PACKTIVITY_ASYNCBACKEND="externalbackend:ExternalBackend:ExternalProxy" + + +%prep +%autosetup -n packtivity-0.15.0 + +%build +%py3_build + +%install +%py3_install +install -d -m755 %{buildroot}/%{_pkgdocdir} +if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi +if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi +if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi +if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi +pushd %{buildroot} +if [ -d usr/lib ]; then + find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/lib64 ]; then + find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/bin ]; then + find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/sbin ]; then + find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst +fi +touch doclist.lst +if [ -d usr/share/man ]; then + find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst +fi +popd +mv %{buildroot}/filelist.lst . +mv %{buildroot}/doclist.lst . + +%files -n python3-packtivity -f filelist.lst +%dir %{python3_sitelib}/* + +%files help -f doclist.lst +%{_docdir}/* + +%changelog +* Wed May 10 2023 Python_Bot <Python_Bot@openeuler.org> - 0.15.0-1 +- Package Spec generated @@ -0,0 +1 @@ +bcd46057db63a36c936a21d6c32df2ae packtivity-0.15.0.tar.gz |