1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
|
%global _empty_manifest_terminate_build 0
Name: python-packtivity
Version: 0.15.0
Release: 1
Summary: packtivity - general purpose schema + bindings for PROV activities
License: MIT License
URL: https://github.com/yadage/packtivity
Source0: https://mirrors.aliyun.com/pypi/web/packages/20/bb/3f7b9406f81288eea20e00214416f616ea766d3861a331b46d4b2d86dae1/packtivity-0.15.0.tar.gz
BuildArch: noarch
Requires: python3-requests[security]
Requires: python3-jsonschema
Requires: python3-jsonref
Requires: python3-pyyaml
Requires: python3-click
Requires: python3-glob2
Requires: python3-jsonpointer
Requires: python3-jsonpath-rw
Requires: python3-jq
Requires: python3-yadage-schemas
Requires: python3-mock
Requires: python3-checksumdir
Requires: python3-celery
Requires: python3-redis
Requires: python3-importlib-metadata
%description
# packtivity
[](https://zenodo.org/badge/latestdoi/53696818)
[](https://coveralls.io/github/diana-hep/packtivity)
[](http://packtivity.readthedocs.io/en/latest/?badge=latest)
[](https://pypi.python.org/pypi/packtivity)
This package aims to collect implementations of both synchronous and asynchronous execution of preserved, but parametrized scientific computational tasks that come with batteries included, i.e. with a full specification of their software dependencies. In that sense they are *packaged activities* -- packtivities.
This package provides tools to validate and execute data processing tasks that are written according to the "packtivity" JSON schemas defined in https://github.com/diana-hep/yadage-schemas.
Packtivities define
* the software environment
* parametrized process descriptions (what programs to run within these environment) and
* produces human and machine readable outputs (as JSON) of the resulting data fragments.
At run-time they are paired with a concrete set of parameters supplied as JSON documents and and external storage/state to actually execute these tasks.
## Packtivity in Yadage
This package is used by https://github.com/lukasheinrich/yadage to execute the individual steps of yadage workflows.
## Example Packtivity spec
This packtivity spec is part of a number of yadage workflow and runs the Delphes detector simulation on a HepMC file and outputs events in the LHCO and ROOT file formats. This packtivity is (stored in a public location)[https://github.com/lukasheinrich/yadage-workflows/blob/master/phenochain/delphes.yml] from which it can be later retrieved:
process:
process_type: 'string-interpolated-cmd'
cmd: 'DelphesHepMC {delphes_card} {outputroot} {inputhepmc} && root2lhco {outputroot} {outputlhco}'
publisher:
publisher_type: 'frompar-pub'
outputmap:
lhcofile: outputlhco
rootfile: outputroot
environment:
environment_type: 'docker-encapsulated'
image: lukasheinrich/root-delphes
## Usage
You can run the packtivity in a synchronous way by specifying the spec (can point to GitHub), all necessary parameters and attaching an external state (via the `--read` and `--write` flags).
packtivity-run -t from-github/phenochain delphes.yml \
-p inputhepmc="$PWD/pythia/output.hepmc" \
-p outputroot="'{workdir}/output.root'" \
-p outputlhco="'{workdir}/output.lhco'" \
-p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \
--read pythia --write outdir
## Asynchronous Backends
In order to facilitate usage of distributed resources, a number of Asynchronous
backends can be specified. Here is an example for IPython Parallel clusters
packtivity-run -b ipcluster --asyncwait \
-t from-github/phenochain delphes.yml \
-p inputhepmc="$PWD/pythia/output.hepmc" \
-p outputroot="'{workdir}/output.root'" \
-p outputlhco="'{workdir}/output.lhco'" \
-p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \
--read pythia --write outdir
You can replacing the `--asyncwait` with `--async` flag in order to get a JSONable proxy representation with which to later on check on the job status. By default the proxy information is written to `proxy.json` (customizable via the `-x` flag):
packtivity-run -b celery --async \
-t from-github/phenochain delphes.yml \
-p inputhepmc="$PWD/pythia/output.hepmc" \
-p outputroot="'{workdir}/output.root'" \
-p outputlhco="'{workdir}/output.lhco'" \
-p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \
--read pythia --write outdir
And at a later point in time you can check via:
packtivity-checkproxy proxy.json
## External Backends
Users can implement their own backends to handle the JSON documents describing the packtivities. It can be enabled
by using the `fromenv` backend and setting an environment variable specifying the module holding the backend and proxy
classes. The format of the environment variable is `module:backendclass:proxyclass`. E.g.:
export PACKTIVITY_ASYNCBACKEND="externalbackend:ExternalBackend:ExternalProxy"
%package -n python3-packtivity
Summary: packtivity - general purpose schema + bindings for PROV activities
Provides: python-packtivity
BuildRequires: python3-devel
BuildRequires: python3-setuptools
BuildRequires: python3-pip
%description -n python3-packtivity
# packtivity
[](https://zenodo.org/badge/latestdoi/53696818)
[](https://coveralls.io/github/diana-hep/packtivity)
[](http://packtivity.readthedocs.io/en/latest/?badge=latest)
[](https://pypi.python.org/pypi/packtivity)
This package aims to collect implementations of both synchronous and asynchronous execution of preserved, but parametrized scientific computational tasks that come with batteries included, i.e. with a full specification of their software dependencies. In that sense they are *packaged activities* -- packtivities.
This package provides tools to validate and execute data processing tasks that are written according to the "packtivity" JSON schemas defined in https://github.com/diana-hep/yadage-schemas.
Packtivities define
* the software environment
* parametrized process descriptions (what programs to run within these environment) and
* produces human and machine readable outputs (as JSON) of the resulting data fragments.
At run-time they are paired with a concrete set of parameters supplied as JSON documents and and external storage/state to actually execute these tasks.
## Packtivity in Yadage
This package is used by https://github.com/lukasheinrich/yadage to execute the individual steps of yadage workflows.
## Example Packtivity spec
This packtivity spec is part of a number of yadage workflow and runs the Delphes detector simulation on a HepMC file and outputs events in the LHCO and ROOT file formats. This packtivity is (stored in a public location)[https://github.com/lukasheinrich/yadage-workflows/blob/master/phenochain/delphes.yml] from which it can be later retrieved:
process:
process_type: 'string-interpolated-cmd'
cmd: 'DelphesHepMC {delphes_card} {outputroot} {inputhepmc} && root2lhco {outputroot} {outputlhco}'
publisher:
publisher_type: 'frompar-pub'
outputmap:
lhcofile: outputlhco
rootfile: outputroot
environment:
environment_type: 'docker-encapsulated'
image: lukasheinrich/root-delphes
## Usage
You can run the packtivity in a synchronous way by specifying the spec (can point to GitHub), all necessary parameters and attaching an external state (via the `--read` and `--write` flags).
packtivity-run -t from-github/phenochain delphes.yml \
-p inputhepmc="$PWD/pythia/output.hepmc" \
-p outputroot="'{workdir}/output.root'" \
-p outputlhco="'{workdir}/output.lhco'" \
-p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \
--read pythia --write outdir
## Asynchronous Backends
In order to facilitate usage of distributed resources, a number of Asynchronous
backends can be specified. Here is an example for IPython Parallel clusters
packtivity-run -b ipcluster --asyncwait \
-t from-github/phenochain delphes.yml \
-p inputhepmc="$PWD/pythia/output.hepmc" \
-p outputroot="'{workdir}/output.root'" \
-p outputlhco="'{workdir}/output.lhco'" \
-p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \
--read pythia --write outdir
You can replacing the `--asyncwait` with `--async` flag in order to get a JSONable proxy representation with which to later on check on the job status. By default the proxy information is written to `proxy.json` (customizable via the `-x` flag):
packtivity-run -b celery --async \
-t from-github/phenochain delphes.yml \
-p inputhepmc="$PWD/pythia/output.hepmc" \
-p outputroot="'{workdir}/output.root'" \
-p outputlhco="'{workdir}/output.lhco'" \
-p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \
--read pythia --write outdir
And at a later point in time you can check via:
packtivity-checkproxy proxy.json
## External Backends
Users can implement their own backends to handle the JSON documents describing the packtivities. It can be enabled
by using the `fromenv` backend and setting an environment variable specifying the module holding the backend and proxy
classes. The format of the environment variable is `module:backendclass:proxyclass`. E.g.:
export PACKTIVITY_ASYNCBACKEND="externalbackend:ExternalBackend:ExternalProxy"
%package help
Summary: Development documents and examples for packtivity
Provides: python3-packtivity-doc
%description help
# packtivity
[](https://zenodo.org/badge/latestdoi/53696818)
[](https://coveralls.io/github/diana-hep/packtivity)
[](http://packtivity.readthedocs.io/en/latest/?badge=latest)
[](https://pypi.python.org/pypi/packtivity)
This package aims to collect implementations of both synchronous and asynchronous execution of preserved, but parametrized scientific computational tasks that come with batteries included, i.e. with a full specification of their software dependencies. In that sense they are *packaged activities* -- packtivities.
This package provides tools to validate and execute data processing tasks that are written according to the "packtivity" JSON schemas defined in https://github.com/diana-hep/yadage-schemas.
Packtivities define
* the software environment
* parametrized process descriptions (what programs to run within these environment) and
* produces human and machine readable outputs (as JSON) of the resulting data fragments.
At run-time they are paired with a concrete set of parameters supplied as JSON documents and and external storage/state to actually execute these tasks.
## Packtivity in Yadage
This package is used by https://github.com/lukasheinrich/yadage to execute the individual steps of yadage workflows.
## Example Packtivity spec
This packtivity spec is part of a number of yadage workflow and runs the Delphes detector simulation on a HepMC file and outputs events in the LHCO and ROOT file formats. This packtivity is (stored in a public location)[https://github.com/lukasheinrich/yadage-workflows/blob/master/phenochain/delphes.yml] from which it can be later retrieved:
process:
process_type: 'string-interpolated-cmd'
cmd: 'DelphesHepMC {delphes_card} {outputroot} {inputhepmc} && root2lhco {outputroot} {outputlhco}'
publisher:
publisher_type: 'frompar-pub'
outputmap:
lhcofile: outputlhco
rootfile: outputroot
environment:
environment_type: 'docker-encapsulated'
image: lukasheinrich/root-delphes
## Usage
You can run the packtivity in a synchronous way by specifying the spec (can point to GitHub), all necessary parameters and attaching an external state (via the `--read` and `--write` flags).
packtivity-run -t from-github/phenochain delphes.yml \
-p inputhepmc="$PWD/pythia/output.hepmc" \
-p outputroot="'{workdir}/output.root'" \
-p outputlhco="'{workdir}/output.lhco'" \
-p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \
--read pythia --write outdir
## Asynchronous Backends
In order to facilitate usage of distributed resources, a number of Asynchronous
backends can be specified. Here is an example for IPython Parallel clusters
packtivity-run -b ipcluster --asyncwait \
-t from-github/phenochain delphes.yml \
-p inputhepmc="$PWD/pythia/output.hepmc" \
-p outputroot="'{workdir}/output.root'" \
-p outputlhco="'{workdir}/output.lhco'" \
-p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \
--read pythia --write outdir
You can replacing the `--asyncwait` with `--async` flag in order to get a JSONable proxy representation with which to later on check on the job status. By default the proxy information is written to `proxy.json` (customizable via the `-x` flag):
packtivity-run -b celery --async \
-t from-github/phenochain delphes.yml \
-p inputhepmc="$PWD/pythia/output.hepmc" \
-p outputroot="'{workdir}/output.root'" \
-p outputlhco="'{workdir}/output.lhco'" \
-p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \
--read pythia --write outdir
And at a later point in time you can check via:
packtivity-checkproxy proxy.json
## External Backends
Users can implement their own backends to handle the JSON documents describing the packtivities. It can be enabled
by using the `fromenv` backend and setting an environment variable specifying the module holding the backend and proxy
classes. The format of the environment variable is `module:backendclass:proxyclass`. E.g.:
export PACKTIVITY_ASYNCBACKEND="externalbackend:ExternalBackend:ExternalProxy"
%prep
%autosetup -n packtivity-0.15.0
%build
%py3_build
%install
%py3_install
install -d -m755 %{buildroot}/%{_pkgdocdir}
if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi
if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi
if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi
if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi
pushd %{buildroot}
if [ -d usr/lib ]; then
find usr/lib -type f -printf "\"/%h/%f\"\n" >> filelist.lst
fi
if [ -d usr/lib64 ]; then
find usr/lib64 -type f -printf "\"/%h/%f\"\n" >> filelist.lst
fi
if [ -d usr/bin ]; then
find usr/bin -type f -printf "\"/%h/%f\"\n" >> filelist.lst
fi
if [ -d usr/sbin ]; then
find usr/sbin -type f -printf "\"/%h/%f\"\n" >> filelist.lst
fi
touch doclist.lst
if [ -d usr/share/man ]; then
find usr/share/man -type f -printf "\"/%h/%f.gz\"\n" >> doclist.lst
fi
popd
mv %{buildroot}/filelist.lst .
mv %{buildroot}/doclist.lst .
%files -n python3-packtivity -f filelist.lst
%dir %{python3_sitelib}/*
%files help -f doclist.lst
%{_docdir}/*
%changelog
* Thu Jun 08 2023 Python_Bot <Python_Bot@openeuler.org> - 0.15.0-1
- Package Spec generated
|
