summaryrefslogtreecommitdiff
path: root/python-dsub.spec
diff options
context:
space:
mode:
Diffstat (limited to 'python-dsub.spec')
-rw-r--r--python-dsub.spec315
1 files changed, 315 insertions, 0 deletions
diff --git a/python-dsub.spec b/python-dsub.spec
new file mode 100644
index 0000000..9218fc7
--- /dev/null
+++ b/python-dsub.spec
@@ -0,0 +1,315 @@
+%global _empty_manifest_terminate_build 0
+Name: python-dsub
+Version: 0.4.9
+Release: 1
+Summary: A command-line tool that makes it easy to submit and run batch scripts in the cloud
+License: Apache
+URL: https://github.com/DataBiosphere/dsub
+Source0: https://mirrors.nju.edu.cn/pypi/web/packages/7a/36/6e2013482f87a93f4334525435c6a8e1050b028899d9054050bf62d2d23c/dsub-0.4.9.tar.gz
+BuildArch: noarch
+
+Requires: python3-funcsigs
+Requires: python3-google-api-core
+Requires: python3-google-api-python-client
+Requires: python3-google-auth-httplib2
+Requires: python3-google-auth
+Requires: python3-google-cloud-batch
+Requires: python3-httplib2
+Requires: python3-mock
+Requires: python3-parameterized
+Requires: python3-pyasn1-modules
+Requires: python3-pyasn1
+Requires: python3-dateutil
+Requires: python3-pytz
+Requires: python3-pyyaml
+Requires: python3-rsa
+Requires: python3-tabulate
+Requires: python3-tenacity
+Requires: python3-uritemplate
+
+%description
+my-job-name RUNNING 2
+my-job-name SUCCESS 1
+```
+In this mode, dstat prints one line per (job name, task status) pair. You can
+see at a glance how many tasks are finished, how many are still running, and
+how many are failed/canceled.
+### Deleting a job
+The `ddel` command will delete running jobs.
+By default, only jobs submitted by the current user will be deleted.
+Use the `--users` flag to specify other users, or `'*'` for all users.
+To delete a running job:
+ ddel --provider google-v2 --project my-cloud-project --jobs job-id
+If the job is a batch job, all running tasks will be deleted.
+To delete specific tasks:
+ ddel \
+ --provider google-v2 \
+ --project my-cloud-project \
+ --jobs job-id \
+ --tasks task-id1 task-id2
+To delete all running jobs for the current user:
+ ddel --provider google-v2 --project my-cloud-project --jobs '*'
+## Service Accounts and Scope (Google providers only)
+When you run the `dsub` command with the `google-v2` or `google-cls-v2`
+provider, there are two different sets of credentials to consider:
+- Account submitting the `pipelines.run()` request to run your command/script on a VM
+- Account accessing Cloud resources (such as files in GCS) when executing your command/script
+The account used to submit the `pipelines.run()` request is typically your
+end user credentials. You would have set this up by running:
+ gcloud auth application-default login
+The account used on the VM is a [service account](https://cloud.google.com/iam/docs/service-accounts).
+The image below illustrates this:
+![Pipelines Runner Architecture](./docs/images/pipelines_runner_architecture.png)
+By default, `dsub` will use the [default Compute Engine service account](https://cloud.google.com/compute/docs/access/service-accounts#default_service_account)
+as the authorized service account on the VM instance. You can choose to specify
+the email address of another service acount using `--service-account`.
+By default, `dsub` will grant the following access scopes to the service account:
+- https://www.googleapis.com/auth/bigquery
+- https://www.googleapis.com/auth/compute
+- https://www.googleapis.com/auth/devstorage.full_control
+- https://www.googleapis.com/auth/genomics
+- https://www.googleapis.com/auth/logging.write
+- https://www.googleapis.com/auth/monitoring.write
+In addition, [the API](https://cloud.google.com/life-sciences/docs/reference/rest/v2beta/projects.locations.pipelines/run#serviceaccount) will always add this scope:
+- https://www.googleapis.com/auth/cloud-platform
+You can choose to specify scopes using `--scopes`.
+### Recommendations for service accounts
+While it is straightforward to use the default service account, this account also
+has broad privileges granted to it by default. Following the
+[Principle of Least Privilege](https://en.wikipedia.org/wiki/Principle_of_least_privilege)
+you may want to create and use a service account that has only sufficient privileges
+granted in order to run your `dsub` command/script.
+To create a new service account, follow the steps below:
+1. Execute the `gcloud iam service-accounts create` command. The email address
+of the service account will be `sa-name@project-id.iam.gserviceaccount.com`.
+ gcloud iam service-accounts create "sa-name"
+2. Grant IAM access on buckets, etc. to the service account.
+ gsutil iam ch serviceAccount:sa-name@project-id.iam.gserviceaccount.com:roles/storage.objectAdmin gs://bucket-name
+3. Update your `dsub` command to include `--service-account`
+ dsub \
+ --service-account sa-name@project-id.iam.gserviceaccount.com
+## What next?
+* See the examples:
+ * [Custom scripts](https://github.com/DataBiosphere/dsub/tree/main/examples/custom_scripts)
+ * [Decompress files](https://github.com/DataBiosphere/dsub/tree/main/examples/decompress)
+ * [FastQC](https://github.com/DataBiosphere/dsub/tree/main/examples/fastqc)
+ * [Samtools index](https://github.com/DataBiosphere/dsub/tree/main/examples/samtools)
+* See more documentation for:
+ * [Scripts, Commands, and Docker](https://github.com/DataBiosphere/dsub/blob/main/docs/code.md)
+ * [Input and Output File Handling](https://github.com/DataBiosphere/dsub/blob/main/docs/input_output.md)
+ * [Logging](https://github.com/DataBiosphere/dsub/blob/main/docs/logging.md)
+ * [Compute Resources](https://github.com/DataBiosphere/dsub/blob/main/docs/compute_resources.md)
+ * [Compute Quotas](https://github.com/DataBiosphere/dsub/blob/main/docs/compute_quotas.md)
+ * [Job Control](https://github.com/DataBiosphere/dsub/blob/main/docs/job_control.md)
+ * [Retries](https://github.com/DataBiosphere/dsub/blob/main/docs/retries.md)
+ * [Checking Status and Troubleshooting Jobs](https://github.com/DataBiosphere/dsub/blob/main/docs/troubleshooting.md)
+ * [Backend providers](https://github.com/DataBiosphere/dsub/blob/main/docs/providers/README.md)
+
+%package -n python3-dsub
+Summary: A command-line tool that makes it easy to submit and run batch scripts in the cloud
+Provides: python-dsub
+BuildRequires: python3-devel
+BuildRequires: python3-setuptools
+BuildRequires: python3-pip
+%description -n python3-dsub
+my-job-name RUNNING 2
+my-job-name SUCCESS 1
+```
+In this mode, dstat prints one line per (job name, task status) pair. You can
+see at a glance how many tasks are finished, how many are still running, and
+how many are failed/canceled.
+### Deleting a job
+The `ddel` command will delete running jobs.
+By default, only jobs submitted by the current user will be deleted.
+Use the `--users` flag to specify other users, or `'*'` for all users.
+To delete a running job:
+ ddel --provider google-v2 --project my-cloud-project --jobs job-id
+If the job is a batch job, all running tasks will be deleted.
+To delete specific tasks:
+ ddel \
+ --provider google-v2 \
+ --project my-cloud-project \
+ --jobs job-id \
+ --tasks task-id1 task-id2
+To delete all running jobs for the current user:
+ ddel --provider google-v2 --project my-cloud-project --jobs '*'
+## Service Accounts and Scope (Google providers only)
+When you run the `dsub` command with the `google-v2` or `google-cls-v2`
+provider, there are two different sets of credentials to consider:
+- Account submitting the `pipelines.run()` request to run your command/script on a VM
+- Account accessing Cloud resources (such as files in GCS) when executing your command/script
+The account used to submit the `pipelines.run()` request is typically your
+end user credentials. You would have set this up by running:
+ gcloud auth application-default login
+The account used on the VM is a [service account](https://cloud.google.com/iam/docs/service-accounts).
+The image below illustrates this:
+![Pipelines Runner Architecture](./docs/images/pipelines_runner_architecture.png)
+By default, `dsub` will use the [default Compute Engine service account](https://cloud.google.com/compute/docs/access/service-accounts#default_service_account)
+as the authorized service account on the VM instance. You can choose to specify
+the email address of another service acount using `--service-account`.
+By default, `dsub` will grant the following access scopes to the service account:
+- https://www.googleapis.com/auth/bigquery
+- https://www.googleapis.com/auth/compute
+- https://www.googleapis.com/auth/devstorage.full_control
+- https://www.googleapis.com/auth/genomics
+- https://www.googleapis.com/auth/logging.write
+- https://www.googleapis.com/auth/monitoring.write
+In addition, [the API](https://cloud.google.com/life-sciences/docs/reference/rest/v2beta/projects.locations.pipelines/run#serviceaccount) will always add this scope:
+- https://www.googleapis.com/auth/cloud-platform
+You can choose to specify scopes using `--scopes`.
+### Recommendations for service accounts
+While it is straightforward to use the default service account, this account also
+has broad privileges granted to it by default. Following the
+[Principle of Least Privilege](https://en.wikipedia.org/wiki/Principle_of_least_privilege)
+you may want to create and use a service account that has only sufficient privileges
+granted in order to run your `dsub` command/script.
+To create a new service account, follow the steps below:
+1. Execute the `gcloud iam service-accounts create` command. The email address
+of the service account will be `sa-name@project-id.iam.gserviceaccount.com`.
+ gcloud iam service-accounts create "sa-name"
+2. Grant IAM access on buckets, etc. to the service account.
+ gsutil iam ch serviceAccount:sa-name@project-id.iam.gserviceaccount.com:roles/storage.objectAdmin gs://bucket-name
+3. Update your `dsub` command to include `--service-account`
+ dsub \
+ --service-account sa-name@project-id.iam.gserviceaccount.com
+## What next?
+* See the examples:
+ * [Custom scripts](https://github.com/DataBiosphere/dsub/tree/main/examples/custom_scripts)
+ * [Decompress files](https://github.com/DataBiosphere/dsub/tree/main/examples/decompress)
+ * [FastQC](https://github.com/DataBiosphere/dsub/tree/main/examples/fastqc)
+ * [Samtools index](https://github.com/DataBiosphere/dsub/tree/main/examples/samtools)
+* See more documentation for:
+ * [Scripts, Commands, and Docker](https://github.com/DataBiosphere/dsub/blob/main/docs/code.md)
+ * [Input and Output File Handling](https://github.com/DataBiosphere/dsub/blob/main/docs/input_output.md)
+ * [Logging](https://github.com/DataBiosphere/dsub/blob/main/docs/logging.md)
+ * [Compute Resources](https://github.com/DataBiosphere/dsub/blob/main/docs/compute_resources.md)
+ * [Compute Quotas](https://github.com/DataBiosphere/dsub/blob/main/docs/compute_quotas.md)
+ * [Job Control](https://github.com/DataBiosphere/dsub/blob/main/docs/job_control.md)
+ * [Retries](https://github.com/DataBiosphere/dsub/blob/main/docs/retries.md)
+ * [Checking Status and Troubleshooting Jobs](https://github.com/DataBiosphere/dsub/blob/main/docs/troubleshooting.md)
+ * [Backend providers](https://github.com/DataBiosphere/dsub/blob/main/docs/providers/README.md)
+
+%package help
+Summary: Development documents and examples for dsub
+Provides: python3-dsub-doc
+%description help
+my-job-name RUNNING 2
+my-job-name SUCCESS 1
+```
+In this mode, dstat prints one line per (job name, task status) pair. You can
+see at a glance how many tasks are finished, how many are still running, and
+how many are failed/canceled.
+### Deleting a job
+The `ddel` command will delete running jobs.
+By default, only jobs submitted by the current user will be deleted.
+Use the `--users` flag to specify other users, or `'*'` for all users.
+To delete a running job:
+ ddel --provider google-v2 --project my-cloud-project --jobs job-id
+If the job is a batch job, all running tasks will be deleted.
+To delete specific tasks:
+ ddel \
+ --provider google-v2 \
+ --project my-cloud-project \
+ --jobs job-id \
+ --tasks task-id1 task-id2
+To delete all running jobs for the current user:
+ ddel --provider google-v2 --project my-cloud-project --jobs '*'
+## Service Accounts and Scope (Google providers only)
+When you run the `dsub` command with the `google-v2` or `google-cls-v2`
+provider, there are two different sets of credentials to consider:
+- Account submitting the `pipelines.run()` request to run your command/script on a VM
+- Account accessing Cloud resources (such as files in GCS) when executing your command/script
+The account used to submit the `pipelines.run()` request is typically your
+end user credentials. You would have set this up by running:
+ gcloud auth application-default login
+The account used on the VM is a [service account](https://cloud.google.com/iam/docs/service-accounts).
+The image below illustrates this:
+![Pipelines Runner Architecture](./docs/images/pipelines_runner_architecture.png)
+By default, `dsub` will use the [default Compute Engine service account](https://cloud.google.com/compute/docs/access/service-accounts#default_service_account)
+as the authorized service account on the VM instance. You can choose to specify
+the email address of another service acount using `--service-account`.
+By default, `dsub` will grant the following access scopes to the service account:
+- https://www.googleapis.com/auth/bigquery
+- https://www.googleapis.com/auth/compute
+- https://www.googleapis.com/auth/devstorage.full_control
+- https://www.googleapis.com/auth/genomics
+- https://www.googleapis.com/auth/logging.write
+- https://www.googleapis.com/auth/monitoring.write
+In addition, [the API](https://cloud.google.com/life-sciences/docs/reference/rest/v2beta/projects.locations.pipelines/run#serviceaccount) will always add this scope:
+- https://www.googleapis.com/auth/cloud-platform
+You can choose to specify scopes using `--scopes`.
+### Recommendations for service accounts
+While it is straightforward to use the default service account, this account also
+has broad privileges granted to it by default. Following the
+[Principle of Least Privilege](https://en.wikipedia.org/wiki/Principle_of_least_privilege)
+you may want to create and use a service account that has only sufficient privileges
+granted in order to run your `dsub` command/script.
+To create a new service account, follow the steps below:
+1. Execute the `gcloud iam service-accounts create` command. The email address
+of the service account will be `sa-name@project-id.iam.gserviceaccount.com`.
+ gcloud iam service-accounts create "sa-name"
+2. Grant IAM access on buckets, etc. to the service account.
+ gsutil iam ch serviceAccount:sa-name@project-id.iam.gserviceaccount.com:roles/storage.objectAdmin gs://bucket-name
+3. Update your `dsub` command to include `--service-account`
+ dsub \
+ --service-account sa-name@project-id.iam.gserviceaccount.com
+## What next?
+* See the examples:
+ * [Custom scripts](https://github.com/DataBiosphere/dsub/tree/main/examples/custom_scripts)
+ * [Decompress files](https://github.com/DataBiosphere/dsub/tree/main/examples/decompress)
+ * [FastQC](https://github.com/DataBiosphere/dsub/tree/main/examples/fastqc)
+ * [Samtools index](https://github.com/DataBiosphere/dsub/tree/main/examples/samtools)
+* See more documentation for:
+ * [Scripts, Commands, and Docker](https://github.com/DataBiosphere/dsub/blob/main/docs/code.md)
+ * [Input and Output File Handling](https://github.com/DataBiosphere/dsub/blob/main/docs/input_output.md)
+ * [Logging](https://github.com/DataBiosphere/dsub/blob/main/docs/logging.md)
+ * [Compute Resources](https://github.com/DataBiosphere/dsub/blob/main/docs/compute_resources.md)
+ * [Compute Quotas](https://github.com/DataBiosphere/dsub/blob/main/docs/compute_quotas.md)
+ * [Job Control](https://github.com/DataBiosphere/dsub/blob/main/docs/job_control.md)
+ * [Retries](https://github.com/DataBiosphere/dsub/blob/main/docs/retries.md)
+ * [Checking Status and Troubleshooting Jobs](https://github.com/DataBiosphere/dsub/blob/main/docs/troubleshooting.md)
+ * [Backend providers](https://github.com/DataBiosphere/dsub/blob/main/docs/providers/README.md)
+
+%prep
+%autosetup -n dsub-0.4.9
+
+%build
+%py3_build
+
+%install
+%py3_install
+install -d -m755 %{buildroot}/%{_pkgdocdir}
+if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi
+if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi
+if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi
+if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi
+pushd %{buildroot}
+if [ -d usr/lib ]; then
+ find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/lib64 ]; then
+ find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/bin ]; then
+ find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/sbin ]; then
+ find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+touch doclist.lst
+if [ -d usr/share/man ]; then
+ find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst
+fi
+popd
+mv %{buildroot}/filelist.lst .
+mv %{buildroot}/doclist.lst .
+
+%files -n python3-dsub -f filelist.lst
+%dir %{python3_sitelib}/*
+
+%files help -f doclist.lst
+%{_docdir}/*
+
+%changelog
+* Wed May 31 2023 Python_Bot <Python_Bot@openeuler.org> - 0.4.9-1
+- Package Spec generated
generated by cgit v1.2.3 (git 2.25.1) at 2025-06-25 15:36:39 +0000