%global _empty_manifest_terminate_build 0 Name: python-dsub Version: 0.4.9 Release: 1 Summary: A command-line tool that makes it easy to submit and run batch scripts in the cloud License: Apache URL: https://github.com/DataBiosphere/dsub Source0: https://mirrors.nju.edu.cn/pypi/web/packages/7a/36/6e2013482f87a93f4334525435c6a8e1050b028899d9054050bf62d2d23c/dsub-0.4.9.tar.gz BuildArch: noarch Requires: python3-funcsigs Requires: python3-google-api-core Requires: python3-google-api-python-client Requires: python3-google-auth-httplib2 Requires: python3-google-auth Requires: python3-google-cloud-batch Requires: python3-httplib2 Requires: python3-mock Requires: python3-parameterized Requires: python3-pyasn1-modules Requires: python3-pyasn1 Requires: python3-dateutil Requires: python3-pytz Requires: python3-pyyaml Requires: python3-rsa Requires: python3-tabulate Requires: python3-tenacity Requires: python3-uritemplate %description my-job-name RUNNING 2 my-job-name SUCCESS 1 ``` In this mode, dstat prints one line per (job name, task status) pair. You can see at a glance how many tasks are finished, how many are still running, and how many are failed/canceled. ### Deleting a job The `ddel` command will delete running jobs. By default, only jobs submitted by the current user will be deleted. Use the `--users` flag to specify other users, or `'*'` for all users. To delete a running job: ddel --provider google-v2 --project my-cloud-project --jobs job-id If the job is a batch job, all running tasks will be deleted. To delete specific tasks: ddel \ --provider google-v2 \ --project my-cloud-project \ --jobs job-id \ --tasks task-id1 task-id2 To delete all running jobs for the current user: ddel --provider google-v2 --project my-cloud-project --jobs '*' ## Service Accounts and Scope (Google providers only) When you run the `dsub` command with the `google-v2` or `google-cls-v2` provider, there are two different sets of credentials to consider: - Account submitting the `pipelines.run()` request to run your command/script on a VM - Account accessing Cloud resources (such as files in GCS) when executing your command/script The account used to submit the `pipelines.run()` request is typically your end user credentials. You would have set this up by running: gcloud auth application-default login The account used on the VM is a [service account](https://cloud.google.com/iam/docs/service-accounts). The image below illustrates this: ![Pipelines Runner Architecture](./docs/images/pipelines_runner_architecture.png) By default, `dsub` will use the [default Compute Engine service account](https://cloud.google.com/compute/docs/access/service-accounts#default_service_account) as the authorized service account on the VM instance. You can choose to specify the email address of another service acount using `--service-account`. By default, `dsub` will grant the following access scopes to the service account: - https://www.googleapis.com/auth/bigquery - https://www.googleapis.com/auth/compute - https://www.googleapis.com/auth/devstorage.full_control - https://www.googleapis.com/auth/genomics - https://www.googleapis.com/auth/logging.write - https://www.googleapis.com/auth/monitoring.write In addition, [the API](https://cloud.google.com/life-sciences/docs/reference/rest/v2beta/projects.locations.pipelines/run#serviceaccount) will always add this scope: - https://www.googleapis.com/auth/cloud-platform You can choose to specify scopes using `--scopes`. ### Recommendations for service accounts While it is straightforward to use the default service account, this account also has broad privileges granted to it by default. Following the [Principle of Least Privilege](https://en.wikipedia.org/wiki/Principle_of_least_privilege) you may want to create and use a service account that has only sufficient privileges granted in order to run your `dsub` command/script. To create a new service account, follow the steps below: 1. Execute the `gcloud iam service-accounts create` command. The email address of the service account will be `sa-name@project-id.iam.gserviceaccount.com`. gcloud iam service-accounts create "sa-name" 2. Grant IAM access on buckets, etc. to the service account. gsutil iam ch serviceAccount:sa-name@project-id.iam.gserviceaccount.com:roles/storage.objectAdmin gs://bucket-name 3. Update your `dsub` command to include `--service-account` dsub \ --service-account sa-name@project-id.iam.gserviceaccount.com ## What next? * See the examples: * [Custom scripts](https://github.com/DataBiosphere/dsub/tree/main/examples/custom_scripts) * [Decompress files](https://github.com/DataBiosphere/dsub/tree/main/examples/decompress) * [FastQC](https://github.com/DataBiosphere/dsub/tree/main/examples/fastqc) * [Samtools index](https://github.com/DataBiosphere/dsub/tree/main/examples/samtools) * See more documentation for: * [Scripts, Commands, and Docker](https://github.com/DataBiosphere/dsub/blob/main/docs/code.md) * [Input and Output File Handling](https://github.com/DataBiosphere/dsub/blob/main/docs/input_output.md) * [Logging](https://github.com/DataBiosphere/dsub/blob/main/docs/logging.md) * [Compute Resources](https://github.com/DataBiosphere/dsub/blob/main/docs/compute_resources.md) * [Compute Quotas](https://github.com/DataBiosphere/dsub/blob/main/docs/compute_quotas.md) * [Job Control](https://github.com/DataBiosphere/dsub/blob/main/docs/job_control.md) * [Retries](https://github.com/DataBiosphere/dsub/blob/main/docs/retries.md) * [Checking Status and Troubleshooting Jobs](https://github.com/DataBiosphere/dsub/blob/main/docs/troubleshooting.md) * [Backend providers](https://github.com/DataBiosphere/dsub/blob/main/docs/providers/README.md) %package -n python3-dsub Summary: A command-line tool that makes it easy to submit and run batch scripts in the cloud Provides: python-dsub BuildRequires: python3-devel BuildRequires: python3-setuptools BuildRequires: python3-pip %description -n python3-dsub my-job-name RUNNING 2 my-job-name SUCCESS 1 ``` In this mode, dstat prints one line per (job name, task status) pair. You can see at a glance how many tasks are finished, how many are still running, and how many are failed/canceled. ### Deleting a job The `ddel` command will delete running jobs. By default, only jobs submitted by the current user will be deleted. Use the `--users` flag to specify other users, or `'*'` for all users. To delete a running job: ddel --provider google-v2 --project my-cloud-project --jobs job-id If the job is a batch job, all running tasks will be deleted. To delete specific tasks: ddel \ --provider google-v2 \ --project my-cloud-project \ --jobs job-id \ --tasks task-id1 task-id2 To delete all running jobs for the current user: ddel --provider google-v2 --project my-cloud-project --jobs '*' ## Service Accounts and Scope (Google providers only) When you run the `dsub` command with the `google-v2` or `google-cls-v2` provider, there are two different sets of credentials to consider: - Account submitting the `pipelines.run()` request to run your command/script on a VM - Account accessing Cloud resources (such as files in GCS) when executing your command/script The account used to submit the `pipelines.run()` request is typically your end user credentials. You would have set this up by running: gcloud auth application-default login The account used on the VM is a [service account](https://cloud.google.com/iam/docs/service-accounts). The image below illustrates this: ![Pipelines Runner Architecture](./docs/images/pipelines_runner_architecture.png) By default, `dsub` will use the [default Compute Engine service account](https://cloud.google.com/compute/docs/access/service-accounts#default_service_account) as the authorized service account on the VM instance. You can choose to specify the email address of another service acount using `--service-account`. By default, `dsub` will grant the following access scopes to the service account: - https://www.googleapis.com/auth/bigquery - https://www.googleapis.com/auth/compute - https://www.googleapis.com/auth/devstorage.full_control - https://www.googleapis.com/auth/genomics - https://www.googleapis.com/auth/logging.write - https://www.googleapis.com/auth/monitoring.write In addition, [the API](https://cloud.google.com/life-sciences/docs/reference/rest/v2beta/projects.locations.pipelines/run#serviceaccount) will always add this scope: - https://www.googleapis.com/auth/cloud-platform You can choose to specify scopes using `--scopes`. ### Recommendations for service accounts While it is straightforward to use the default service account, this account also has broad privileges granted to it by default. Following the [Principle of Least Privilege](https://en.wikipedia.org/wiki/Principle_of_least_privilege) you may want to create and use a service account that has only sufficient privileges granted in order to run your `dsub` command/script. To create a new service account, follow the steps below: 1. Execute the `gcloud iam service-accounts create` command. The email address of the service account will be `sa-name@project-id.iam.gserviceaccount.com`. gcloud iam service-accounts create "sa-name" 2. Grant IAM access on buckets, etc. to the service account. gsutil iam ch serviceAccount:sa-name@project-id.iam.gserviceaccount.com:roles/storage.objectAdmin gs://bucket-name 3. Update your `dsub` command to include `--service-account` dsub \ --service-account sa-name@project-id.iam.gserviceaccount.com ## What next? * See the examples: * [Custom scripts](https://github.com/DataBiosphere/dsub/tree/main/examples/custom_scripts) * [Decompress files](https://github.com/DataBiosphere/dsub/tree/main/examples/decompress) * [FastQC](https://github.com/DataBiosphere/dsub/tree/main/examples/fastqc) * [Samtools index](https://github.com/DataBiosphere/dsub/tree/main/examples/samtools) * See more documentation for: * [Scripts, Commands, and Docker](https://github.com/DataBiosphere/dsub/blob/main/docs/code.md) * [Input and Output File Handling](https://github.com/DataBiosphere/dsub/blob/main/docs/input_output.md) * [Logging](https://github.com/DataBiosphere/dsub/blob/main/docs/logging.md) * [Compute Resources](https://github.com/DataBiosphere/dsub/blob/main/docs/compute_resources.md) * [Compute Quotas](https://github.com/DataBiosphere/dsub/blob/main/docs/compute_quotas.md) * [Job Control](https://github.com/DataBiosphere/dsub/blob/main/docs/job_control.md) * [Retries](https://github.com/DataBiosphere/dsub/blob/main/docs/retries.md) * [Checking Status and Troubleshooting Jobs](https://github.com/DataBiosphere/dsub/blob/main/docs/troubleshooting.md) * [Backend providers](https://github.com/DataBiosphere/dsub/blob/main/docs/providers/README.md) %package help Summary: Development documents and examples for dsub Provides: python3-dsub-doc %description help my-job-name RUNNING 2 my-job-name SUCCESS 1 ``` In this mode, dstat prints one line per (job name, task status) pair. You can see at a glance how many tasks are finished, how many are still running, and how many are failed/canceled. ### Deleting a job The `ddel` command will delete running jobs. By default, only jobs submitted by the current user will be deleted. Use the `--users` flag to specify other users, or `'*'` for all users. To delete a running job: ddel --provider google-v2 --project my-cloud-project --jobs job-id If the job is a batch job, all running tasks will be deleted. To delete specific tasks: ddel \ --provider google-v2 \ --project my-cloud-project \ --jobs job-id \ --tasks task-id1 task-id2 To delete all running jobs for the current user: ddel --provider google-v2 --project my-cloud-project --jobs '*' ## Service Accounts and Scope (Google providers only) When you run the `dsub` command with the `google-v2` or `google-cls-v2` provider, there are two different sets of credentials to consider: - Account submitting the `pipelines.run()` request to run your command/script on a VM - Account accessing Cloud resources (such as files in GCS) when executing your command/script The account used to submit the `pipelines.run()` request is typically your end user credentials. You would have set this up by running: gcloud auth application-default login The account used on the VM is a [service account](https://cloud.google.com/iam/docs/service-accounts). The image below illustrates this: ![Pipelines Runner Architecture](./docs/images/pipelines_runner_architecture.png) By default, `dsub` will use the [default Compute Engine service account](https://cloud.google.com/compute/docs/access/service-accounts#default_service_account) as the authorized service account on the VM instance. You can choose to specify the email address of another service acount using `--service-account`. By default, `dsub` will grant the following access scopes to the service account: - https://www.googleapis.com/auth/bigquery - https://www.googleapis.com/auth/compute - https://www.googleapis.com/auth/devstorage.full_control - https://www.googleapis.com/auth/genomics - https://www.googleapis.com/auth/logging.write - https://www.googleapis.com/auth/monitoring.write In addition, [the API](https://cloud.google.com/life-sciences/docs/reference/rest/v2beta/projects.locations.pipelines/run#serviceaccount) will always add this scope: - https://www.googleapis.com/auth/cloud-platform You can choose to specify scopes using `--scopes`. ### Recommendations for service accounts While it is straightforward to use the default service account, this account also has broad privileges granted to it by default. Following the [Principle of Least Privilege](https://en.wikipedia.org/wiki/Principle_of_least_privilege) you may want to create and use a service account that has only sufficient privileges granted in order to run your `dsub` command/script. To create a new service account, follow the steps below: 1. Execute the `gcloud iam service-accounts create` command. The email address of the service account will be `sa-name@project-id.iam.gserviceaccount.com`. gcloud iam service-accounts create "sa-name" 2. Grant IAM access on buckets, etc. to the service account. gsutil iam ch serviceAccount:sa-name@project-id.iam.gserviceaccount.com:roles/storage.objectAdmin gs://bucket-name 3. Update your `dsub` command to include `--service-account` dsub \ --service-account sa-name@project-id.iam.gserviceaccount.com ## What next? * See the examples: * [Custom scripts](https://github.com/DataBiosphere/dsub/tree/main/examples/custom_scripts) * [Decompress files](https://github.com/DataBiosphere/dsub/tree/main/examples/decompress) * [FastQC](https://github.com/DataBiosphere/dsub/tree/main/examples/fastqc) * [Samtools index](https://github.com/DataBiosphere/dsub/tree/main/examples/samtools) * See more documentation for: * [Scripts, Commands, and Docker](https://github.com/DataBiosphere/dsub/blob/main/docs/code.md) * [Input and Output File Handling](https://github.com/DataBiosphere/dsub/blob/main/docs/input_output.md) * [Logging](https://github.com/DataBiosphere/dsub/blob/main/docs/logging.md) * [Compute Resources](https://github.com/DataBiosphere/dsub/blob/main/docs/compute_resources.md) * [Compute Quotas](https://github.com/DataBiosphere/dsub/blob/main/docs/compute_quotas.md) * [Job Control](https://github.com/DataBiosphere/dsub/blob/main/docs/job_control.md) * [Retries](https://github.com/DataBiosphere/dsub/blob/main/docs/retries.md) * [Checking Status and Troubleshooting Jobs](https://github.com/DataBiosphere/dsub/blob/main/docs/troubleshooting.md) * [Backend providers](https://github.com/DataBiosphere/dsub/blob/main/docs/providers/README.md) %prep %autosetup -n dsub-0.4.9 %build %py3_build %install %py3_install install -d -m755 %{buildroot}/%{_pkgdocdir} if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi pushd %{buildroot} if [ -d usr/lib ]; then find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/lib64 ]; then find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/bin ]; then find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/sbin ]; then find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst fi touch doclist.lst if [ -d usr/share/man ]; then find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst fi popd mv %{buildroot}/filelist.lst . mv %{buildroot}/doclist.lst . %files -n python3-dsub -f filelist.lst %dir %{python3_sitelib}/* %files help -f doclist.lst %{_docdir}/* %changelog * Wed May 31 2023 Python_Bot - 0.4.9-1 - Package Spec generated