diff options
author | CoprDistGit <infra@openeuler.org> | 2023-05-31 07:41:12 +0000 |
---|---|---|
committer | CoprDistGit <infra@openeuler.org> | 2023-05-31 07:41:12 +0000 |
commit | 3fe1442e8b2b21dc415a4a438bb538f8deb59995 (patch) | |
tree | 132f190de4bee881ea98aa3bec69255e07f0de7a | |
parent | 0748f09a2a395b48322a130738bf9c4d8ade9acf (diff) |
automatic import of python-django-lightweight-queue
-rw-r--r-- | .gitignore | 1 | ||||
-rw-r--r-- | python-django-lightweight-queue.spec | 756 | ||||
-rw-r--r-- | sources | 1 |
3 files changed, 758 insertions, 0 deletions
@@ -0,0 +1 @@ +/django-lightweight-queue-4.11.0.tar.gz diff --git a/python-django-lightweight-queue.spec b/python-django-lightweight-queue.spec new file mode 100644 index 0000000..e24025b --- /dev/null +++ b/python-django-lightweight-queue.spec @@ -0,0 +1,756 @@ +%global _empty_manifest_terminate_build 0 +Name: python-django-lightweight-queue +Version: 4.11.0 +Release: 1 +Summary: Lightweight & modular queue and cron system for Django +License: BSD-3-Clause +URL: https://github.com/thread/django-lightweight-queue/ +Source0: https://mirrors.nju.edu.cn/pypi/web/packages/d1/ae/ee6ece695c0fe03105eb79ff1592bf17ca68ac2a8954fc41e08e76452fb1/django-lightweight-queue-4.11.0.tar.gz +BuildArch: noarch + +Requires: python3-django +Requires: python3-daemonize +Requires: python3-prometheus-client +Requires: python3-typing-extensions +Requires: python3-redis +Requires: python3-tqdm + +%description +# Django Lightweight Queue + +DLQ is a lightweight & modular queue and cron system for Django. It powers +millions of production jobs every day at Thread. + +## Installation + +```shell +pip install django-lightweight-queue[redis] +``` + +Currently the only production-ready backends are redis-based, so the `redis` +extra is essentially required. Additional non-redis backed production-ready +backends are great candidates for community contributions. + +## Basic Usage + +Start by adding `django_lightweight_queue` to your `INSTALLED_APPS`: + +```python +INSTALLED_APPS = [ + "django.contrib.admin", + "django.contrib.auth", + ..., + "django_lightweight_queue", +] +``` + +After that, define your task in any file you want: + +```python +import time +from django_lightweight_queue import task + +# Define a task +@task() +def long_running_task(first_arg, second_arg): + time.sleep(first_arg * second_arg) + +# Request that the task be executed at some point +long_running_task(4, second_arg=9) +``` + +See the docstring on the [`task`](django_lightweight_queue/task.py) decorator +for more details. + +## Configuration + +All automatically picked up configuration options begin with `LIGHTWEIGHT_QUEUE_` +and can be found in `app_settings.py`. They should be placed in the usual Django +settings files, for example: + +```python +LIGHTWEIGHT_QUEUE_BACKEND = 'django_lightweight_queue.backends.redis.RedisBackend' +``` + +#### Special Configuration + +If desired, specific configuration overrides can be placed in a standalone +python file which passed on the command line. This is useful for applying +customisations for specific servers. + +For example, given a `special.py` containing: + +```python +LIGHTWEIGHT_QUEUE_REDIS_PORT = 12345 +``` + +and then running: + +``` +$ python manage.py queue_runner --extra-settings=special.py +``` + +will result in the runner to use the settings from the specified configuration +file in preference to settings from the Django environment. Any settings not +present in the specified file are inherited from the global configuration. + +## Backends + +There are four built-in backends: + +### Synchronous (Development backend) + +`django_lightweight_queue.backends.synchronous.SynchronousBackend` + +Executes the task inline, without any actual queuing. + +### Redis (Production backend) + +`django_lightweight_queue.backends.redis.RedisBackend` + +Executes tasks at-most-once using [Redis][redis] for storage of the enqueued tasks. + +### Reliable Redis (Production backend) + +`django_lightweight_queue.backends.reliable_redis.ReliableRedisBackend` + +Executes tasks at-least-once using [Redis][redis] for storage of the enqueued tasks (subject to Redis consistency). Does not guarantee the task _completes_. + +### Debug Web (Debug backend) + +`django_lightweight_queue.backends.debug_web.DebugWebBackend` + +Instead of running jobs it prints the url to a view that can be used to run a task in a transaction which will be rolled back. This is useful for debugging and optimising tasks. + +Use this to append the appropriate URLs to the bottom of your root `urls.py`: + +```python +from django.conf import settings +from django.urls import path, include + +urlpatterns = [ + ... +] + +if settings.DEBUG: + urlpatterns += [ + path( + "", + include( + "django_lightweight_queue.urls", namespace="django-lightweight-queue" + ), + ) + ] +``` + +This backend may require an extra setting if your debug site is not on localhost: + +```python +# defaults to http://localhost:8000 +LIGHTWEIGHT_QUEUE_SITE_URL = "http://example.com:8000" +``` + +[redis]: https://redis.io/ + +## Running Workers + +The queue runner is implemented as a Django management command: + +``` +$ python manage.py queue_runner +``` + +Workers can be distributed over multiple hosts by telling each runner that it is +part of a pool: + +``` +$ python manage.py queue_runner --machine 2 --of 4 +``` + +Alternatively a runner can be told explicitly how to behave by having +extra settings loaded (any `LIGHTWEIGHT_QUEUE_*` constants found in the file +will replace equivalent django settings) and being configured to run exactly as +the settings describe: + +``` +$ python manage.py queue_runner --exact-configuration --extra-settings=special.py +``` + +When using `--exact-configuration` the number of workers is configured exactly, +rather than being treated as the configuration for a pool. Additionally, +exactly-configured runners will _not_ run any cron workers. + +#### Example + +Given a Django configuration containing: + +```python +LIGHTWEIGHT_QUEUE_WORKERS = { + 'queue1': 3, +} +``` + +and a `special.py` containing: + +```python +LIGHTWEIGHT_QUEUE_WORKERS = { + 'queue1': 2, +} +``` + +Running any of: + +``` +$ python manage.py queue_runner --machine 1 --of 3 # or, +$ python manage.py queue_runner --machine 2 --of 3 # or, +$ python manage.py queue_runner --machine 3 --of 3 +``` + +will result in one worker for `queue1` on the current machine, while: + +``` +$ python manage.py queue_runner --exact-configuration --extra-settings=special.py +``` + +will result in two workers on the current machine. + +## Cron Tasks + +DLQ supports the use of a cron-like specification of Django management commands +to be run at certain times. + +To specify that a management command should be run at a given time, place a +`cron.py` file in the root folder of the Django app which defines the command +and which contains a `CONFIG` variable: + +```python +CONFIG = ( + { + 'command': 'my_cron_command', + # Day values 1-7 to match datetime.datetime.utcnow().isoweekday() + 'days': '*', + 'hours': '*', + 'minutes': '*', + # Equivalent behaviour to the kwarg to `task` of the same name + 'sigkill_on_stop': True, + }, +) +``` + +## Maintainers + +This repository was created by [Chris Lamb](https://github.com/lamby) at +[Thread](https://www.thread.com/), and continues to be maintained by the [Thread +engineering team](https://github.com/thread). + + +%package -n python3-django-lightweight-queue +Summary: Lightweight & modular queue and cron system for Django +Provides: python-django-lightweight-queue +BuildRequires: python3-devel +BuildRequires: python3-setuptools +BuildRequires: python3-pip +%description -n python3-django-lightweight-queue +# Django Lightweight Queue + +DLQ is a lightweight & modular queue and cron system for Django. It powers +millions of production jobs every day at Thread. + +## Installation + +```shell +pip install django-lightweight-queue[redis] +``` + +Currently the only production-ready backends are redis-based, so the `redis` +extra is essentially required. Additional non-redis backed production-ready +backends are great candidates for community contributions. + +## Basic Usage + +Start by adding `django_lightweight_queue` to your `INSTALLED_APPS`: + +```python +INSTALLED_APPS = [ + "django.contrib.admin", + "django.contrib.auth", + ..., + "django_lightweight_queue", +] +``` + +After that, define your task in any file you want: + +```python +import time +from django_lightweight_queue import task + +# Define a task +@task() +def long_running_task(first_arg, second_arg): + time.sleep(first_arg * second_arg) + +# Request that the task be executed at some point +long_running_task(4, second_arg=9) +``` + +See the docstring on the [`task`](django_lightweight_queue/task.py) decorator +for more details. + +## Configuration + +All automatically picked up configuration options begin with `LIGHTWEIGHT_QUEUE_` +and can be found in `app_settings.py`. They should be placed in the usual Django +settings files, for example: + +```python +LIGHTWEIGHT_QUEUE_BACKEND = 'django_lightweight_queue.backends.redis.RedisBackend' +``` + +#### Special Configuration + +If desired, specific configuration overrides can be placed in a standalone +python file which passed on the command line. This is useful for applying +customisations for specific servers. + +For example, given a `special.py` containing: + +```python +LIGHTWEIGHT_QUEUE_REDIS_PORT = 12345 +``` + +and then running: + +``` +$ python manage.py queue_runner --extra-settings=special.py +``` + +will result in the runner to use the settings from the specified configuration +file in preference to settings from the Django environment. Any settings not +present in the specified file are inherited from the global configuration. + +## Backends + +There are four built-in backends: + +### Synchronous (Development backend) + +`django_lightweight_queue.backends.synchronous.SynchronousBackend` + +Executes the task inline, without any actual queuing. + +### Redis (Production backend) + +`django_lightweight_queue.backends.redis.RedisBackend` + +Executes tasks at-most-once using [Redis][redis] for storage of the enqueued tasks. + +### Reliable Redis (Production backend) + +`django_lightweight_queue.backends.reliable_redis.ReliableRedisBackend` + +Executes tasks at-least-once using [Redis][redis] for storage of the enqueued tasks (subject to Redis consistency). Does not guarantee the task _completes_. + +### Debug Web (Debug backend) + +`django_lightweight_queue.backends.debug_web.DebugWebBackend` + +Instead of running jobs it prints the url to a view that can be used to run a task in a transaction which will be rolled back. This is useful for debugging and optimising tasks. + +Use this to append the appropriate URLs to the bottom of your root `urls.py`: + +```python +from django.conf import settings +from django.urls import path, include + +urlpatterns = [ + ... +] + +if settings.DEBUG: + urlpatterns += [ + path( + "", + include( + "django_lightweight_queue.urls", namespace="django-lightweight-queue" + ), + ) + ] +``` + +This backend may require an extra setting if your debug site is not on localhost: + +```python +# defaults to http://localhost:8000 +LIGHTWEIGHT_QUEUE_SITE_URL = "http://example.com:8000" +``` + +[redis]: https://redis.io/ + +## Running Workers + +The queue runner is implemented as a Django management command: + +``` +$ python manage.py queue_runner +``` + +Workers can be distributed over multiple hosts by telling each runner that it is +part of a pool: + +``` +$ python manage.py queue_runner --machine 2 --of 4 +``` + +Alternatively a runner can be told explicitly how to behave by having +extra settings loaded (any `LIGHTWEIGHT_QUEUE_*` constants found in the file +will replace equivalent django settings) and being configured to run exactly as +the settings describe: + +``` +$ python manage.py queue_runner --exact-configuration --extra-settings=special.py +``` + +When using `--exact-configuration` the number of workers is configured exactly, +rather than being treated as the configuration for a pool. Additionally, +exactly-configured runners will _not_ run any cron workers. + +#### Example + +Given a Django configuration containing: + +```python +LIGHTWEIGHT_QUEUE_WORKERS = { + 'queue1': 3, +} +``` + +and a `special.py` containing: + +```python +LIGHTWEIGHT_QUEUE_WORKERS = { + 'queue1': 2, +} +``` + +Running any of: + +``` +$ python manage.py queue_runner --machine 1 --of 3 # or, +$ python manage.py queue_runner --machine 2 --of 3 # or, +$ python manage.py queue_runner --machine 3 --of 3 +``` + +will result in one worker for `queue1` on the current machine, while: + +``` +$ python manage.py queue_runner --exact-configuration --extra-settings=special.py +``` + +will result in two workers on the current machine. + +## Cron Tasks + +DLQ supports the use of a cron-like specification of Django management commands +to be run at certain times. + +To specify that a management command should be run at a given time, place a +`cron.py` file in the root folder of the Django app which defines the command +and which contains a `CONFIG` variable: + +```python +CONFIG = ( + { + 'command': 'my_cron_command', + # Day values 1-7 to match datetime.datetime.utcnow().isoweekday() + 'days': '*', + 'hours': '*', + 'minutes': '*', + # Equivalent behaviour to the kwarg to `task` of the same name + 'sigkill_on_stop': True, + }, +) +``` + +## Maintainers + +This repository was created by [Chris Lamb](https://github.com/lamby) at +[Thread](https://www.thread.com/), and continues to be maintained by the [Thread +engineering team](https://github.com/thread). + + +%package help +Summary: Development documents and examples for django-lightweight-queue +Provides: python3-django-lightweight-queue-doc +%description help +# Django Lightweight Queue + +DLQ is a lightweight & modular queue and cron system for Django. It powers +millions of production jobs every day at Thread. + +## Installation + +```shell +pip install django-lightweight-queue[redis] +``` + +Currently the only production-ready backends are redis-based, so the `redis` +extra is essentially required. Additional non-redis backed production-ready +backends are great candidates for community contributions. + +## Basic Usage + +Start by adding `django_lightweight_queue` to your `INSTALLED_APPS`: + +```python +INSTALLED_APPS = [ + "django.contrib.admin", + "django.contrib.auth", + ..., + "django_lightweight_queue", +] +``` + +After that, define your task in any file you want: + +```python +import time +from django_lightweight_queue import task + +# Define a task +@task() +def long_running_task(first_arg, second_arg): + time.sleep(first_arg * second_arg) + +# Request that the task be executed at some point +long_running_task(4, second_arg=9) +``` + +See the docstring on the [`task`](django_lightweight_queue/task.py) decorator +for more details. + +## Configuration + +All automatically picked up configuration options begin with `LIGHTWEIGHT_QUEUE_` +and can be found in `app_settings.py`. They should be placed in the usual Django +settings files, for example: + +```python +LIGHTWEIGHT_QUEUE_BACKEND = 'django_lightweight_queue.backends.redis.RedisBackend' +``` + +#### Special Configuration + +If desired, specific configuration overrides can be placed in a standalone +python file which passed on the command line. This is useful for applying +customisations for specific servers. + +For example, given a `special.py` containing: + +```python +LIGHTWEIGHT_QUEUE_REDIS_PORT = 12345 +``` + +and then running: + +``` +$ python manage.py queue_runner --extra-settings=special.py +``` + +will result in the runner to use the settings from the specified configuration +file in preference to settings from the Django environment. Any settings not +present in the specified file are inherited from the global configuration. + +## Backends + +There are four built-in backends: + +### Synchronous (Development backend) + +`django_lightweight_queue.backends.synchronous.SynchronousBackend` + +Executes the task inline, without any actual queuing. + +### Redis (Production backend) + +`django_lightweight_queue.backends.redis.RedisBackend` + +Executes tasks at-most-once using [Redis][redis] for storage of the enqueued tasks. + +### Reliable Redis (Production backend) + +`django_lightweight_queue.backends.reliable_redis.ReliableRedisBackend` + +Executes tasks at-least-once using [Redis][redis] for storage of the enqueued tasks (subject to Redis consistency). Does not guarantee the task _completes_. + +### Debug Web (Debug backend) + +`django_lightweight_queue.backends.debug_web.DebugWebBackend` + +Instead of running jobs it prints the url to a view that can be used to run a task in a transaction which will be rolled back. This is useful for debugging and optimising tasks. + +Use this to append the appropriate URLs to the bottom of your root `urls.py`: + +```python +from django.conf import settings +from django.urls import path, include + +urlpatterns = [ + ... +] + +if settings.DEBUG: + urlpatterns += [ + path( + "", + include( + "django_lightweight_queue.urls", namespace="django-lightweight-queue" + ), + ) + ] +``` + +This backend may require an extra setting if your debug site is not on localhost: + +```python +# defaults to http://localhost:8000 +LIGHTWEIGHT_QUEUE_SITE_URL = "http://example.com:8000" +``` + +[redis]: https://redis.io/ + +## Running Workers + +The queue runner is implemented as a Django management command: + +``` +$ python manage.py queue_runner +``` + +Workers can be distributed over multiple hosts by telling each runner that it is +part of a pool: + +``` +$ python manage.py queue_runner --machine 2 --of 4 +``` + +Alternatively a runner can be told explicitly how to behave by having +extra settings loaded (any `LIGHTWEIGHT_QUEUE_*` constants found in the file +will replace equivalent django settings) and being configured to run exactly as +the settings describe: + +``` +$ python manage.py queue_runner --exact-configuration --extra-settings=special.py +``` + +When using `--exact-configuration` the number of workers is configured exactly, +rather than being treated as the configuration for a pool. Additionally, +exactly-configured runners will _not_ run any cron workers. + +#### Example + +Given a Django configuration containing: + +```python +LIGHTWEIGHT_QUEUE_WORKERS = { + 'queue1': 3, +} +``` + +and a `special.py` containing: + +```python +LIGHTWEIGHT_QUEUE_WORKERS = { + 'queue1': 2, +} +``` + +Running any of: + +``` +$ python manage.py queue_runner --machine 1 --of 3 # or, +$ python manage.py queue_runner --machine 2 --of 3 # or, +$ python manage.py queue_runner --machine 3 --of 3 +``` + +will result in one worker for `queue1` on the current machine, while: + +``` +$ python manage.py queue_runner --exact-configuration --extra-settings=special.py +``` + +will result in two workers on the current machine. + +## Cron Tasks + +DLQ supports the use of a cron-like specification of Django management commands +to be run at certain times. + +To specify that a management command should be run at a given time, place a +`cron.py` file in the root folder of the Django app which defines the command +and which contains a `CONFIG` variable: + +```python +CONFIG = ( + { + 'command': 'my_cron_command', + # Day values 1-7 to match datetime.datetime.utcnow().isoweekday() + 'days': '*', + 'hours': '*', + 'minutes': '*', + # Equivalent behaviour to the kwarg to `task` of the same name + 'sigkill_on_stop': True, + }, +) +``` + +## Maintainers + +This repository was created by [Chris Lamb](https://github.com/lamby) at +[Thread](https://www.thread.com/), and continues to be maintained by the [Thread +engineering team](https://github.com/thread). + + +%prep +%autosetup -n django-lightweight-queue-4.11.0 + +%build +%py3_build + +%install +%py3_install +install -d -m755 %{buildroot}/%{_pkgdocdir} +if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi +if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi +if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi +if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi +pushd %{buildroot} +if [ -d usr/lib ]; then + find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/lib64 ]; then + find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/bin ]; then + find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/sbin ]; then + find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst +fi +touch doclist.lst +if [ -d usr/share/man ]; then + find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst +fi +popd +mv %{buildroot}/filelist.lst . +mv %{buildroot}/doclist.lst . + +%files -n python3-django-lightweight-queue -f filelist.lst +%dir %{python3_sitelib}/* + +%files help -f doclist.lst +%{_docdir}/* + +%changelog +* Wed May 31 2023 Python_Bot <Python_Bot@openeuler.org> - 4.11.0-1 +- Package Spec generated @@ -0,0 +1 @@ +f3cf56b4a2aa846108d057fbfbaf90a4 django-lightweight-queue-4.11.0.tar.gz |