%global _empty_manifest_terminate_build 0 Name: python-jupyterhub-idle-culler Version: 1.2.1 Release: 1 Summary: please add a summary manually as the author left a blank one License: 3-BSD URL: https://jupyter.org Source0: https://mirrors.nju.edu.cn/pypi/web/packages/e8/be/f225c5d4cd186f793e46ababecee6ee62b2a57906a0ec1fc0453cda6c6ed/jupyterhub-idle-culler-1.2.1.tar.gz BuildArch: noarch Requires: python3-tornado Requires: python3-dateutil %description # JupyterHub Idle Culler Service [![GitHub Workflow Status - Test](https://img.shields.io/github/workflow/status/jupyterhub/jupyterhub-idle-culler/Test?logo=github&label=tests)](https://github.com/jupyterhub/jupyterhub-idle-culler/actions) [![Latest PyPI version](https://img.shields.io/pypi/v/jupyterhub-idle-culler?logo=pypi&logoColor=white)](https://pypi.python.org/pypi/jupyterhub-idle-culler) [![GitHub](https://img.shields.io/badge/issue_tracking-github-blue?logo=github)](https://github.com/jupyterhub/jupyterhub-idle-culler/issues) [![Discourse](https://img.shields.io/badge/help_forum-discourse-blue?logo=discourse)](https://discourse.jupyter.org/c/jupyterhub) [![Gitter](https://img.shields.io/badge/social_chat-gitter-blue?logo=gitter)](https://gitter.im/jupyterhub/jupyterhub) `jupyterhub-idle-culler` provides a JupyterHub service to identify and shut down idle or long-running Jupyter Notebook servers. The exact actions performed are dependent on the used spawner for the Jupyter Notebook server (e.g. the default [LocalProcessSpawner](https://jupyterhub.readthedocs.io/en/stable/api/spawner.html#localprocessspawner>), [kubespawner](https://github.com/jupyterhub/kubespawner), or [dockerspawner](https://github.com/jupyterhub/dockerspawner)). In addition, if explicitly requested, all users whose Jupyter Notebook servers have been shut down this way are deleted as JupyterHub users from the internal database. This neither affects the authentication method which continues to allow those users to log in nor does it delete persisted user data (e.g. stored in docker volumes for dockerspawner or in persisted volumes for kubespawner). ## Setup ### Installation ```bash pip install jupyterhub-idle-culler ``` ### Permissions Prior to JupyterHub 2.0, the `jupyterhub-idle-culler` required full administrative privileges, in order to have sufficient permissions to stop servers on behalf of users. JupyterHub 2.0 introduces [scopes][] to allow for more fine-grained permission control. This means that the configured culler service does not need full administrative privileges anymore. It can be assigned only the permissions it needs. [scopes]: https://jupyterhub.readthedocs.io/en/latest/rbac/scopes.html#available-scopes `jupyterhub-idle-culler` requires the following scopes to function: - `list:users` - access to the user list API, our source of information about who to cull - `read:users:activity` - read the last_activity field of the user - `delete:servers` - management of servers (this includes stopping servers) - `admin:users` (**optional**) - only needed if using `--cull-users` To assign the service the appropriate permissions, declare a role in your `jupyterhub_config.py`: ```python c.JupyterHub.load_roles = [ { "name": "jupyterhub-idle-culler-role", "scopes": [ "list:users", "read:users:activity", "delete:servers", # "admin:users", # if using --cull-users ], # assignment of role's permissions to: "services": ["jupyterhub-idle-culler-service"], } ] ``` ### As a hub managed service In `jupyterhub_config.py`, add the following dictionary for the idle-culler Service to the `c.JupyterHub.services` list: ```python c.JupyterHub.services = [ { "name": "jupyterhub-idle-culler-service", "command": [ sys.executable, "-m", "jupyterhub_idle_culler", "--timeout=3600", ], # "admin": True, } ] ``` where: - `"command"` indicates that the Service will be managed by the Hub, and - `"admin": True` grants admin permissions to this Service and is only meant for use with jupyterhub < 2.0; see [above][permissions]. ### As a standalone script `jupyterhub-idle-culler` can also be run as a standalone script. It can access the hub's api with a service token. Register the service token with JupyterHub in `jupyterhub_config.py`: ```python c.JupyterHub.services = [ { "name": "jupyterhub-idle-culler-service", "api_token": "...", # "admin": True, } ] ``` where: - `"api_token"` contains a secret token, e.g. generated by `openssl rand -hex 32`, and - `"admin": True` grants admin permissions to this Service and is only meant for use with jupyterhub < 2.0; see [above][permissions]. and store the same token in a `JUPYTERHUB_API_TOKEN` environment variable. Then start `jupyterhub-idle-culler` manually. ```bash export JUPYTERHUB_API_TOKEN=api_token_above... python3 -m jupyterhub-idle-culler [--timeout=900] [--url=http://localhost:8081/hub/api] ``` ## Command line flags ``` --api-page-size Number of users to request per page, when using JupyterHub 2.0's paginated user list API. Default: user the server-side default configured page size. (default 0) --concurrency Limit the number of concurrent requests made to the Hub. Deleting a lot of users at the same time can slow down the Hub, so limit the number of API requests we have outstanding at any given time. (default 10) --cull-admin-users Whether admin users should be culled (only if --cull-users=true). (default True) --cull-every The interval (in seconds) for checking for idle servers to cull. (default 0) --cull-users Cull users in addition to servers. This is for use in temporary-user cases such as tmpnb. (default False) --internal-certs-location The location of generated internal-ssl certificates (only needed with --ssl- enabled=true). (default internal-ssl) --max-age The maximum age (in seconds) of servers that should be culled even if they are active. (default 0) --remove-named-servers Remove named servers in addition to stopping them. This is useful for a BinderHub that uses authentication and named servers. (default False) --ssl-enabled Whether the Jupyter API endpoint has TLS enabled. (default False) --timeout The idle timeout (in seconds). (default 600) --url The JupyterHub API URL. ``` ## Caveats 1. last_activity is not updated with high frequency, so cull timeout should be greater than the sum of: - single-user websocket ping interval (default: 30 seconds) - `JupyterHub.last_activity_interval` (default: 5 minutes) 2. The same `--timeout` and `--max-age` values are used to cull users and users' servers. If you want a different value for users and servers, you should add this script to the services list twice, just with different `name`s, different values, and one with the `--cull-users` option. 3. By default HTTP requests to the hub timeout after 60 seconds. This can be changed by setting the `JUPYTERHUB_REQUEST_TIMEOUT` environment variable. ## How it works jupytehrub-idle-culler lists available users via JupyterHub's [/users][users-api] REST API. [users-api]: https://jupyterhub.readthedocs.io/en/stable/_static/rest-api/index.html#path--users jupyterhub-idle-culler culls user servers using JupyterHub's REST API ([/users/{name}/server](https://jupyterhub.readthedocs.io/en/stable/_static/rest-api/index.html#operation--users--name--server-delete) or [/users/{name}/servers/{server_name}](https://jupyterhub.readthedocs.io/en/stable/_static/rest-api/index.html#operation--users--name--servers--server_name--delete)), and makes the culling decisions based on its configuration and what JupyterHub reports about the user servers via its REST API [(/users)][users-api] where user servers' `last_activity` is reported back. The `last_activity` that JupyterHub reports is the most recent summary of information updated at a regular interval via the [`update_last_activity` function](https://github.com/jupyterhub/jupyterhub/blob/1.4.2/jupyterhub/app.py#L2646) that combines two sources of information. 1. **The proxy's routes data** The `update_last_activity` function will [ask the proxy](https://jupyterhub.readthedocs.io/en/stable/reference/proxy.html#retrieving-routes) for the active routes like `/user/user1` and collects associated `last_activity` data if it is available. This activity represents successfully proxies network traffic. `last_activity` data for routes will be available when using [configurable-http-proxy](https://github.com/jupyterhub/configurable-http-proxy#readme) as JupyterHub does by default, but if for example [traefik-proxy](https://github.com/jupyterhub/traefik-proxy#readme) is used as it is in the [TLJH distribution](https://tljh.jupyter.org), no such data will be available. 2. **The user server's activity reports** The `update_last_activity` function also reads JupyterHub's database that keeps state about servers `last_activity`. These database records are updated whenever a server notifies JupyterHub about activity, as they are responsible to do. Servers notify JupyterHub about activity by being started by the [`jupyterhub-singleuser`](https://github.com/jupyterhub/jupyterhub/blob/1.4.2/setup.py#L115) script that is made available by installing jupyterhub (or `jupyterhub-base` on conda-forge). The `jupyterhub-singleuser` script launches a modified server application that keeps JupyterHub updated with the server activity via the [`notify_activity`](https://github.com/jupyterhub/jupyterhub/blob/1.4.2/jupyterhub/singleuser/mixins.py#L497) function. The `notify_activity` function in turn make use of the server applications `last_activity` function (see implementation in [NotebookApp](https://github.com/jupyter/notebook/blob/v6.4.0/notebook/notebookapp.py#L392-L397) and [ServerApp](https://github.com/jupyter-server/jupyter_server/blob/v1.9.0/jupyter_server/serverapp.py#L375) respectively) that that combines information from API activity, kernel activity, kernel shutdown, and terminal activity. This activity also covers activity of applications like RStudio running via `jupyter-server-proxy`. Here is a summary of what's described so far: 1. jupyterhub-idle-culler culls servers via JupyterHub's REST API. 2. jupyterhub-idle-culler makes decisions based on information retrieved by JupyterHub REST API. 3. JupyterHub REST API reports information regularly updated by summarizing information gained by: asking the proxy about routes' activity, and by retaining activity information reported by the servers. Now, as the server's kernel activity influence the activity that servers will notify JupyterHub about, the kernel activity in turn influences jupyterhub-idle-culler. Due to this, it can be relevant to also learn a little about a mechanism to _cull idle kernels_ as well even though jupyterhub-idle-culler isn't involved in that. The default kernel manager, the MappingKernelManager, can be configured to cull idle kernels. Its configuration is documented in [NotebookApp's](https://jupyter-notebook.readthedocs.io/en/stable/config.html#options) and [ServerApp's](https://jupyter-server.readthedocs.io/en/latest/full-config.html) respective documentation, and here are some relevant kernel culling configuration options: - `MappingKernelManager.cull_busy` - `MappingKernelManager.cull_idle_timeout` - `MappingKernelManager.cull_interval` - `MappingKernelManager.cull_connected` Note that `cull_connected` can be tricky to understand for JupyterLab as a browser having a web-socket connection to a kernel or not isn't as obvious as it was in the classical Jupyter notebook UI. See [this issue for more details](https://github.com/jupyterlab/jupyterlab/issues/6893). Also note that configuration of MappingKernelManager should be made on the user server itself, for example via a `jupyter_notebook_config.py` file in `/etc/jupyter` or `/usr/local/etc/jupyter` rather than where JupyterHub is running. Finally, note that a Jupyter Notebook server can shut itself down without without intervention by jupyterhub-idle-culler if `NotebookApp.shutdown_no_activity_timeout` is configured. ### Caveats #### Pagination JupyterHub 2.0 introduces pagination to the [/users][users-api] API endpoint. This pagination does not guarantee a consistent snapshot for consecutive requests spread over time, so it is possible for a highly active hub to occasionally miss culling users crossing page boundaries between requests. This is expected to be an infrequent occurrence and only result in delaying a server being culled by one cull interval in realistic scenarios, so of minor consequence in JupyterHub. The issue can be mitigated by requesting a larger page size, via e.g. `--api-page-size=200`, but feel free to open an issue if this is causing a problem for you. %package -n python3-jupyterhub-idle-culler Summary: please add a summary manually as the author left a blank one Provides: python-jupyterhub-idle-culler BuildRequires: python3-devel BuildRequires: python3-setuptools BuildRequires: python3-pip %description -n python3-jupyterhub-idle-culler # JupyterHub Idle Culler Service [![GitHub Workflow Status - Test](https://img.shields.io/github/workflow/status/jupyterhub/jupyterhub-idle-culler/Test?logo=github&label=tests)](https://github.com/jupyterhub/jupyterhub-idle-culler/actions) [![Latest PyPI version](https://img.shields.io/pypi/v/jupyterhub-idle-culler?logo=pypi&logoColor=white)](https://pypi.python.org/pypi/jupyterhub-idle-culler) [![GitHub](https://img.shields.io/badge/issue_tracking-github-blue?logo=github)](https://github.com/jupyterhub/jupyterhub-idle-culler/issues) [![Discourse](https://img.shields.io/badge/help_forum-discourse-blue?logo=discourse)](https://discourse.jupyter.org/c/jupyterhub) [![Gitter](https://img.shields.io/badge/social_chat-gitter-blue?logo=gitter)](https://gitter.im/jupyterhub/jupyterhub) `jupyterhub-idle-culler` provides a JupyterHub service to identify and shut down idle or long-running Jupyter Notebook servers. The exact actions performed are dependent on the used spawner for the Jupyter Notebook server (e.g. the default [LocalProcessSpawner](https://jupyterhub.readthedocs.io/en/stable/api/spawner.html#localprocessspawner>), [kubespawner](https://github.com/jupyterhub/kubespawner), or [dockerspawner](https://github.com/jupyterhub/dockerspawner)). In addition, if explicitly requested, all users whose Jupyter Notebook servers have been shut down this way are deleted as JupyterHub users from the internal database. This neither affects the authentication method which continues to allow those users to log in nor does it delete persisted user data (e.g. stored in docker volumes for dockerspawner or in persisted volumes for kubespawner). ## Setup ### Installation ```bash pip install jupyterhub-idle-culler ``` ### Permissions Prior to JupyterHub 2.0, the `jupyterhub-idle-culler` required full administrative privileges, in order to have sufficient permissions to stop servers on behalf of users. JupyterHub 2.0 introduces [scopes][] to allow for more fine-grained permission control. This means that the configured culler service does not need full administrative privileges anymore. It can be assigned only the permissions it needs. [scopes]: https://jupyterhub.readthedocs.io/en/latest/rbac/scopes.html#available-scopes `jupyterhub-idle-culler` requires the following scopes to function: - `list:users` - access to the user list API, our source of information about who to cull - `read:users:activity` - read the last_activity field of the user - `delete:servers` - management of servers (this includes stopping servers) - `admin:users` (**optional**) - only needed if using `--cull-users` To assign the service the appropriate permissions, declare a role in your `jupyterhub_config.py`: ```python c.JupyterHub.load_roles = [ { "name": "jupyterhub-idle-culler-role", "scopes": [ "list:users", "read:users:activity", "delete:servers", # "admin:users", # if using --cull-users ], # assignment of role's permissions to: "services": ["jupyterhub-idle-culler-service"], } ] ``` ### As a hub managed service In `jupyterhub_config.py`, add the following dictionary for the idle-culler Service to the `c.JupyterHub.services` list: ```python c.JupyterHub.services = [ { "name": "jupyterhub-idle-culler-service", "command": [ sys.executable, "-m", "jupyterhub_idle_culler", "--timeout=3600", ], # "admin": True, } ] ``` where: - `"command"` indicates that the Service will be managed by the Hub, and - `"admin": True` grants admin permissions to this Service and is only meant for use with jupyterhub < 2.0; see [above][permissions]. ### As a standalone script `jupyterhub-idle-culler` can also be run as a standalone script. It can access the hub's api with a service token. Register the service token with JupyterHub in `jupyterhub_config.py`: ```python c.JupyterHub.services = [ { "name": "jupyterhub-idle-culler-service", "api_token": "...", # "admin": True, } ] ``` where: - `"api_token"` contains a secret token, e.g. generated by `openssl rand -hex 32`, and - `"admin": True` grants admin permissions to this Service and is only meant for use with jupyterhub < 2.0; see [above][permissions]. and store the same token in a `JUPYTERHUB_API_TOKEN` environment variable. Then start `jupyterhub-idle-culler` manually. ```bash export JUPYTERHUB_API_TOKEN=api_token_above... python3 -m jupyterhub-idle-culler [--timeout=900] [--url=http://localhost:8081/hub/api] ``` ## Command line flags ``` --api-page-size Number of users to request per page, when using JupyterHub 2.0's paginated user list API. Default: user the server-side default configured page size. (default 0) --concurrency Limit the number of concurrent requests made to the Hub. Deleting a lot of users at the same time can slow down the Hub, so limit the number of API requests we have outstanding at any given time. (default 10) --cull-admin-users Whether admin users should be culled (only if --cull-users=true). (default True) --cull-every The interval (in seconds) for checking for idle servers to cull. (default 0) --cull-users Cull users in addition to servers. This is for use in temporary-user cases such as tmpnb. (default False) --internal-certs-location The location of generated internal-ssl certificates (only needed with --ssl- enabled=true). (default internal-ssl) --max-age The maximum age (in seconds) of servers that should be culled even if they are active. (default 0) --remove-named-servers Remove named servers in addition to stopping them. This is useful for a BinderHub that uses authentication and named servers. (default False) --ssl-enabled Whether the Jupyter API endpoint has TLS enabled. (default False) --timeout The idle timeout (in seconds). (default 600) --url The JupyterHub API URL. ``` ## Caveats 1. last_activity is not updated with high frequency, so cull timeout should be greater than the sum of: - single-user websocket ping interval (default: 30 seconds) - `JupyterHub.last_activity_interval` (default: 5 minutes) 2. The same `--timeout` and `--max-age` values are used to cull users and users' servers. If you want a different value for users and servers, you should add this script to the services list twice, just with different `name`s, different values, and one with the `--cull-users` option. 3. By default HTTP requests to the hub timeout after 60 seconds. This can be changed by setting the `JUPYTERHUB_REQUEST_TIMEOUT` environment variable. ## How it works jupytehrub-idle-culler lists available users via JupyterHub's [/users][users-api] REST API. [users-api]: https://jupyterhub.readthedocs.io/en/stable/_static/rest-api/index.html#path--users jupyterhub-idle-culler culls user servers using JupyterHub's REST API ([/users/{name}/server](https://jupyterhub.readthedocs.io/en/stable/_static/rest-api/index.html#operation--users--name--server-delete) or [/users/{name}/servers/{server_name}](https://jupyterhub.readthedocs.io/en/stable/_static/rest-api/index.html#operation--users--name--servers--server_name--delete)), and makes the culling decisions based on its configuration and what JupyterHub reports about the user servers via its REST API [(/users)][users-api] where user servers' `last_activity` is reported back. The `last_activity` that JupyterHub reports is the most recent summary of information updated at a regular interval via the [`update_last_activity` function](https://github.com/jupyterhub/jupyterhub/blob/1.4.2/jupyterhub/app.py#L2646) that combines two sources of information. 1. **The proxy's routes data** The `update_last_activity` function will [ask the proxy](https://jupyterhub.readthedocs.io/en/stable/reference/proxy.html#retrieving-routes) for the active routes like `/user/user1` and collects associated `last_activity` data if it is available. This activity represents successfully proxies network traffic. `last_activity` data for routes will be available when using [configurable-http-proxy](https://github.com/jupyterhub/configurable-http-proxy#readme) as JupyterHub does by default, but if for example [traefik-proxy](https://github.com/jupyterhub/traefik-proxy#readme) is used as it is in the [TLJH distribution](https://tljh.jupyter.org), no such data will be available. 2. **The user server's activity reports** The `update_last_activity` function also reads JupyterHub's database that keeps state about servers `last_activity`. These database records are updated whenever a server notifies JupyterHub about activity, as they are responsible to do. Servers notify JupyterHub about activity by being started by the [`jupyterhub-singleuser`](https://github.com/jupyterhub/jupyterhub/blob/1.4.2/setup.py#L115) script that is made available by installing jupyterhub (or `jupyterhub-base` on conda-forge). The `jupyterhub-singleuser` script launches a modified server application that keeps JupyterHub updated with the server activity via the [`notify_activity`](https://github.com/jupyterhub/jupyterhub/blob/1.4.2/jupyterhub/singleuser/mixins.py#L497) function. The `notify_activity` function in turn make use of the server applications `last_activity` function (see implementation in [NotebookApp](https://github.com/jupyter/notebook/blob/v6.4.0/notebook/notebookapp.py#L392-L397) and [ServerApp](https://github.com/jupyter-server/jupyter_server/blob/v1.9.0/jupyter_server/serverapp.py#L375) respectively) that that combines information from API activity, kernel activity, kernel shutdown, and terminal activity. This activity also covers activity of applications like RStudio running via `jupyter-server-proxy`. Here is a summary of what's described so far: 1. jupyterhub-idle-culler culls servers via JupyterHub's REST API. 2. jupyterhub-idle-culler makes decisions based on information retrieved by JupyterHub REST API. 3. JupyterHub REST API reports information regularly updated by summarizing information gained by: asking the proxy about routes' activity, and by retaining activity information reported by the servers. Now, as the server's kernel activity influence the activity that servers will notify JupyterHub about, the kernel activity in turn influences jupyterhub-idle-culler. Due to this, it can be relevant to also learn a little about a mechanism to _cull idle kernels_ as well even though jupyterhub-idle-culler isn't involved in that. The default kernel manager, the MappingKernelManager, can be configured to cull idle kernels. Its configuration is documented in [NotebookApp's](https://jupyter-notebook.readthedocs.io/en/stable/config.html#options) and [ServerApp's](https://jupyter-server.readthedocs.io/en/latest/full-config.html) respective documentation, and here are some relevant kernel culling configuration options: - `MappingKernelManager.cull_busy` - `MappingKernelManager.cull_idle_timeout` - `MappingKernelManager.cull_interval` - `MappingKernelManager.cull_connected` Note that `cull_connected` can be tricky to understand for JupyterLab as a browser having a web-socket connection to a kernel or not isn't as obvious as it was in the classical Jupyter notebook UI. See [this issue for more details](https://github.com/jupyterlab/jupyterlab/issues/6893). Also note that configuration of MappingKernelManager should be made on the user server itself, for example via a `jupyter_notebook_config.py` file in `/etc/jupyter` or `/usr/local/etc/jupyter` rather than where JupyterHub is running. Finally, note that a Jupyter Notebook server can shut itself down without without intervention by jupyterhub-idle-culler if `NotebookApp.shutdown_no_activity_timeout` is configured. ### Caveats #### Pagination JupyterHub 2.0 introduces pagination to the [/users][users-api] API endpoint. This pagination does not guarantee a consistent snapshot for consecutive requests spread over time, so it is possible for a highly active hub to occasionally miss culling users crossing page boundaries between requests. This is expected to be an infrequent occurrence and only result in delaying a server being culled by one cull interval in realistic scenarios, so of minor consequence in JupyterHub. The issue can be mitigated by requesting a larger page size, via e.g. `--api-page-size=200`, but feel free to open an issue if this is causing a problem for you. %package help Summary: Development documents and examples for jupyterhub-idle-culler Provides: python3-jupyterhub-idle-culler-doc %description help # JupyterHub Idle Culler Service [![GitHub Workflow Status - Test](https://img.shields.io/github/workflow/status/jupyterhub/jupyterhub-idle-culler/Test?logo=github&label=tests)](https://github.com/jupyterhub/jupyterhub-idle-culler/actions) [![Latest PyPI version](https://img.shields.io/pypi/v/jupyterhub-idle-culler?logo=pypi&logoColor=white)](https://pypi.python.org/pypi/jupyterhub-idle-culler) [![GitHub](https://img.shields.io/badge/issue_tracking-github-blue?logo=github)](https://github.com/jupyterhub/jupyterhub-idle-culler/issues) [![Discourse](https://img.shields.io/badge/help_forum-discourse-blue?logo=discourse)](https://discourse.jupyter.org/c/jupyterhub) [![Gitter](https://img.shields.io/badge/social_chat-gitter-blue?logo=gitter)](https://gitter.im/jupyterhub/jupyterhub) `jupyterhub-idle-culler` provides a JupyterHub service to identify and shut down idle or long-running Jupyter Notebook servers. The exact actions performed are dependent on the used spawner for the Jupyter Notebook server (e.g. the default [LocalProcessSpawner](https://jupyterhub.readthedocs.io/en/stable/api/spawner.html#localprocessspawner>), [kubespawner](https://github.com/jupyterhub/kubespawner), or [dockerspawner](https://github.com/jupyterhub/dockerspawner)). In addition, if explicitly requested, all users whose Jupyter Notebook servers have been shut down this way are deleted as JupyterHub users from the internal database. This neither affects the authentication method which continues to allow those users to log in nor does it delete persisted user data (e.g. stored in docker volumes for dockerspawner or in persisted volumes for kubespawner). ## Setup ### Installation ```bash pip install jupyterhub-idle-culler ``` ### Permissions Prior to JupyterHub 2.0, the `jupyterhub-idle-culler` required full administrative privileges, in order to have sufficient permissions to stop servers on behalf of users. JupyterHub 2.0 introduces [scopes][] to allow for more fine-grained permission control. This means that the configured culler service does not need full administrative privileges anymore. It can be assigned only the permissions it needs. [scopes]: https://jupyterhub.readthedocs.io/en/latest/rbac/scopes.html#available-scopes `jupyterhub-idle-culler` requires the following scopes to function: - `list:users` - access to the user list API, our source of information about who to cull - `read:users:activity` - read the last_activity field of the user - `delete:servers` - management of servers (this includes stopping servers) - `admin:users` (**optional**) - only needed if using `--cull-users` To assign the service the appropriate permissions, declare a role in your `jupyterhub_config.py`: ```python c.JupyterHub.load_roles = [ { "name": "jupyterhub-idle-culler-role", "scopes": [ "list:users", "read:users:activity", "delete:servers", # "admin:users", # if using --cull-users ], # assignment of role's permissions to: "services": ["jupyterhub-idle-culler-service"], } ] ``` ### As a hub managed service In `jupyterhub_config.py`, add the following dictionary for the idle-culler Service to the `c.JupyterHub.services` list: ```python c.JupyterHub.services = [ { "name": "jupyterhub-idle-culler-service", "command": [ sys.executable, "-m", "jupyterhub_idle_culler", "--timeout=3600", ], # "admin": True, } ] ``` where: - `"command"` indicates that the Service will be managed by the Hub, and - `"admin": True` grants admin permissions to this Service and is only meant for use with jupyterhub < 2.0; see [above][permissions]. ### As a standalone script `jupyterhub-idle-culler` can also be run as a standalone script. It can access the hub's api with a service token. Register the service token with JupyterHub in `jupyterhub_config.py`: ```python c.JupyterHub.services = [ { "name": "jupyterhub-idle-culler-service", "api_token": "...", # "admin": True, } ] ``` where: - `"api_token"` contains a secret token, e.g. generated by `openssl rand -hex 32`, and - `"admin": True` grants admin permissions to this Service and is only meant for use with jupyterhub < 2.0; see [above][permissions]. and store the same token in a `JUPYTERHUB_API_TOKEN` environment variable. Then start `jupyterhub-idle-culler` manually. ```bash export JUPYTERHUB_API_TOKEN=api_token_above... python3 -m jupyterhub-idle-culler [--timeout=900] [--url=http://localhost:8081/hub/api] ``` ## Command line flags ``` --api-page-size Number of users to request per page, when using JupyterHub 2.0's paginated user list API. Default: user the server-side default configured page size. (default 0) --concurrency Limit the number of concurrent requests made to the Hub. Deleting a lot of users at the same time can slow down the Hub, so limit the number of API requests we have outstanding at any given time. (default 10) --cull-admin-users Whether admin users should be culled (only if --cull-users=true). (default True) --cull-every The interval (in seconds) for checking for idle servers to cull. (default 0) --cull-users Cull users in addition to servers. This is for use in temporary-user cases such as tmpnb. (default False) --internal-certs-location The location of generated internal-ssl certificates (only needed with --ssl- enabled=true). (default internal-ssl) --max-age The maximum age (in seconds) of servers that should be culled even if they are active. (default 0) --remove-named-servers Remove named servers in addition to stopping them. This is useful for a BinderHub that uses authentication and named servers. (default False) --ssl-enabled Whether the Jupyter API endpoint has TLS enabled. (default False) --timeout The idle timeout (in seconds). (default 600) --url The JupyterHub API URL. ``` ## Caveats 1. last_activity is not updated with high frequency, so cull timeout should be greater than the sum of: - single-user websocket ping interval (default: 30 seconds) - `JupyterHub.last_activity_interval` (default: 5 minutes) 2. The same `--timeout` and `--max-age` values are used to cull users and users' servers. If you want a different value for users and servers, you should add this script to the services list twice, just with different `name`s, different values, and one with the `--cull-users` option. 3. By default HTTP requests to the hub timeout after 60 seconds. This can be changed by setting the `JUPYTERHUB_REQUEST_TIMEOUT` environment variable. ## How it works jupytehrub-idle-culler lists available users via JupyterHub's [/users][users-api] REST API. [users-api]: https://jupyterhub.readthedocs.io/en/stable/_static/rest-api/index.html#path--users jupyterhub-idle-culler culls user servers using JupyterHub's REST API ([/users/{name}/server](https://jupyterhub.readthedocs.io/en/stable/_static/rest-api/index.html#operation--users--name--server-delete) or [/users/{name}/servers/{server_name}](https://jupyterhub.readthedocs.io/en/stable/_static/rest-api/index.html#operation--users--name--servers--server_name--delete)), and makes the culling decisions based on its configuration and what JupyterHub reports about the user servers via its REST API [(/users)][users-api] where user servers' `last_activity` is reported back. The `last_activity` that JupyterHub reports is the most recent summary of information updated at a regular interval via the [`update_last_activity` function](https://github.com/jupyterhub/jupyterhub/blob/1.4.2/jupyterhub/app.py#L2646) that combines two sources of information. 1. **The proxy's routes data** The `update_last_activity` function will [ask the proxy](https://jupyterhub.readthedocs.io/en/stable/reference/proxy.html#retrieving-routes) for the active routes like `/user/user1` and collects associated `last_activity` data if it is available. This activity represents successfully proxies network traffic. `last_activity` data for routes will be available when using [configurable-http-proxy](https://github.com/jupyterhub/configurable-http-proxy#readme) as JupyterHub does by default, but if for example [traefik-proxy](https://github.com/jupyterhub/traefik-proxy#readme) is used as it is in the [TLJH distribution](https://tljh.jupyter.org), no such data will be available. 2. **The user server's activity reports** The `update_last_activity` function also reads JupyterHub's database that keeps state about servers `last_activity`. These database records are updated whenever a server notifies JupyterHub about activity, as they are responsible to do. Servers notify JupyterHub about activity by being started by the [`jupyterhub-singleuser`](https://github.com/jupyterhub/jupyterhub/blob/1.4.2/setup.py#L115) script that is made available by installing jupyterhub (or `jupyterhub-base` on conda-forge). The `jupyterhub-singleuser` script launches a modified server application that keeps JupyterHub updated with the server activity via the [`notify_activity`](https://github.com/jupyterhub/jupyterhub/blob/1.4.2/jupyterhub/singleuser/mixins.py#L497) function. The `notify_activity` function in turn make use of the server applications `last_activity` function (see implementation in [NotebookApp](https://github.com/jupyter/notebook/blob/v6.4.0/notebook/notebookapp.py#L392-L397) and [ServerApp](https://github.com/jupyter-server/jupyter_server/blob/v1.9.0/jupyter_server/serverapp.py#L375) respectively) that that combines information from API activity, kernel activity, kernel shutdown, and terminal activity. This activity also covers activity of applications like RStudio running via `jupyter-server-proxy`. Here is a summary of what's described so far: 1. jupyterhub-idle-culler culls servers via JupyterHub's REST API. 2. jupyterhub-idle-culler makes decisions based on information retrieved by JupyterHub REST API. 3. JupyterHub REST API reports information regularly updated by summarizing information gained by: asking the proxy about routes' activity, and by retaining activity information reported by the servers. Now, as the server's kernel activity influence the activity that servers will notify JupyterHub about, the kernel activity in turn influences jupyterhub-idle-culler. Due to this, it can be relevant to also learn a little about a mechanism to _cull idle kernels_ as well even though jupyterhub-idle-culler isn't involved in that. The default kernel manager, the MappingKernelManager, can be configured to cull idle kernels. Its configuration is documented in [NotebookApp's](https://jupyter-notebook.readthedocs.io/en/stable/config.html#options) and [ServerApp's](https://jupyter-server.readthedocs.io/en/latest/full-config.html) respective documentation, and here are some relevant kernel culling configuration options: - `MappingKernelManager.cull_busy` - `MappingKernelManager.cull_idle_timeout` - `MappingKernelManager.cull_interval` - `MappingKernelManager.cull_connected` Note that `cull_connected` can be tricky to understand for JupyterLab as a browser having a web-socket connection to a kernel or not isn't as obvious as it was in the classical Jupyter notebook UI. See [this issue for more details](https://github.com/jupyterlab/jupyterlab/issues/6893). Also note that configuration of MappingKernelManager should be made on the user server itself, for example via a `jupyter_notebook_config.py` file in `/etc/jupyter` or `/usr/local/etc/jupyter` rather than where JupyterHub is running. Finally, note that a Jupyter Notebook server can shut itself down without without intervention by jupyterhub-idle-culler if `NotebookApp.shutdown_no_activity_timeout` is configured. ### Caveats #### Pagination JupyterHub 2.0 introduces pagination to the [/users][users-api] API endpoint. This pagination does not guarantee a consistent snapshot for consecutive requests spread over time, so it is possible for a highly active hub to occasionally miss culling users crossing page boundaries between requests. This is expected to be an infrequent occurrence and only result in delaying a server being culled by one cull interval in realistic scenarios, so of minor consequence in JupyterHub. The issue can be mitigated by requesting a larger page size, via e.g. `--api-page-size=200`, but feel free to open an issue if this is causing a problem for you. %prep %autosetup -n jupyterhub-idle-culler-1.2.1 %build %py3_build %install %py3_install install -d -m755 %{buildroot}/%{_pkgdocdir} if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi pushd %{buildroot} if [ -d usr/lib ]; then find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/lib64 ]; then find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/bin ]; then find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/sbin ]; then find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst fi touch doclist.lst if [ -d usr/share/man ]; then find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst fi popd mv %{buildroot}/filelist.lst . mv %{buildroot}/doclist.lst . %files -n python3-jupyterhub-idle-culler -f filelist.lst %dir %{python3_sitelib}/* %files help -f doclist.lst %{_docdir}/* %changelog * Sun Apr 23 2023 Python_Bot - 1.2.1-1 - Package Spec generated