%global _empty_manifest_terminate_build 0 Name: python-alex-ber-utils Version: 0.6.6 Release: 1 Summary: AlexBerUtils is collection of the small utilities License: Apache 2.0 URL: https://github.com/alex-ber/AlexBerUtils Source0: https://mirrors.nju.edu.cn/pypi/web/packages/1e/7f/0ad16bd3d077a14b0288d8aeb7e653bd3401fc4d45cfe364ec600fc6aed4/alex_ber_utils-0.6.6.tar.gz BuildArch: noarch Requires: python3-PyYAML Requires: python3-dotenv Requires: python3-fabric Requires: python3-multidispatch Requires: python3-attrs Requires: python3-mock Requires: python3-py Requires: python3-pytest Requires: python3-pytest-assume Requires: python3-pytest-mock Requires: python3-PyYAML Requires: python3-toml Requires: python3-pluggy Requires: python3-packaging Requires: python3-iniconfig Requires: python3-pyparsing Requires: python3-PyYAML Requires: python3-HiYaPyCo Requires: python3-Jinja2 Requires: python3-PyYAML Requires: python3-HiYaPyCo Requires: python3-Jinja2 %description ## AlexBerUtils AlexBerUtils is collection of the small utilities. See CHANGELOG.md for detail description. ### Getting Help ### QuickStart ```bash python3 -m pip install -U alex-ber-utils ``` ### Installing from Github ```bash python3 -m pip install -U https://github.com/alex-ber/AlexBerUtils/archive/master.zip ``` Optionally installing tests requirements. ```bash python3 -m pip install -U https://github.com/alex-ber/AlexBerUtils/archive/master.zip#egg=alex-ber-utils[tests] ``` Or explicitly: ```bash wget https://github.com/alex-ber/AlexBerUtils/archive/master.zip -O master.zip; unzip master.zip; rm master.zip ``` And then installing from source (see below). ### Installing from source ```bash python3 -m pip install . # only installs "required" ``` ```bash python3 -m pip install .[tests] # installs dependencies for tests ``` ```bash python3 -m pip install .[md] # installs multidispatcher (used in method_overloading_test.py) ``` ```bash python3 -m pip install .[fabric] # installs fabric (used in fabs.py) ``` ```bash python3 -m pip install .[yml] # installs Yml related dependencies # (used in ymlparsers.py, init_app_conf.py, deploys.py; # optionally used in ymlparsers_extra.py, emails.py) ``` ```bash python3 -m pip install .[env] # installs pydotenv (optionally used in deploys.py and mains.py) ``` #### Alternatively you install install from requirements file: ```bash python3 -m pip install -r requirements.txt # only installs "required" ``` ```bash python3 -m pip install -r requirements-tests.txt # installs dependencies for tests ``` ```bash python3 -m pip install -r requirements-md.txt # installs multidispatcher (used in method_overloading_test.py) ``` ```bash python3 -m pip install -r requirements-fabric.txt # installs fabric (used in fabs.py) ``` ```bash python3 -m pip install -r requirements-yml.txt # installs Yml related dependencies # (used in ymlparsers.py, init_app_conf.py, deploys.py; # optionally used in ymlparsers_extra.py, emails.py) ``` ```bash python3 -m pip install -r requirements-env.txt # installs pydotenv (optionally used in deploys.py) ``` ### Using Docker `alexberkovich/AlexBerUtils:latest` contains all `AlexBerUtils` dependencies. This Dockerfile is very simple, you can take relevant part for you and put them into your Dockerfile. ## Alternatively, you can use it as base Docker image for your project and add/upgrade another dependencies as you need. For example: ```Dockerfile FROM alexberkovich/alex_ber_utils:latest COPY requirements.txt etc/requirements.txt RUN set -ex && \ #latest pip,setuptools,wheel pip install --upgrade pip setuptools wheel && \ pip install alex_ber_utils pip install -r etc/requirements.txt CMD ["/bin/sh"] #CMD tail -f /dev/null ``` where `requirements.txt` is requirements for your project. ## From the directory with setup.py ```bash python3 setup.py test #run all tests ``` or ```bash pytest ``` ## Installing new version See https://docs.python.org/3.1/distutils/uploading.html ## Installing new version to venv ```bash python38 -m pip uninstall --yes alex_ber_utils python38 setup.py clean sdist bdist_wheel python38 -m pip install --find-links=./dist alex_ber_utils==0.6.5 ``` ##Manual upload ```bash #python setup.py clean sdist upload ``` ## Requirements AlexBerUtils requires the following modules. * Python 3.6+ * PyYAML==5.1 # Changelog All notable changes to this project will be documented in this file. \#https://pypi.org/manage/project/alex-ber-utils/releases/ ## Unreleased ## [0.6.6] - 13-06-2021 ### Added - `stdLogging` module. The main function is `initStream()`. This is Thin adapter layer that redirects stdout/stderr (or any other stream-like object) to standard Python's logger. Based on https://github.com/fx-kirin/py-stdlogging/blob/master/stdlogging.py See https://github.com/fx-kirin/py-stdlogging/pull/1 Quote from https://stackoverflow.com/questions/47325506/making-python-loggers-log-all-stdout-and-stderr-messages : "But be careful to capture stdout because it's very fragile". I decided to focus on redirecting stderr only to the logger. If you want you can also redirect stdout, by making 2 calls to initStream() package-level method. But, because of https://unix.stackexchange.com/questions/616616/separate-stdout-and-stderr-for-docker-run it is sufficient only to do it for stderr for me. See [https://alex-ber.medium.com/stdlogging-module-d5d69ff7103f] for details. #### Changed - `Doockerfiles` base-image. Now, you can transparentely switch betwee AMD64 to ARM 64 proccessor. - `cffi` dependency from 1.14.3 to 1.14.5. - `cryptography` dependency from 3.1.1 to 3.4.7. ## [0.6.5] - 12-04-2021 ### Added - `FixRelCwd` context-manager in `mains` module - This context-manager temporary changes current working directory to the one where relPackage is installed. What if you have some script or application that use relative path and you want to invoke in from another directory. To get things more complicated. maybe your “external” code also use *relative* path, but relative to another directory. See [https://alex-ber.medium.com/making-more-yo-relative-path-to-file-to-work-fbf6280f9511] for details. - `GuardedWorkerException` context-manager in `mains` module - context manager that mitigate exception propogation from another process. It is very difficult if not impossible to pickle exceptions back to the parent process. Simple ones work, but many others don’t. For example, CalledProcessError is not pickable (my guess, this is because of stdout, stderr data-members). This means, that if child process raise CalledProcessError that is not catched, it will propagate to the parent process, but the propagation will fail, apparently because of bug in Python itself. This cause *pool.join() to halt forever — and thus memory leak!* See [https://alex-ber.medium.com/exception-propagation-from-another-process-bb09894ba4ce] for details. - `join_files()` function in `files` module - Suppose, that you have some multi-threaded/multi-process application where each thread/process creates some file (each thread/process create different file) and you want to join them to one file. See [https://alex-ber.medium.com/join-files-cc5e38e3c658] for details. #### Changed - `fixabscwd()` function in `mains` module - minour refactoring - moving out some internal helper function for reuse in new function. - Base docker image version to alexberkovich/alpine-anaconda3:0.2.1-slim. alexberkovich/alpine-anaconda3:0.1.1 has some minor changes relative to alexberkovich/alpine-anaconda3:0.1.1. See [https://github.com/alex-ber/alpine-anaconda3/blob/master/CHANGELOG.md] for details. #### Updated ### Documentation - See [https://github.com/alex-ber/AlexBerUtils/issues/8] Config file from another directory is not resolved (using `argumentParser` with `--general.config.file` can't be passed to `init_app_conf.parse_config()`) ## [0.6.4] - 12/12/2020 #### Changed - Base docker image version to alexberkovich/alpine-anaconda3:0.1.1-slim. alexberkovich/alpine-anaconda3:0.1.1 has some minor changes relative to alexberkovich/alpine-anaconda3:0.1.0. See [https://github.com/alex-ber/alpine-anaconda3/blob/master/CHANGELOG.md] for details. alexberkovich/alpine-anaconda3:0.1.1-slim is "slim" version of the same docker image, most unused packaged are removed. - update versions to pip==20.3.1 setuptools==51.0.0 wheel==0.36.1 ### Removed - Script check_d.py ## [0.6.3] - 18/11/2020 #### Changed - Base docker image version to alexberkovich/alpine-anaconda3:0.1.0, it has fix for potential security risk: Git was changed not to store credential as plain text, but to keep them in memory for 1 hour, see https://git-scm.com/docs/git-credential-cache #### Updated ### Documentation - My `deploys` module [https://medium.com/analytics-vidhya/my-deploys-module-26c5599f1b15 for documentation] is updated to contain `fix_retry_env()` function in `mains` module. ### Added ### Documentation - `fix_retry_env()` function in `mains` module. [https://alex-ber.medium.com/make-path-to-file-on-windows-works-on-linux-402ed3624f66] ## [0.6.2] - 17/11/2020 ### Deprecation - `method_overloading_test.py` is deprecated and will be removed once AlexBerUtils will support Python 3.9. It will happen approximately at 01.11.2021. This test uses `multidispatch` project that wasn't updated since 2014. In Python 3.8 it has following warning: `multidispatch.py:163: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working from collections import MutableMapping` ### Added - class `OsEnvrionPathRetry`, function `fix_retry_env()` to `mains` module. ### Changed - `OsEnvrionPathExpender` - refactored, functionality is preserved. ## [0.6.1] - 16/11/2020 ### Added - optional Dockerfile - optional .env.docker for reference. - Support of Python 3.8 is validated, see https://github.com/alex-ber/AlexBerUtils/issues/5 - Email formatting changed in Python 3.8, see https://github.com/alex-ber/AlexBerUtils/issues/7 Note that it is possible that `7bit` will be replaced with `8bit` as `Content-Transfer-Encoding`, that I'm considering as ok. - `check_all.py` to run all unit test. - `check_d.py` for sanity test. - .dockerignore - requirements*.txt - dependencies version changed, see https://github.com/alex-ber/AlexBerUtils/issues/6 - Because of pytest upgrade `conftest.py` was changed: `pytest_configure()` was added to support dynamically used marks. - In `ymlparsers_test.py` deprecation warning removed (it will be error in Python 3.9) `collections.Mapping` was changed to `collections.abc.Mapping`. ### Changed - README.MD added section about Docker usage. - setup.py to indicate support of Python 3.8 ## [0.5.3] - 10/09/2020 ### Changed - `alexber.utils.emails.initConfig` is fixed. Before this default variables where ignored. - 2 Unit tests for `init_app_conf` are fixed. These fix are minors. ### Documentation - `importer` module [https://medium.com/analytics-vidhya/how-to-write-easily-customizable-code-8b00b43406b2] - `fixabscwd()` function in `mains` module. [https://medium.com/@alex_ber/making-relative-path-to-file-to-work-d5d0f1da67bf] - `fix_retry_env()` function in `mains` module. [https://alex-ber.medium.com/make-path-to-file-on-windows-works-on-linux-402ed3624f66] - My `parser` module [https://medium.com/analytics-vidhya/my-parser-module-429ed1457718] - My `ymlparsers` module [https://medium.com/analytics-vidhya/my-ymlparsers-module-88221edf16a6] - My major `init_app_conf` module [https://medium.com/analytics-vidhya/my-major-init-app-conf-module-1a5d9fb3998c] - My `deploys` module [https://medium.com/analytics-vidhya/my-deploys-module-26c5599f1b15] - My `emails` module [https://medium.com/analytics-vidhya/my-emails-module-3ad36a4861c5] - My `processinvokes` module [https://medium.com/analytics-vidhya/my-processinvokes-module-de4d301518df] ## [0.5.2] - 21/06/2020 ### Added - `path()` function in `mains` module. For older Python version uses `importlib_resources` module. For newer version built in `importlib.resources`. - `load_env()` function in `mains` module. Added kwargs forwarding. if dotenv_path or stream is present it will be used. if ENV_PCK is present, dotenv_path will be constructed from ENV_PCK and ENV_NAME. Otherwise, kwargs will be forwarded as is to load_dotenv. - `fix_env()` function in `mains` module. For each key in ENV_KEYS, this method prepends full_prefix to os.environ[key]. full_prefix is calculated as absolute path of `__init__.py` of ENV_PCK. ### Changed ``processinvokes`` function ```run_sub_process``` - documentation typo fixed. Lower Python version to 3.6. ## [0.5.1] - 06-05-2020 ### Added - `mains` module explanation article https://medium.com/@alex_ber/making-relative-path-to-file-to-work-d5d0f1da67bf is published. - `fabs` module. It adds cp method to fabric.Connection. This method is Linux-like cp command. It copies single file to remote (Posix) machine. - Spited dependency list for setup.py req.txt (inexact versions, direct dependency only) and for reproducible installation requirements.txt (exact versions, all, including transitive dependencies). - Added req-fabric.txt, requirements-fabric.txt - Fabric, used in `fabs` module. - Added req-yml.txt, requirements-yml.txt - Yml-related dependencies, used in ymlparsers.py and in init_app_conf.py, deploys.py; optionally used in ymlparsers_extra.py, emails.py. Main dependency is HiYaPyCo. I'm using feature that is availlable in the minimal version. HiYaPyCo depends upon PyYAML and Jinja2. Limitations for Jinja2 is from HiYaPyCo project. - Added req-env.txt, requirements-env.txt - pydotenv, optionally used in deploys.py. - Added `inspects.has_method`(cls, methodName). Check if class cls has method with name methodName directly, or in one of it's super-classes. - Added `pareser.parse_sys_args` function parses command line arguments. - Added `ymlparsers` module - `load`/`safe_dump` a Hierarchical Yml files. This is essentially wrapper arround HiYaPyCo project with streamlined and extended API and couple of work-arrounds. Note: this module doesn't use any package-level variables in hiYaPyCo module, including hiYaPyCo.jinja2env. This module do use Jinja2's `Environment`. It also has another defaults for `load`/`safe_dump` methods. They can be overridden in `initConfig()` function. `safe_dump()` method supports simple Python objects like primitive types (str, integer, etc), list, dict, **OrderedDict**. `as_str()`- convenient method for getting str representation of the data, for example of dict. `DisableVarSubst` - use of this context manager disables variable substation in the `load()` function. `initConfig` - this method reset some defaults. If running from the MainThread, this method is idempotent. - Added `init_app_conf` **major** module. The main function is `parse_config`. This function parses command line arguments first. Than it parse yml files. Command line arguments overrides yml files arguments. Parameters of yml files we always try to convert on best-effort basses. Parameters of system args we try convert according to `implicit_convert` param. If you supply `implicit_convert=True`, than `mask_value()` will be applied to the flat map (first parameter). Otherwise, `implicit_convert` wiil have the value that was set in `intiConfig()`. By default it is `True`. Command line key --general.profiles or appropriate key default yml file is used to find 'profiles'. Let suppose, that --config_file is resolved to config.yml. If 'profiles' is not empty, than it will be used to calculate filenames that will be used to override default yml file. Let suppose, 'profiles' resolved to ['dev', 'local']. Than first config.yml will be loaded, than it will be overridden with config-dev.yml, than it will be overridden with config-local.yml. At last, it will be overridden with system args. This entry can be always be overridden with system args. `ymlparsers` and `parser` modules serves as Low-Level API for this module. `mask_value()` implemented as a wrapper to `parsers.safe_eval()` method with support for boolean variables. This implementation is used to get type for arguments that we get from system args. This mechanism can be easily replaced with your own one. `to_convex_map()` This method receives dictionary with 'flat keys', it has simple key:value structure where value can't be another dictionary. It will return dictionary of dictionaries with natural key mapping, optionally, entries will be filtered out according to white_list_flat_keys and, optionally, value will be implicitly converted to appropriate type. In order to simulate dictionary of dictionaries 'flat keys' compose key from outer dict with key from inner dict separated with dot. For example, 'general.profiles' 'flat key' corresponds to convex map with 'general' key with dictionary as value that have one of the keys 'profiles' with corresponding value. If you supply `implicit_convert=True`, than `mask_value()` will be applied to the values of the received flat dictionary. Otherwise, `implicit_convert` wiil have the value that was set in `intiConfig()`. By default it is True. `merge_list_value_in_dicts` - merges value of 2 dicts. This value represents list of values. Value from flat_d is roughly obtained by flat_d[main_key+'.'+sub_key]. Value from d is roughly obtained by d[main_key][sub_key]. If you supply `implicit_convert=True`, than `mask_value()` will be applied to the flat map (first parameter). Otherwise, `implicit_convert` wiil have the value that was set in `intiConfig()`. By default it is `True`. `initConfig` - you can set default value of `implicit_convert`. By default it is `True`. This parameters is used if `implicit_convert` wasn't explicitly supplied. This method is idempotent. - Added `deploys` module. This module is usable in your deployment script. See also `fabs` module. This method use `parsers`, ymlparsers`, `init_app_conf` as it's low-level API. `init_app_conf` usage is limited. The main function is `load_config()`. It is simplified method for parsing yml configuration file with optionally overrided profiles only. See `init_app_conf.parse_config()` for another variant. `split_path` - Split filename in 2 part parts by split_dirname. first_part will ends with split_dirname. second_part will start immediately after split_dirname. `add_to_zip_copy_function` - Factory method that returns closure that can be used as copy_function param in `shutil.copytree()`. - Added `emails` module. This module contains extensions of the logging handlers. This module optionally depends on `ymlparseser` module. It is better to use `EmailStatus` context manager with configured `emailLogger`. It is intended to configure first your `emailLogger` with `OneMemoryHandler` (together with `SMTPHandler`). Than the code block that you want to aggregate log messages from is better to be enclosed with `EmailStatus` context manager. `alexber.utils.emails.SMTPHandler` is customization of `logging.handlers.SMTPHandler`. It's purpose is to connect to SMTP server and actually send the e-mail. Unlike `logging.handlers.SMTPHandler` this class expects for record.msg to be built EmailMessage. You can also change use of underline SMTP class to SMTP_SSL, LMTP, etc. This implementation is *thread-safe*. `alexber.utils.emails.OneMemoryHandler` is variant of `logging.handlers.MemoryHandler`. This handler aggregates log messages until `FINISHED` log-level is received or application is going to terminate abruptly (see docstring of `calc_abrupt_vars()` method for the details) and we have some log messages in the buffer. On such event all messages (in the current Thread) are aggregated to the single `EmailMessage`. The subject of the `EmailMessage` is determined by `get_subject()` method. If you want to change delimeters used to indicate variable declaration inside template, see docstring of the `get_subject()` method. It is better to use `EmailStatus` context manager with configured emailLogger. See docstring of `EmailStatus`. This implementation is *thread-safe*. `alexber.utils.emails.EmailStatus` - if contextmanager exits with exception (it fails), than e-mail with subject formatted with faildargs and faildkwargs will be send. Otherwise, e-mail with subject formatted with successargs and successkwargs will be send. All messages (in the current Thread) will be aggregated to one long e-mail with the subject described in `OneMemoryHandler.get_subject()` method. `alexber.utils.emails.initConfig` - this method reset some defaults. This method is idempotent. By default, `SMTP` class from `smtplib` is used to send actual e-mail. You can change it to `SMTP_SSL`, `LMTP`, or another class by specifying default_smpt_cls_name. You can also specified default port for sending e-mails. `processInvokes` module has one primary function - `run_sub_process()` This method run subprocess and logs it's out to the logger. This method is sophisticated decorator to `subprocess.run()`. It is useful, when your subprocess run's a lot of time and you're interesting to receive it's `stdout` and `stderr`. By default, it's streamed to log. You can easily customize this behavior, see `initConig()` method. `initConig()` This method can be optionally called prior any call to another function in this module. You can use your custom class for the logging. For example, FilePipe. ### Changed - Spited dependency list for setup.py req.txt (inexact versions, direct dependency only) and for reproducible installation requirements.txt (exact versions, all, including transitive dependencies). - README.md changed, added section 'Alternatively you install install from requirements file:'. Some other misc changed done. - CHANGELOG.md version 0.4.1 misc changes. - Misc improvement in unit tests. - Fixed `parser.safe_eval` - safe_eval('%(message)s') was blow up, now it returns value as is. See https://github.com/alex-ber/AlexBerUtils/issues/2 - Enhanced `importer.importer` - added support for PEP 420 (implicit Namespace Packages). Namespace packages are a mechanism for splitting a single Python package across multiple directories on disk. When interpreted encounter with non-empty __path__ attribute it adds modules found in those locations to the current package. See https://github.com/alex-ber/AlexBerUtils/issues/3 - In all documentation refference to `pip3` was changed to `python3 -m pip` ## [0.4.1] - 2020-04-02 **BREAKING CHANGE** I highly recommend not to use 0.3.X versions. ### Removed - module `warns` is droped ### Changed - *Limitation:*: `mains` module wasn't tested with frozen python script (frozen using py2exe). - module `mains` is rewritten. Function `initConf` is dropped entirely. - module `mains` now works with logger and with warnings (it was wrong decision to work with warnings). ## [0.3.4] - 2020-04-02 ### Changed - CHANGELOG.md fixed - `warns` module bug fixed, now warnings.warn() works. - FixabscwdWarning is added to simplify warnings disabling. - Changing how `mains` module use `warns`. ## [0.3.3] - 2020-04-02 ### Changed - CHANGELOG.md fixed ## [0.3.2] - 2020-04-01 ### Changed - To REAMDE.md add `Installing new version` section - Fix typo in REAMDE.md (tests, not test). - Fixing bug: now, you're able to import package in the Python interpreter (`setups.py` fixed) - Fixing bug: `warns` module now doesn't change log_level in the preconfigured logger in any cases. - **BREAKING CHANGE**: In`mains` module method `warnsInitConfig()` was renamed to `mainInitConfig()` Also singature was changed. - `mains` module minor refactored. ### Added - Unit tests are added for `warns` module - Unit tests are added for `mains` module ## [0.3.1] - 2020-04-01 ### Changed - Tests minor improvements. - Excluded tests, data from setup.py (from being installed from the sdist.) - Created MANIFEST.in ### Added - `warns `module is added: It provides better integration between warnings and logger. Unlike `logging._showwarning()` this variant will always go through logger. `warns.initConfig()` has optional file parameter (it's file-like object) to redirect warnings. Default value is `sys.stderr`. If logger for `log_name` (default is `py.warnings`) will be configured before call to `showwarning()` method, than warning will go to the logger's handler with `log_level` (default is `logging.WARNING`). If logger for `log_name` (default is `py.warnings`) willn't be configured before call to showwarning() method, than warning will be done to `file` (default is `sys.stderr`) with `log_level` (default is `logging.WARNING`). - `main` module is added: `main.fixabscwd()` changes `os.getcwd()` to be the directory of the `__main__` module. `main.warnsInitConfig()` reexports `warns.initConfig()` for convenience. ### Added - Tests for alexber.utils.thread_locals added. ## [0.2.5] - 2019-05-22 ### Changed - Fixed bug in UploadCommand, git push should be before git tag. ## [0.2.4] - 2019-05-22 ### Changed - Fixed bug in setup.py, incorrect order between VERSION and UploadCommand (no tag was created on upload) ## [0.2.1] - 2019-05-22 ### Changed - setup url fixed. - Added import of Enum to alexber.utils package. ## [0.2.0] - 2019-05-22 ### Changed - setup.py - keywords added. ## [0.1.1] - 2019-05-22 ### Changed - README.md fixed typo. ## [0.1.0] - 2019-05-22 ### Changed - alexber.utils.UploadCommand - bug fixed, failed on git tag, because VERSION was undefined. ## [0.0.1] - 2019-05-22 ### Added - alexber.utils.StrAsReprMixinEnum - Enum Mixin that has __str__() equal to __repr__(). - alexber.utils.AutoNameMixinEnum- Enum Mixin that generate value equal to the name. - alexber.utils.MissingNoneMixinEnum - Enum Mixin will return None if value will not be found. - alexber.utils.LookUpMixinEnum - Enim Mixin that is designed to be used for lookup by value. If lookup fail, None will be return. Also, __str__() will return the same value as __repr__(). - alexber.utils.threadlocal_var, get_threadlocal_var, del_threadlocal_var. Inspired by https://stackoverflow.com/questions/1408171/thread-local-storage-in-python - alexber.utils.UploadCommand - Support setup.py upload. UploadCommand is intented to be used only from setup.py It's builds Source and Wheel distribution. It's uploads the package to PyPI via Twine. It's pushes the git tags. - alexber.utils.uuid1mc is is a hybrid between version 1 & version 4. This is v1 with random MAC ("v1mc"). uuid1mc() is deliberately generating v1 UUIDs with a random broadcast MAC address. The resulting v1 UUID is time dependant (like regular v1), but lacks all host-specific information (like v4). Note: somebody reported that ran into trouble using UUID1 in Amazon EC2 instances. - alexber.utils.importer.importer - Convert str to Python construct that target is represented. - alexber.utils.importer.new_instance - Convert str to Python construct that target is represented. args and kwargs will be passed in to appropriate __new__() / __init__() / __init_subclass__() methods. - alexber.utils.inspects.issetdescriptor - Return true if the object is a method descriptor with setters. But not if ismethod() or isclass() or isfunction() are true. - alexber.utils.inspects.ismethod - Return false if object is not a class and not a function. Otherwise, return true iff signature has 2 params. - alexber.utils.parsers.safe_eval - The purpose of this function is convert numbers from str to correct type. This function support convertion of built-in Python number to correct type (int, float) This function doesn't support decimal.Decimal or datetime.datetime or numpy types. - alexber.utils.parsers.is_empty - if value is None returns True. if value is empty iterable (for example, empty str or emptry list),returns true otherwise false. Note: For not iterable values, behaivour is undefined. - alexber.utils.parsers.parse_boolean - if value is None returns None. if value is boolean, it is returned as it is. if value is str and value is equals ignoring case to "True", True is returned. if value is str and value is equals ignoring case to "False", False is returned. For every other value, the answer is undefined. - alexber.utils.props.Properties - A Python replacement for java.util.Properties class This is modelled as closely as possible to the Java original. Created - Anand B Pillai . Update to Python 3 by Alex. Also there are some tweeks that was done by Alex. %package -n python3-alex-ber-utils Summary: AlexBerUtils is collection of the small utilities Provides: python-alex-ber-utils BuildRequires: python3-devel BuildRequires: python3-setuptools BuildRequires: python3-pip %description -n python3-alex-ber-utils ## AlexBerUtils AlexBerUtils is collection of the small utilities. See CHANGELOG.md for detail description. ### Getting Help ### QuickStart ```bash python3 -m pip install -U alex-ber-utils ``` ### Installing from Github ```bash python3 -m pip install -U https://github.com/alex-ber/AlexBerUtils/archive/master.zip ``` Optionally installing tests requirements. ```bash python3 -m pip install -U https://github.com/alex-ber/AlexBerUtils/archive/master.zip#egg=alex-ber-utils[tests] ``` Or explicitly: ```bash wget https://github.com/alex-ber/AlexBerUtils/archive/master.zip -O master.zip; unzip master.zip; rm master.zip ``` And then installing from source (see below). ### Installing from source ```bash python3 -m pip install . # only installs "required" ``` ```bash python3 -m pip install .[tests] # installs dependencies for tests ``` ```bash python3 -m pip install .[md] # installs multidispatcher (used in method_overloading_test.py) ``` ```bash python3 -m pip install .[fabric] # installs fabric (used in fabs.py) ``` ```bash python3 -m pip install .[yml] # installs Yml related dependencies # (used in ymlparsers.py, init_app_conf.py, deploys.py; # optionally used in ymlparsers_extra.py, emails.py) ``` ```bash python3 -m pip install .[env] # installs pydotenv (optionally used in deploys.py and mains.py) ``` #### Alternatively you install install from requirements file: ```bash python3 -m pip install -r requirements.txt # only installs "required" ``` ```bash python3 -m pip install -r requirements-tests.txt # installs dependencies for tests ``` ```bash python3 -m pip install -r requirements-md.txt # installs multidispatcher (used in method_overloading_test.py) ``` ```bash python3 -m pip install -r requirements-fabric.txt # installs fabric (used in fabs.py) ``` ```bash python3 -m pip install -r requirements-yml.txt # installs Yml related dependencies # (used in ymlparsers.py, init_app_conf.py, deploys.py; # optionally used in ymlparsers_extra.py, emails.py) ``` ```bash python3 -m pip install -r requirements-env.txt # installs pydotenv (optionally used in deploys.py) ``` ### Using Docker `alexberkovich/AlexBerUtils:latest` contains all `AlexBerUtils` dependencies. This Dockerfile is very simple, you can take relevant part for you and put them into your Dockerfile. ## Alternatively, you can use it as base Docker image for your project and add/upgrade another dependencies as you need. For example: ```Dockerfile FROM alexberkovich/alex_ber_utils:latest COPY requirements.txt etc/requirements.txt RUN set -ex && \ #latest pip,setuptools,wheel pip install --upgrade pip setuptools wheel && \ pip install alex_ber_utils pip install -r etc/requirements.txt CMD ["/bin/sh"] #CMD tail -f /dev/null ``` where `requirements.txt` is requirements for your project. ## From the directory with setup.py ```bash python3 setup.py test #run all tests ``` or ```bash pytest ``` ## Installing new version See https://docs.python.org/3.1/distutils/uploading.html ## Installing new version to venv ```bash python38 -m pip uninstall --yes alex_ber_utils python38 setup.py clean sdist bdist_wheel python38 -m pip install --find-links=./dist alex_ber_utils==0.6.5 ``` ##Manual upload ```bash #python setup.py clean sdist upload ``` ## Requirements AlexBerUtils requires the following modules. * Python 3.6+ * PyYAML==5.1 # Changelog All notable changes to this project will be documented in this file. \#https://pypi.org/manage/project/alex-ber-utils/releases/ ## Unreleased ## [0.6.6] - 13-06-2021 ### Added - `stdLogging` module. The main function is `initStream()`. This is Thin adapter layer that redirects stdout/stderr (or any other stream-like object) to standard Python's logger. Based on https://github.com/fx-kirin/py-stdlogging/blob/master/stdlogging.py See https://github.com/fx-kirin/py-stdlogging/pull/1 Quote from https://stackoverflow.com/questions/47325506/making-python-loggers-log-all-stdout-and-stderr-messages : "But be careful to capture stdout because it's very fragile". I decided to focus on redirecting stderr only to the logger. If you want you can also redirect stdout, by making 2 calls to initStream() package-level method. But, because of https://unix.stackexchange.com/questions/616616/separate-stdout-and-stderr-for-docker-run it is sufficient only to do it for stderr for me. See [https://alex-ber.medium.com/stdlogging-module-d5d69ff7103f] for details. #### Changed - `Doockerfiles` base-image. Now, you can transparentely switch betwee AMD64 to ARM 64 proccessor. - `cffi` dependency from 1.14.3 to 1.14.5. - `cryptography` dependency from 3.1.1 to 3.4.7. ## [0.6.5] - 12-04-2021 ### Added - `FixRelCwd` context-manager in `mains` module - This context-manager temporary changes current working directory to the one where relPackage is installed. What if you have some script or application that use relative path and you want to invoke in from another directory. To get things more complicated. maybe your “external” code also use *relative* path, but relative to another directory. See [https://alex-ber.medium.com/making-more-yo-relative-path-to-file-to-work-fbf6280f9511] for details. - `GuardedWorkerException` context-manager in `mains` module - context manager that mitigate exception propogation from another process. It is very difficult if not impossible to pickle exceptions back to the parent process. Simple ones work, but many others don’t. For example, CalledProcessError is not pickable (my guess, this is because of stdout, stderr data-members). This means, that if child process raise CalledProcessError that is not catched, it will propagate to the parent process, but the propagation will fail, apparently because of bug in Python itself. This cause *pool.join() to halt forever — and thus memory leak!* See [https://alex-ber.medium.com/exception-propagation-from-another-process-bb09894ba4ce] for details. - `join_files()` function in `files` module - Suppose, that you have some multi-threaded/multi-process application where each thread/process creates some file (each thread/process create different file) and you want to join them to one file. See [https://alex-ber.medium.com/join-files-cc5e38e3c658] for details. #### Changed - `fixabscwd()` function in `mains` module - minour refactoring - moving out some internal helper function for reuse in new function. - Base docker image version to alexberkovich/alpine-anaconda3:0.2.1-slim. alexberkovich/alpine-anaconda3:0.1.1 has some minor changes relative to alexberkovich/alpine-anaconda3:0.1.1. See [https://github.com/alex-ber/alpine-anaconda3/blob/master/CHANGELOG.md] for details. #### Updated ### Documentation - See [https://github.com/alex-ber/AlexBerUtils/issues/8] Config file from another directory is not resolved (using `argumentParser` with `--general.config.file` can't be passed to `init_app_conf.parse_config()`) ## [0.6.4] - 12/12/2020 #### Changed - Base docker image version to alexberkovich/alpine-anaconda3:0.1.1-slim. alexberkovich/alpine-anaconda3:0.1.1 has some minor changes relative to alexberkovich/alpine-anaconda3:0.1.0. See [https://github.com/alex-ber/alpine-anaconda3/blob/master/CHANGELOG.md] for details. alexberkovich/alpine-anaconda3:0.1.1-slim is "slim" version of the same docker image, most unused packaged are removed. - update versions to pip==20.3.1 setuptools==51.0.0 wheel==0.36.1 ### Removed - Script check_d.py ## [0.6.3] - 18/11/2020 #### Changed - Base docker image version to alexberkovich/alpine-anaconda3:0.1.0, it has fix for potential security risk: Git was changed not to store credential as plain text, but to keep them in memory for 1 hour, see https://git-scm.com/docs/git-credential-cache #### Updated ### Documentation - My `deploys` module [https://medium.com/analytics-vidhya/my-deploys-module-26c5599f1b15 for documentation] is updated to contain `fix_retry_env()` function in `mains` module. ### Added ### Documentation - `fix_retry_env()` function in `mains` module. [https://alex-ber.medium.com/make-path-to-file-on-windows-works-on-linux-402ed3624f66] ## [0.6.2] - 17/11/2020 ### Deprecation - `method_overloading_test.py` is deprecated and will be removed once AlexBerUtils will support Python 3.9. It will happen approximately at 01.11.2021. This test uses `multidispatch` project that wasn't updated since 2014. In Python 3.8 it has following warning: `multidispatch.py:163: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working from collections import MutableMapping` ### Added - class `OsEnvrionPathRetry`, function `fix_retry_env()` to `mains` module. ### Changed - `OsEnvrionPathExpender` - refactored, functionality is preserved. ## [0.6.1] - 16/11/2020 ### Added - optional Dockerfile - optional .env.docker for reference. - Support of Python 3.8 is validated, see https://github.com/alex-ber/AlexBerUtils/issues/5 - Email formatting changed in Python 3.8, see https://github.com/alex-ber/AlexBerUtils/issues/7 Note that it is possible that `7bit` will be replaced with `8bit` as `Content-Transfer-Encoding`, that I'm considering as ok. - `check_all.py` to run all unit test. - `check_d.py` for sanity test. - .dockerignore - requirements*.txt - dependencies version changed, see https://github.com/alex-ber/AlexBerUtils/issues/6 - Because of pytest upgrade `conftest.py` was changed: `pytest_configure()` was added to support dynamically used marks. - In `ymlparsers_test.py` deprecation warning removed (it will be error in Python 3.9) `collections.Mapping` was changed to `collections.abc.Mapping`. ### Changed - README.MD added section about Docker usage. - setup.py to indicate support of Python 3.8 ## [0.5.3] - 10/09/2020 ### Changed - `alexber.utils.emails.initConfig` is fixed. Before this default variables where ignored. - 2 Unit tests for `init_app_conf` are fixed. These fix are minors. ### Documentation - `importer` module [https://medium.com/analytics-vidhya/how-to-write-easily-customizable-code-8b00b43406b2] - `fixabscwd()` function in `mains` module. [https://medium.com/@alex_ber/making-relative-path-to-file-to-work-d5d0f1da67bf] - `fix_retry_env()` function in `mains` module. [https://alex-ber.medium.com/make-path-to-file-on-windows-works-on-linux-402ed3624f66] - My `parser` module [https://medium.com/analytics-vidhya/my-parser-module-429ed1457718] - My `ymlparsers` module [https://medium.com/analytics-vidhya/my-ymlparsers-module-88221edf16a6] - My major `init_app_conf` module [https://medium.com/analytics-vidhya/my-major-init-app-conf-module-1a5d9fb3998c] - My `deploys` module [https://medium.com/analytics-vidhya/my-deploys-module-26c5599f1b15] - My `emails` module [https://medium.com/analytics-vidhya/my-emails-module-3ad36a4861c5] - My `processinvokes` module [https://medium.com/analytics-vidhya/my-processinvokes-module-de4d301518df] ## [0.5.2] - 21/06/2020 ### Added - `path()` function in `mains` module. For older Python version uses `importlib_resources` module. For newer version built in `importlib.resources`. - `load_env()` function in `mains` module. Added kwargs forwarding. if dotenv_path or stream is present it will be used. if ENV_PCK is present, dotenv_path will be constructed from ENV_PCK and ENV_NAME. Otherwise, kwargs will be forwarded as is to load_dotenv. - `fix_env()` function in `mains` module. For each key in ENV_KEYS, this method prepends full_prefix to os.environ[key]. full_prefix is calculated as absolute path of `__init__.py` of ENV_PCK. ### Changed ``processinvokes`` function ```run_sub_process``` - documentation typo fixed. Lower Python version to 3.6. ## [0.5.1] - 06-05-2020 ### Added - `mains` module explanation article https://medium.com/@alex_ber/making-relative-path-to-file-to-work-d5d0f1da67bf is published. - `fabs` module. It adds cp method to fabric.Connection. This method is Linux-like cp command. It copies single file to remote (Posix) machine. - Spited dependency list for setup.py req.txt (inexact versions, direct dependency only) and for reproducible installation requirements.txt (exact versions, all, including transitive dependencies). - Added req-fabric.txt, requirements-fabric.txt - Fabric, used in `fabs` module. - Added req-yml.txt, requirements-yml.txt - Yml-related dependencies, used in ymlparsers.py and in init_app_conf.py, deploys.py; optionally used in ymlparsers_extra.py, emails.py. Main dependency is HiYaPyCo. I'm using feature that is availlable in the minimal version. HiYaPyCo depends upon PyYAML and Jinja2. Limitations for Jinja2 is from HiYaPyCo project. - Added req-env.txt, requirements-env.txt - pydotenv, optionally used in deploys.py. - Added `inspects.has_method`(cls, methodName). Check if class cls has method with name methodName directly, or in one of it's super-classes. - Added `pareser.parse_sys_args` function parses command line arguments. - Added `ymlparsers` module - `load`/`safe_dump` a Hierarchical Yml files. This is essentially wrapper arround HiYaPyCo project with streamlined and extended API and couple of work-arrounds. Note: this module doesn't use any package-level variables in hiYaPyCo module, including hiYaPyCo.jinja2env. This module do use Jinja2's `Environment`. It also has another defaults for `load`/`safe_dump` methods. They can be overridden in `initConfig()` function. `safe_dump()` method supports simple Python objects like primitive types (str, integer, etc), list, dict, **OrderedDict**. `as_str()`- convenient method for getting str representation of the data, for example of dict. `DisableVarSubst` - use of this context manager disables variable substation in the `load()` function. `initConfig` - this method reset some defaults. If running from the MainThread, this method is idempotent. - Added `init_app_conf` **major** module. The main function is `parse_config`. This function parses command line arguments first. Than it parse yml files. Command line arguments overrides yml files arguments. Parameters of yml files we always try to convert on best-effort basses. Parameters of system args we try convert according to `implicit_convert` param. If you supply `implicit_convert=True`, than `mask_value()` will be applied to the flat map (first parameter). Otherwise, `implicit_convert` wiil have the value that was set in `intiConfig()`. By default it is `True`. Command line key --general.profiles or appropriate key default yml file is used to find 'profiles'. Let suppose, that --config_file is resolved to config.yml. If 'profiles' is not empty, than it will be used to calculate filenames that will be used to override default yml file. Let suppose, 'profiles' resolved to ['dev', 'local']. Than first config.yml will be loaded, than it will be overridden with config-dev.yml, than it will be overridden with config-local.yml. At last, it will be overridden with system args. This entry can be always be overridden with system args. `ymlparsers` and `parser` modules serves as Low-Level API for this module. `mask_value()` implemented as a wrapper to `parsers.safe_eval()` method with support for boolean variables. This implementation is used to get type for arguments that we get from system args. This mechanism can be easily replaced with your own one. `to_convex_map()` This method receives dictionary with 'flat keys', it has simple key:value structure where value can't be another dictionary. It will return dictionary of dictionaries with natural key mapping, optionally, entries will be filtered out according to white_list_flat_keys and, optionally, value will be implicitly converted to appropriate type. In order to simulate dictionary of dictionaries 'flat keys' compose key from outer dict with key from inner dict separated with dot. For example, 'general.profiles' 'flat key' corresponds to convex map with 'general' key with dictionary as value that have one of the keys 'profiles' with corresponding value. If you supply `implicit_convert=True`, than `mask_value()` will be applied to the values of the received flat dictionary. Otherwise, `implicit_convert` wiil have the value that was set in `intiConfig()`. By default it is True. `merge_list_value_in_dicts` - merges value of 2 dicts. This value represents list of values. Value from flat_d is roughly obtained by flat_d[main_key+'.'+sub_key]. Value from d is roughly obtained by d[main_key][sub_key]. If you supply `implicit_convert=True`, than `mask_value()` will be applied to the flat map (first parameter). Otherwise, `implicit_convert` wiil have the value that was set in `intiConfig()`. By default it is `True`. `initConfig` - you can set default value of `implicit_convert`. By default it is `True`. This parameters is used if `implicit_convert` wasn't explicitly supplied. This method is idempotent. - Added `deploys` module. This module is usable in your deployment script. See also `fabs` module. This method use `parsers`, ymlparsers`, `init_app_conf` as it's low-level API. `init_app_conf` usage is limited. The main function is `load_config()`. It is simplified method for parsing yml configuration file with optionally overrided profiles only. See `init_app_conf.parse_config()` for another variant. `split_path` - Split filename in 2 part parts by split_dirname. first_part will ends with split_dirname. second_part will start immediately after split_dirname. `add_to_zip_copy_function` - Factory method that returns closure that can be used as copy_function param in `shutil.copytree()`. - Added `emails` module. This module contains extensions of the logging handlers. This module optionally depends on `ymlparseser` module. It is better to use `EmailStatus` context manager with configured `emailLogger`. It is intended to configure first your `emailLogger` with `OneMemoryHandler` (together with `SMTPHandler`). Than the code block that you want to aggregate log messages from is better to be enclosed with `EmailStatus` context manager. `alexber.utils.emails.SMTPHandler` is customization of `logging.handlers.SMTPHandler`. It's purpose is to connect to SMTP server and actually send the e-mail. Unlike `logging.handlers.SMTPHandler` this class expects for record.msg to be built EmailMessage. You can also change use of underline SMTP class to SMTP_SSL, LMTP, etc. This implementation is *thread-safe*. `alexber.utils.emails.OneMemoryHandler` is variant of `logging.handlers.MemoryHandler`. This handler aggregates log messages until `FINISHED` log-level is received or application is going to terminate abruptly (see docstring of `calc_abrupt_vars()` method for the details) and we have some log messages in the buffer. On such event all messages (in the current Thread) are aggregated to the single `EmailMessage`. The subject of the `EmailMessage` is determined by `get_subject()` method. If you want to change delimeters used to indicate variable declaration inside template, see docstring of the `get_subject()` method. It is better to use `EmailStatus` context manager with configured emailLogger. See docstring of `EmailStatus`. This implementation is *thread-safe*. `alexber.utils.emails.EmailStatus` - if contextmanager exits with exception (it fails), than e-mail with subject formatted with faildargs and faildkwargs will be send. Otherwise, e-mail with subject formatted with successargs and successkwargs will be send. All messages (in the current Thread) will be aggregated to one long e-mail with the subject described in `OneMemoryHandler.get_subject()` method. `alexber.utils.emails.initConfig` - this method reset some defaults. This method is idempotent. By default, `SMTP` class from `smtplib` is used to send actual e-mail. You can change it to `SMTP_SSL`, `LMTP`, or another class by specifying default_smpt_cls_name. You can also specified default port for sending e-mails. `processInvokes` module has one primary function - `run_sub_process()` This method run subprocess and logs it's out to the logger. This method is sophisticated decorator to `subprocess.run()`. It is useful, when your subprocess run's a lot of time and you're interesting to receive it's `stdout` and `stderr`. By default, it's streamed to log. You can easily customize this behavior, see `initConig()` method. `initConig()` This method can be optionally called prior any call to another function in this module. You can use your custom class for the logging. For example, FilePipe. ### Changed - Spited dependency list for setup.py req.txt (inexact versions, direct dependency only) and for reproducible installation requirements.txt (exact versions, all, including transitive dependencies). - README.md changed, added section 'Alternatively you install install from requirements file:'. Some other misc changed done. - CHANGELOG.md version 0.4.1 misc changes. - Misc improvement in unit tests. - Fixed `parser.safe_eval` - safe_eval('%(message)s') was blow up, now it returns value as is. See https://github.com/alex-ber/AlexBerUtils/issues/2 - Enhanced `importer.importer` - added support for PEP 420 (implicit Namespace Packages). Namespace packages are a mechanism for splitting a single Python package across multiple directories on disk. When interpreted encounter with non-empty __path__ attribute it adds modules found in those locations to the current package. See https://github.com/alex-ber/AlexBerUtils/issues/3 - In all documentation refference to `pip3` was changed to `python3 -m pip` ## [0.4.1] - 2020-04-02 **BREAKING CHANGE** I highly recommend not to use 0.3.X versions. ### Removed - module `warns` is droped ### Changed - *Limitation:*: `mains` module wasn't tested with frozen python script (frozen using py2exe). - module `mains` is rewritten. Function `initConf` is dropped entirely. - module `mains` now works with logger and with warnings (it was wrong decision to work with warnings). ## [0.3.4] - 2020-04-02 ### Changed - CHANGELOG.md fixed - `warns` module bug fixed, now warnings.warn() works. - FixabscwdWarning is added to simplify warnings disabling. - Changing how `mains` module use `warns`. ## [0.3.3] - 2020-04-02 ### Changed - CHANGELOG.md fixed ## [0.3.2] - 2020-04-01 ### Changed - To REAMDE.md add `Installing new version` section - Fix typo in REAMDE.md (tests, not test). - Fixing bug: now, you're able to import package in the Python interpreter (`setups.py` fixed) - Fixing bug: `warns` module now doesn't change log_level in the preconfigured logger in any cases. - **BREAKING CHANGE**: In`mains` module method `warnsInitConfig()` was renamed to `mainInitConfig()` Also singature was changed. - `mains` module minor refactored. ### Added - Unit tests are added for `warns` module - Unit tests are added for `mains` module ## [0.3.1] - 2020-04-01 ### Changed - Tests minor improvements. - Excluded tests, data from setup.py (from being installed from the sdist.) - Created MANIFEST.in ### Added - `warns `module is added: It provides better integration between warnings and logger. Unlike `logging._showwarning()` this variant will always go through logger. `warns.initConfig()` has optional file parameter (it's file-like object) to redirect warnings. Default value is `sys.stderr`. If logger for `log_name` (default is `py.warnings`) will be configured before call to `showwarning()` method, than warning will go to the logger's handler with `log_level` (default is `logging.WARNING`). If logger for `log_name` (default is `py.warnings`) willn't be configured before call to showwarning() method, than warning will be done to `file` (default is `sys.stderr`) with `log_level` (default is `logging.WARNING`). - `main` module is added: `main.fixabscwd()` changes `os.getcwd()` to be the directory of the `__main__` module. `main.warnsInitConfig()` reexports `warns.initConfig()` for convenience. ### Added - Tests for alexber.utils.thread_locals added. ## [0.2.5] - 2019-05-22 ### Changed - Fixed bug in UploadCommand, git push should be before git tag. ## [0.2.4] - 2019-05-22 ### Changed - Fixed bug in setup.py, incorrect order between VERSION and UploadCommand (no tag was created on upload) ## [0.2.1] - 2019-05-22 ### Changed - setup url fixed. - Added import of Enum to alexber.utils package. ## [0.2.0] - 2019-05-22 ### Changed - setup.py - keywords added. ## [0.1.1] - 2019-05-22 ### Changed - README.md fixed typo. ## [0.1.0] - 2019-05-22 ### Changed - alexber.utils.UploadCommand - bug fixed, failed on git tag, because VERSION was undefined. ## [0.0.1] - 2019-05-22 ### Added - alexber.utils.StrAsReprMixinEnum - Enum Mixin that has __str__() equal to __repr__(). - alexber.utils.AutoNameMixinEnum- Enum Mixin that generate value equal to the name. - alexber.utils.MissingNoneMixinEnum - Enum Mixin will return None if value will not be found. - alexber.utils.LookUpMixinEnum - Enim Mixin that is designed to be used for lookup by value. If lookup fail, None will be return. Also, __str__() will return the same value as __repr__(). - alexber.utils.threadlocal_var, get_threadlocal_var, del_threadlocal_var. Inspired by https://stackoverflow.com/questions/1408171/thread-local-storage-in-python - alexber.utils.UploadCommand - Support setup.py upload. UploadCommand is intented to be used only from setup.py It's builds Source and Wheel distribution. It's uploads the package to PyPI via Twine. It's pushes the git tags. - alexber.utils.uuid1mc is is a hybrid between version 1 & version 4. This is v1 with random MAC ("v1mc"). uuid1mc() is deliberately generating v1 UUIDs with a random broadcast MAC address. The resulting v1 UUID is time dependant (like regular v1), but lacks all host-specific information (like v4). Note: somebody reported that ran into trouble using UUID1 in Amazon EC2 instances. - alexber.utils.importer.importer - Convert str to Python construct that target is represented. - alexber.utils.importer.new_instance - Convert str to Python construct that target is represented. args and kwargs will be passed in to appropriate __new__() / __init__() / __init_subclass__() methods. - alexber.utils.inspects.issetdescriptor - Return true if the object is a method descriptor with setters. But not if ismethod() or isclass() or isfunction() are true. - alexber.utils.inspects.ismethod - Return false if object is not a class and not a function. Otherwise, return true iff signature has 2 params. - alexber.utils.parsers.safe_eval - The purpose of this function is convert numbers from str to correct type. This function support convertion of built-in Python number to correct type (int, float) This function doesn't support decimal.Decimal or datetime.datetime or numpy types. - alexber.utils.parsers.is_empty - if value is None returns True. if value is empty iterable (for example, empty str or emptry list),returns true otherwise false. Note: For not iterable values, behaivour is undefined. - alexber.utils.parsers.parse_boolean - if value is None returns None. if value is boolean, it is returned as it is. if value is str and value is equals ignoring case to "True", True is returned. if value is str and value is equals ignoring case to "False", False is returned. For every other value, the answer is undefined. - alexber.utils.props.Properties - A Python replacement for java.util.Properties class This is modelled as closely as possible to the Java original. Created - Anand B Pillai . Update to Python 3 by Alex. Also there are some tweeks that was done by Alex. %package help Summary: Development documents and examples for alex-ber-utils Provides: python3-alex-ber-utils-doc %description help ## AlexBerUtils AlexBerUtils is collection of the small utilities. See CHANGELOG.md for detail description. ### Getting Help ### QuickStart ```bash python3 -m pip install -U alex-ber-utils ``` ### Installing from Github ```bash python3 -m pip install -U https://github.com/alex-ber/AlexBerUtils/archive/master.zip ``` Optionally installing tests requirements. ```bash python3 -m pip install -U https://github.com/alex-ber/AlexBerUtils/archive/master.zip#egg=alex-ber-utils[tests] ``` Or explicitly: ```bash wget https://github.com/alex-ber/AlexBerUtils/archive/master.zip -O master.zip; unzip master.zip; rm master.zip ``` And then installing from source (see below). ### Installing from source ```bash python3 -m pip install . # only installs "required" ``` ```bash python3 -m pip install .[tests] # installs dependencies for tests ``` ```bash python3 -m pip install .[md] # installs multidispatcher (used in method_overloading_test.py) ``` ```bash python3 -m pip install .[fabric] # installs fabric (used in fabs.py) ``` ```bash python3 -m pip install .[yml] # installs Yml related dependencies # (used in ymlparsers.py, init_app_conf.py, deploys.py; # optionally used in ymlparsers_extra.py, emails.py) ``` ```bash python3 -m pip install .[env] # installs pydotenv (optionally used in deploys.py and mains.py) ``` #### Alternatively you install install from requirements file: ```bash python3 -m pip install -r requirements.txt # only installs "required" ``` ```bash python3 -m pip install -r requirements-tests.txt # installs dependencies for tests ``` ```bash python3 -m pip install -r requirements-md.txt # installs multidispatcher (used in method_overloading_test.py) ``` ```bash python3 -m pip install -r requirements-fabric.txt # installs fabric (used in fabs.py) ``` ```bash python3 -m pip install -r requirements-yml.txt # installs Yml related dependencies # (used in ymlparsers.py, init_app_conf.py, deploys.py; # optionally used in ymlparsers_extra.py, emails.py) ``` ```bash python3 -m pip install -r requirements-env.txt # installs pydotenv (optionally used in deploys.py) ``` ### Using Docker `alexberkovich/AlexBerUtils:latest` contains all `AlexBerUtils` dependencies. This Dockerfile is very simple, you can take relevant part for you and put them into your Dockerfile. ## Alternatively, you can use it as base Docker image for your project and add/upgrade another dependencies as you need. For example: ```Dockerfile FROM alexberkovich/alex_ber_utils:latest COPY requirements.txt etc/requirements.txt RUN set -ex && \ #latest pip,setuptools,wheel pip install --upgrade pip setuptools wheel && \ pip install alex_ber_utils pip install -r etc/requirements.txt CMD ["/bin/sh"] #CMD tail -f /dev/null ``` where `requirements.txt` is requirements for your project. ## From the directory with setup.py ```bash python3 setup.py test #run all tests ``` or ```bash pytest ``` ## Installing new version See https://docs.python.org/3.1/distutils/uploading.html ## Installing new version to venv ```bash python38 -m pip uninstall --yes alex_ber_utils python38 setup.py clean sdist bdist_wheel python38 -m pip install --find-links=./dist alex_ber_utils==0.6.5 ``` ##Manual upload ```bash #python setup.py clean sdist upload ``` ## Requirements AlexBerUtils requires the following modules. * Python 3.6+ * PyYAML==5.1 # Changelog All notable changes to this project will be documented in this file. \#https://pypi.org/manage/project/alex-ber-utils/releases/ ## Unreleased ## [0.6.6] - 13-06-2021 ### Added - `stdLogging` module. The main function is `initStream()`. This is Thin adapter layer that redirects stdout/stderr (or any other stream-like object) to standard Python's logger. Based on https://github.com/fx-kirin/py-stdlogging/blob/master/stdlogging.py See https://github.com/fx-kirin/py-stdlogging/pull/1 Quote from https://stackoverflow.com/questions/47325506/making-python-loggers-log-all-stdout-and-stderr-messages : "But be careful to capture stdout because it's very fragile". I decided to focus on redirecting stderr only to the logger. If you want you can also redirect stdout, by making 2 calls to initStream() package-level method. But, because of https://unix.stackexchange.com/questions/616616/separate-stdout-and-stderr-for-docker-run it is sufficient only to do it for stderr for me. See [https://alex-ber.medium.com/stdlogging-module-d5d69ff7103f] for details. #### Changed - `Doockerfiles` base-image. Now, you can transparentely switch betwee AMD64 to ARM 64 proccessor. - `cffi` dependency from 1.14.3 to 1.14.5. - `cryptography` dependency from 3.1.1 to 3.4.7. ## [0.6.5] - 12-04-2021 ### Added - `FixRelCwd` context-manager in `mains` module - This context-manager temporary changes current working directory to the one where relPackage is installed. What if you have some script or application that use relative path and you want to invoke in from another directory. To get things more complicated. maybe your “external” code also use *relative* path, but relative to another directory. See [https://alex-ber.medium.com/making-more-yo-relative-path-to-file-to-work-fbf6280f9511] for details. - `GuardedWorkerException` context-manager in `mains` module - context manager that mitigate exception propogation from another process. It is very difficult if not impossible to pickle exceptions back to the parent process. Simple ones work, but many others don’t. For example, CalledProcessError is not pickable (my guess, this is because of stdout, stderr data-members). This means, that if child process raise CalledProcessError that is not catched, it will propagate to the parent process, but the propagation will fail, apparently because of bug in Python itself. This cause *pool.join() to halt forever — and thus memory leak!* See [https://alex-ber.medium.com/exception-propagation-from-another-process-bb09894ba4ce] for details. - `join_files()` function in `files` module - Suppose, that you have some multi-threaded/multi-process application where each thread/process creates some file (each thread/process create different file) and you want to join them to one file. See [https://alex-ber.medium.com/join-files-cc5e38e3c658] for details. #### Changed - `fixabscwd()` function in `mains` module - minour refactoring - moving out some internal helper function for reuse in new function. - Base docker image version to alexberkovich/alpine-anaconda3:0.2.1-slim. alexberkovich/alpine-anaconda3:0.1.1 has some minor changes relative to alexberkovich/alpine-anaconda3:0.1.1. See [https://github.com/alex-ber/alpine-anaconda3/blob/master/CHANGELOG.md] for details. #### Updated ### Documentation - See [https://github.com/alex-ber/AlexBerUtils/issues/8] Config file from another directory is not resolved (using `argumentParser` with `--general.config.file` can't be passed to `init_app_conf.parse_config()`) ## [0.6.4] - 12/12/2020 #### Changed - Base docker image version to alexberkovich/alpine-anaconda3:0.1.1-slim. alexberkovich/alpine-anaconda3:0.1.1 has some minor changes relative to alexberkovich/alpine-anaconda3:0.1.0. See [https://github.com/alex-ber/alpine-anaconda3/blob/master/CHANGELOG.md] for details. alexberkovich/alpine-anaconda3:0.1.1-slim is "slim" version of the same docker image, most unused packaged are removed. - update versions to pip==20.3.1 setuptools==51.0.0 wheel==0.36.1 ### Removed - Script check_d.py ## [0.6.3] - 18/11/2020 #### Changed - Base docker image version to alexberkovich/alpine-anaconda3:0.1.0, it has fix for potential security risk: Git was changed not to store credential as plain text, but to keep them in memory for 1 hour, see https://git-scm.com/docs/git-credential-cache #### Updated ### Documentation - My `deploys` module [https://medium.com/analytics-vidhya/my-deploys-module-26c5599f1b15 for documentation] is updated to contain `fix_retry_env()` function in `mains` module. ### Added ### Documentation - `fix_retry_env()` function in `mains` module. [https://alex-ber.medium.com/make-path-to-file-on-windows-works-on-linux-402ed3624f66] ## [0.6.2] - 17/11/2020 ### Deprecation - `method_overloading_test.py` is deprecated and will be removed once AlexBerUtils will support Python 3.9. It will happen approximately at 01.11.2021. This test uses `multidispatch` project that wasn't updated since 2014. In Python 3.8 it has following warning: `multidispatch.py:163: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working from collections import MutableMapping` ### Added - class `OsEnvrionPathRetry`, function `fix_retry_env()` to `mains` module. ### Changed - `OsEnvrionPathExpender` - refactored, functionality is preserved. ## [0.6.1] - 16/11/2020 ### Added - optional Dockerfile - optional .env.docker for reference. - Support of Python 3.8 is validated, see https://github.com/alex-ber/AlexBerUtils/issues/5 - Email formatting changed in Python 3.8, see https://github.com/alex-ber/AlexBerUtils/issues/7 Note that it is possible that `7bit` will be replaced with `8bit` as `Content-Transfer-Encoding`, that I'm considering as ok. - `check_all.py` to run all unit test. - `check_d.py` for sanity test. - .dockerignore - requirements*.txt - dependencies version changed, see https://github.com/alex-ber/AlexBerUtils/issues/6 - Because of pytest upgrade `conftest.py` was changed: `pytest_configure()` was added to support dynamically used marks. - In `ymlparsers_test.py` deprecation warning removed (it will be error in Python 3.9) `collections.Mapping` was changed to `collections.abc.Mapping`. ### Changed - README.MD added section about Docker usage. - setup.py to indicate support of Python 3.8 ## [0.5.3] - 10/09/2020 ### Changed - `alexber.utils.emails.initConfig` is fixed. Before this default variables where ignored. - 2 Unit tests for `init_app_conf` are fixed. These fix are minors. ### Documentation - `importer` module [https://medium.com/analytics-vidhya/how-to-write-easily-customizable-code-8b00b43406b2] - `fixabscwd()` function in `mains` module. [https://medium.com/@alex_ber/making-relative-path-to-file-to-work-d5d0f1da67bf] - `fix_retry_env()` function in `mains` module. [https://alex-ber.medium.com/make-path-to-file-on-windows-works-on-linux-402ed3624f66] - My `parser` module [https://medium.com/analytics-vidhya/my-parser-module-429ed1457718] - My `ymlparsers` module [https://medium.com/analytics-vidhya/my-ymlparsers-module-88221edf16a6] - My major `init_app_conf` module [https://medium.com/analytics-vidhya/my-major-init-app-conf-module-1a5d9fb3998c] - My `deploys` module [https://medium.com/analytics-vidhya/my-deploys-module-26c5599f1b15] - My `emails` module [https://medium.com/analytics-vidhya/my-emails-module-3ad36a4861c5] - My `processinvokes` module [https://medium.com/analytics-vidhya/my-processinvokes-module-de4d301518df] ## [0.5.2] - 21/06/2020 ### Added - `path()` function in `mains` module. For older Python version uses `importlib_resources` module. For newer version built in `importlib.resources`. - `load_env()` function in `mains` module. Added kwargs forwarding. if dotenv_path or stream is present it will be used. if ENV_PCK is present, dotenv_path will be constructed from ENV_PCK and ENV_NAME. Otherwise, kwargs will be forwarded as is to load_dotenv. - `fix_env()` function in `mains` module. For each key in ENV_KEYS, this method prepends full_prefix to os.environ[key]. full_prefix is calculated as absolute path of `__init__.py` of ENV_PCK. ### Changed ``processinvokes`` function ```run_sub_process``` - documentation typo fixed. Lower Python version to 3.6. ## [0.5.1] - 06-05-2020 ### Added - `mains` module explanation article https://medium.com/@alex_ber/making-relative-path-to-file-to-work-d5d0f1da67bf is published. - `fabs` module. It adds cp method to fabric.Connection. This method is Linux-like cp command. It copies single file to remote (Posix) machine. - Spited dependency list for setup.py req.txt (inexact versions, direct dependency only) and for reproducible installation requirements.txt (exact versions, all, including transitive dependencies). - Added req-fabric.txt, requirements-fabric.txt - Fabric, used in `fabs` module. - Added req-yml.txt, requirements-yml.txt - Yml-related dependencies, used in ymlparsers.py and in init_app_conf.py, deploys.py; optionally used in ymlparsers_extra.py, emails.py. Main dependency is HiYaPyCo. I'm using feature that is availlable in the minimal version. HiYaPyCo depends upon PyYAML and Jinja2. Limitations for Jinja2 is from HiYaPyCo project. - Added req-env.txt, requirements-env.txt - pydotenv, optionally used in deploys.py. - Added `inspects.has_method`(cls, methodName). Check if class cls has method with name methodName directly, or in one of it's super-classes. - Added `pareser.parse_sys_args` function parses command line arguments. - Added `ymlparsers` module - `load`/`safe_dump` a Hierarchical Yml files. This is essentially wrapper arround HiYaPyCo project with streamlined and extended API and couple of work-arrounds. Note: this module doesn't use any package-level variables in hiYaPyCo module, including hiYaPyCo.jinja2env. This module do use Jinja2's `Environment`. It also has another defaults for `load`/`safe_dump` methods. They can be overridden in `initConfig()` function. `safe_dump()` method supports simple Python objects like primitive types (str, integer, etc), list, dict, **OrderedDict**. `as_str()`- convenient method for getting str representation of the data, for example of dict. `DisableVarSubst` - use of this context manager disables variable substation in the `load()` function. `initConfig` - this method reset some defaults. If running from the MainThread, this method is idempotent. - Added `init_app_conf` **major** module. The main function is `parse_config`. This function parses command line arguments first. Than it parse yml files. Command line arguments overrides yml files arguments. Parameters of yml files we always try to convert on best-effort basses. Parameters of system args we try convert according to `implicit_convert` param. If you supply `implicit_convert=True`, than `mask_value()` will be applied to the flat map (first parameter). Otherwise, `implicit_convert` wiil have the value that was set in `intiConfig()`. By default it is `True`. Command line key --general.profiles or appropriate key default yml file is used to find 'profiles'. Let suppose, that --config_file is resolved to config.yml. If 'profiles' is not empty, than it will be used to calculate filenames that will be used to override default yml file. Let suppose, 'profiles' resolved to ['dev', 'local']. Than first config.yml will be loaded, than it will be overridden with config-dev.yml, than it will be overridden with config-local.yml. At last, it will be overridden with system args. This entry can be always be overridden with system args. `ymlparsers` and `parser` modules serves as Low-Level API for this module. `mask_value()` implemented as a wrapper to `parsers.safe_eval()` method with support for boolean variables. This implementation is used to get type for arguments that we get from system args. This mechanism can be easily replaced with your own one. `to_convex_map()` This method receives dictionary with 'flat keys', it has simple key:value structure where value can't be another dictionary. It will return dictionary of dictionaries with natural key mapping, optionally, entries will be filtered out according to white_list_flat_keys and, optionally, value will be implicitly converted to appropriate type. In order to simulate dictionary of dictionaries 'flat keys' compose key from outer dict with key from inner dict separated with dot. For example, 'general.profiles' 'flat key' corresponds to convex map with 'general' key with dictionary as value that have one of the keys 'profiles' with corresponding value. If you supply `implicit_convert=True`, than `mask_value()` will be applied to the values of the received flat dictionary. Otherwise, `implicit_convert` wiil have the value that was set in `intiConfig()`. By default it is True. `merge_list_value_in_dicts` - merges value of 2 dicts. This value represents list of values. Value from flat_d is roughly obtained by flat_d[main_key+'.'+sub_key]. Value from d is roughly obtained by d[main_key][sub_key]. If you supply `implicit_convert=True`, than `mask_value()` will be applied to the flat map (first parameter). Otherwise, `implicit_convert` wiil have the value that was set in `intiConfig()`. By default it is `True`. `initConfig` - you can set default value of `implicit_convert`. By default it is `True`. This parameters is used if `implicit_convert` wasn't explicitly supplied. This method is idempotent. - Added `deploys` module. This module is usable in your deployment script. See also `fabs` module. This method use `parsers`, ymlparsers`, `init_app_conf` as it's low-level API. `init_app_conf` usage is limited. The main function is `load_config()`. It is simplified method for parsing yml configuration file with optionally overrided profiles only. See `init_app_conf.parse_config()` for another variant. `split_path` - Split filename in 2 part parts by split_dirname. first_part will ends with split_dirname. second_part will start immediately after split_dirname. `add_to_zip_copy_function` - Factory method that returns closure that can be used as copy_function param in `shutil.copytree()`. - Added `emails` module. This module contains extensions of the logging handlers. This module optionally depends on `ymlparseser` module. It is better to use `EmailStatus` context manager with configured `emailLogger`. It is intended to configure first your `emailLogger` with `OneMemoryHandler` (together with `SMTPHandler`). Than the code block that you want to aggregate log messages from is better to be enclosed with `EmailStatus` context manager. `alexber.utils.emails.SMTPHandler` is customization of `logging.handlers.SMTPHandler`. It's purpose is to connect to SMTP server and actually send the e-mail. Unlike `logging.handlers.SMTPHandler` this class expects for record.msg to be built EmailMessage. You can also change use of underline SMTP class to SMTP_SSL, LMTP, etc. This implementation is *thread-safe*. `alexber.utils.emails.OneMemoryHandler` is variant of `logging.handlers.MemoryHandler`. This handler aggregates log messages until `FINISHED` log-level is received or application is going to terminate abruptly (see docstring of `calc_abrupt_vars()` method for the details) and we have some log messages in the buffer. On such event all messages (in the current Thread) are aggregated to the single `EmailMessage`. The subject of the `EmailMessage` is determined by `get_subject()` method. If you want to change delimeters used to indicate variable declaration inside template, see docstring of the `get_subject()` method. It is better to use `EmailStatus` context manager with configured emailLogger. See docstring of `EmailStatus`. This implementation is *thread-safe*. `alexber.utils.emails.EmailStatus` - if contextmanager exits with exception (it fails), than e-mail with subject formatted with faildargs and faildkwargs will be send. Otherwise, e-mail with subject formatted with successargs and successkwargs will be send. All messages (in the current Thread) will be aggregated to one long e-mail with the subject described in `OneMemoryHandler.get_subject()` method. `alexber.utils.emails.initConfig` - this method reset some defaults. This method is idempotent. By default, `SMTP` class from `smtplib` is used to send actual e-mail. You can change it to `SMTP_SSL`, `LMTP`, or another class by specifying default_smpt_cls_name. You can also specified default port for sending e-mails. `processInvokes` module has one primary function - `run_sub_process()` This method run subprocess and logs it's out to the logger. This method is sophisticated decorator to `subprocess.run()`. It is useful, when your subprocess run's a lot of time and you're interesting to receive it's `stdout` and `stderr`. By default, it's streamed to log. You can easily customize this behavior, see `initConig()` method. `initConig()` This method can be optionally called prior any call to another function in this module. You can use your custom class for the logging. For example, FilePipe. ### Changed - Spited dependency list for setup.py req.txt (inexact versions, direct dependency only) and for reproducible installation requirements.txt (exact versions, all, including transitive dependencies). - README.md changed, added section 'Alternatively you install install from requirements file:'. Some other misc changed done. - CHANGELOG.md version 0.4.1 misc changes. - Misc improvement in unit tests. - Fixed `parser.safe_eval` - safe_eval('%(message)s') was blow up, now it returns value as is. See https://github.com/alex-ber/AlexBerUtils/issues/2 - Enhanced `importer.importer` - added support for PEP 420 (implicit Namespace Packages). Namespace packages are a mechanism for splitting a single Python package across multiple directories on disk. When interpreted encounter with non-empty __path__ attribute it adds modules found in those locations to the current package. See https://github.com/alex-ber/AlexBerUtils/issues/3 - In all documentation refference to `pip3` was changed to `python3 -m pip` ## [0.4.1] - 2020-04-02 **BREAKING CHANGE** I highly recommend not to use 0.3.X versions. ### Removed - module `warns` is droped ### Changed - *Limitation:*: `mains` module wasn't tested with frozen python script (frozen using py2exe). - module `mains` is rewritten. Function `initConf` is dropped entirely. - module `mains` now works with logger and with warnings (it was wrong decision to work with warnings). ## [0.3.4] - 2020-04-02 ### Changed - CHANGELOG.md fixed - `warns` module bug fixed, now warnings.warn() works. - FixabscwdWarning is added to simplify warnings disabling. - Changing how `mains` module use `warns`. ## [0.3.3] - 2020-04-02 ### Changed - CHANGELOG.md fixed ## [0.3.2] - 2020-04-01 ### Changed - To REAMDE.md add `Installing new version` section - Fix typo in REAMDE.md (tests, not test). - Fixing bug: now, you're able to import package in the Python interpreter (`setups.py` fixed) - Fixing bug: `warns` module now doesn't change log_level in the preconfigured logger in any cases. - **BREAKING CHANGE**: In`mains` module method `warnsInitConfig()` was renamed to `mainInitConfig()` Also singature was changed. - `mains` module minor refactored. ### Added - Unit tests are added for `warns` module - Unit tests are added for `mains` module ## [0.3.1] - 2020-04-01 ### Changed - Tests minor improvements. - Excluded tests, data from setup.py (from being installed from the sdist.) - Created MANIFEST.in ### Added - `warns `module is added: It provides better integration between warnings and logger. Unlike `logging._showwarning()` this variant will always go through logger. `warns.initConfig()` has optional file parameter (it's file-like object) to redirect warnings. Default value is `sys.stderr`. If logger for `log_name` (default is `py.warnings`) will be configured before call to `showwarning()` method, than warning will go to the logger's handler with `log_level` (default is `logging.WARNING`). If logger for `log_name` (default is `py.warnings`) willn't be configured before call to showwarning() method, than warning will be done to `file` (default is `sys.stderr`) with `log_level` (default is `logging.WARNING`). - `main` module is added: `main.fixabscwd()` changes `os.getcwd()` to be the directory of the `__main__` module. `main.warnsInitConfig()` reexports `warns.initConfig()` for convenience. ### Added - Tests for alexber.utils.thread_locals added. ## [0.2.5] - 2019-05-22 ### Changed - Fixed bug in UploadCommand, git push should be before git tag. ## [0.2.4] - 2019-05-22 ### Changed - Fixed bug in setup.py, incorrect order between VERSION and UploadCommand (no tag was created on upload) ## [0.2.1] - 2019-05-22 ### Changed - setup url fixed. - Added import of Enum to alexber.utils package. ## [0.2.0] - 2019-05-22 ### Changed - setup.py - keywords added. ## [0.1.1] - 2019-05-22 ### Changed - README.md fixed typo. ## [0.1.0] - 2019-05-22 ### Changed - alexber.utils.UploadCommand - bug fixed, failed on git tag, because VERSION was undefined. ## [0.0.1] - 2019-05-22 ### Added - alexber.utils.StrAsReprMixinEnum - Enum Mixin that has __str__() equal to __repr__(). - alexber.utils.AutoNameMixinEnum- Enum Mixin that generate value equal to the name. - alexber.utils.MissingNoneMixinEnum - Enum Mixin will return None if value will not be found. - alexber.utils.LookUpMixinEnum - Enim Mixin that is designed to be used for lookup by value. If lookup fail, None will be return. Also, __str__() will return the same value as __repr__(). - alexber.utils.threadlocal_var, get_threadlocal_var, del_threadlocal_var. Inspired by https://stackoverflow.com/questions/1408171/thread-local-storage-in-python - alexber.utils.UploadCommand - Support setup.py upload. UploadCommand is intented to be used only from setup.py It's builds Source and Wheel distribution. It's uploads the package to PyPI via Twine. It's pushes the git tags. - alexber.utils.uuid1mc is is a hybrid between version 1 & version 4. This is v1 with random MAC ("v1mc"). uuid1mc() is deliberately generating v1 UUIDs with a random broadcast MAC address. The resulting v1 UUID is time dependant (like regular v1), but lacks all host-specific information (like v4). Note: somebody reported that ran into trouble using UUID1 in Amazon EC2 instances. - alexber.utils.importer.importer - Convert str to Python construct that target is represented. - alexber.utils.importer.new_instance - Convert str to Python construct that target is represented. args and kwargs will be passed in to appropriate __new__() / __init__() / __init_subclass__() methods. - alexber.utils.inspects.issetdescriptor - Return true if the object is a method descriptor with setters. But not if ismethod() or isclass() or isfunction() are true. - alexber.utils.inspects.ismethod - Return false if object is not a class and not a function. Otherwise, return true iff signature has 2 params. - alexber.utils.parsers.safe_eval - The purpose of this function is convert numbers from str to correct type. This function support convertion of built-in Python number to correct type (int, float) This function doesn't support decimal.Decimal or datetime.datetime or numpy types. - alexber.utils.parsers.is_empty - if value is None returns True. if value is empty iterable (for example, empty str or emptry list),returns true otherwise false. Note: For not iterable values, behaivour is undefined. - alexber.utils.parsers.parse_boolean - if value is None returns None. if value is boolean, it is returned as it is. if value is str and value is equals ignoring case to "True", True is returned. if value is str and value is equals ignoring case to "False", False is returned. For every other value, the answer is undefined. - alexber.utils.props.Properties - A Python replacement for java.util.Properties class This is modelled as closely as possible to the Java original. Created - Anand B Pillai . Update to Python 3 by Alex. Also there are some tweeks that was done by Alex. %prep %autosetup -n alex-ber-utils-0.6.6 %build %py3_build %install %py3_install install -d -m755 %{buildroot}/%{_pkgdocdir} if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi pushd %{buildroot} if [ -d usr/lib ]; then find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/lib64 ]; then find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/bin ]; then find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/sbin ]; then find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst fi touch doclist.lst if [ -d usr/share/man ]; then find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst fi popd mv %{buildroot}/filelist.lst . mv %{buildroot}/doclist.lst . %files -n python3-alex-ber-utils -f filelist.lst %dir %{python3_sitelib}/* %files help -f doclist.lst %{_docdir}/* %changelog * Wed May 10 2023 Python_Bot - 0.6.6-1 - Package Spec generated