diff options
author | CoprDistGit <infra@openeuler.org> | 2023-05-10 05:01:32 +0000 |
---|---|---|
committer | CoprDistGit <infra@openeuler.org> | 2023-05-10 05:01:32 +0000 |
commit | 3c1998d1b10010a3af5c9e43eb48436d076b2453 (patch) | |
tree | fa50e211cf47e770d77468ea0bd83c9ea318c609 /python-alex-ber-utils.spec | |
parent | b9bf5f465421cfcb56b068d415c55882b882cc29 (diff) |
automatic import of python-alex-ber-utilsopeneuler20.03
Diffstat (limited to 'python-alex-ber-utils.spec')
-rw-r--r-- | python-alex-ber-utils.spec | 2098 |
1 files changed, 2098 insertions, 0 deletions
diff --git a/python-alex-ber-utils.spec b/python-alex-ber-utils.spec new file mode 100644 index 0000000..f1003c2 --- /dev/null +++ b/python-alex-ber-utils.spec @@ -0,0 +1,2098 @@ +%global _empty_manifest_terminate_build 0 +Name: python-alex-ber-utils +Version: 0.6.6 +Release: 1 +Summary: AlexBerUtils is collection of the small utilities +License: Apache 2.0 +URL: https://github.com/alex-ber/AlexBerUtils +Source0: https://mirrors.nju.edu.cn/pypi/web/packages/1e/7f/0ad16bd3d077a14b0288d8aeb7e653bd3401fc4d45cfe364ec600fc6aed4/alex_ber_utils-0.6.6.tar.gz +BuildArch: noarch + +Requires: python3-PyYAML +Requires: python3-dotenv +Requires: python3-fabric +Requires: python3-multidispatch +Requires: python3-attrs +Requires: python3-mock +Requires: python3-py +Requires: python3-pytest +Requires: python3-pytest-assume +Requires: python3-pytest-mock +Requires: python3-PyYAML +Requires: python3-toml +Requires: python3-pluggy +Requires: python3-packaging +Requires: python3-iniconfig +Requires: python3-pyparsing +Requires: python3-PyYAML +Requires: python3-HiYaPyCo +Requires: python3-Jinja2 +Requires: python3-PyYAML +Requires: python3-HiYaPyCo +Requires: python3-Jinja2 + +%description +## AlexBerUtils + +AlexBerUtils is collection of the small utilities. See CHANGELOG.md for detail description. + + + +### Getting Help + + +### QuickStart +```bash +python3 -m pip install -U alex-ber-utils +``` + + +### Installing from Github + +```bash +python3 -m pip install -U https://github.com/alex-ber/AlexBerUtils/archive/master.zip +``` +Optionally installing tests requirements. + +```bash +python3 -m pip install -U https://github.com/alex-ber/AlexBerUtils/archive/master.zip#egg=alex-ber-utils[tests] +``` + +Or explicitly: + +```bash +wget https://github.com/alex-ber/AlexBerUtils/archive/master.zip -O master.zip; unzip master.zip; rm master.zip +``` +And then installing from source (see below). + + +### Installing from source +```bash +python3 -m pip install . # only installs "required" +``` +```bash +python3 -m pip install .[tests] # installs dependencies for tests +``` +```bash +python3 -m pip install .[md] # installs multidispatcher (used in method_overloading_test.py) +``` +```bash +python3 -m pip install .[fabric] # installs fabric (used in fabs.py) +``` +```bash +python3 -m pip install .[yml] # installs Yml related dependencies + # (used in ymlparsers.py, init_app_conf.py, deploys.py; + # optionally used in ymlparsers_extra.py, emails.py) +``` +```bash +python3 -m pip install .[env] # installs pydotenv (optionally used in deploys.py and mains.py) +``` + +#### Alternatively you install install from requirements file: +```bash +python3 -m pip install -r requirements.txt # only installs "required" +``` +```bash +python3 -m pip install -r requirements-tests.txt # installs dependencies for tests +``` +```bash +python3 -m pip install -r requirements-md.txt # installs multidispatcher (used in method_overloading_test.py) +``` +```bash +python3 -m pip install -r requirements-fabric.txt # installs fabric (used in fabs.py) +``` +```bash +python3 -m pip install -r requirements-yml.txt # installs Yml related dependencies + # (used in ymlparsers.py, init_app_conf.py, deploys.py; + # optionally used in ymlparsers_extra.py, emails.py) +``` +```bash +python3 -m pip install -r requirements-env.txt # installs pydotenv (optionally used in deploys.py) +``` + +### Using Docker +`alexberkovich/AlexBerUtils:latest` contains all `AlexBerUtils` dependencies. +This Dockerfile is very simple, you can take relevant part for you and put them into your Dockerfile. + +## +Alternatively, you can use it as base Docker image for your project and add/upgrade +another dependencies as you need. + +For example: + +```Dockerfile +FROM alexberkovich/alex_ber_utils:latest + +COPY requirements.txt etc/requirements.txt + +RUN set -ex && \ + #latest pip,setuptools,wheel + pip install --upgrade pip setuptools wheel && \ + pip install alex_ber_utils + pip install -r etc/requirements.txt + +CMD ["/bin/sh"] +#CMD tail -f /dev/null +``` + +where `requirements.txt` is requirements for your project. + + + +## + +From the directory with setup.py +```bash +python3 setup.py test #run all tests +``` + +or + +```bash + +pytest +``` + +## Installing new version +See https://docs.python.org/3.1/distutils/uploading.html + + +## Installing new version to venv +```bash +python38 -m pip uninstall --yes alex_ber_utils +python38 setup.py clean sdist bdist_wheel +python38 -m pip install --find-links=./dist alex_ber_utils==0.6.5 +``` + +##Manual upload +```bash +#python setup.py clean sdist upload +``` + + +## Requirements + + +AlexBerUtils requires the following modules. + +* Python 3.6+ + +* PyYAML==5.1 + + +# Changelog +All notable changes to this project will be documented in this file. + +\#https://pypi.org/manage/project/alex-ber-utils/releases/ + +## Unreleased + +## [0.6.6] - 13-06-2021 +### Added +- `stdLogging` module. The main function is `initStream()`. This is Thin adapter layer that redirects stdout/stderr +(or any other stream-like object) to standard Python's logger. Based on +https://github.com/fx-kirin/py-stdlogging/blob/master/stdlogging.py See https://github.com/fx-kirin/py-stdlogging/pull/1 +Quote from https://stackoverflow.com/questions/47325506/making-python-loggers-log-all-stdout-and-stderr-messages : +"But be careful to capture stdout because it's very fragile". I decided to focus on redirecting stderr only to the +logger. If you want you can also redirect stdout, by making 2 calls to initStream() package-level method. +But, because of https://unix.stackexchange.com/questions/616616/separate-stdout-and-stderr-for-docker-run it is +sufficient only to do it for stderr for me. +See [https://alex-ber.medium.com/stdlogging-module-d5d69ff7103f] for details. + +#### Changed +- `Doockerfiles` base-image. Now, you can transparentely switch betwee AMD64 to ARM 64 proccessor. +- `cffi` dependency from 1.14.3 to 1.14.5. +- `cryptography` dependency from 3.1.1 to 3.4.7. + + +## [0.6.5] - 12-04-2021 +### Added +- `FixRelCwd` context-manager in `mains` module - This context-manager temporary changes current working directory to +the one where relPackage is installed. What if you have some script or application that use relative path and you +want to invoke in from another directory. To get things more complicated. maybe your “external” code also use +*relative* path, but relative to another directory. +See [https://alex-ber.medium.com/making-more-yo-relative-path-to-file-to-work-fbf6280f9511] for details. + +- `GuardedWorkerException` context-manager in `mains` module - context manager that mitigate exception propogation +from another process. It is very difficult if not impossible to pickle exceptions back to the parent process. +Simple ones work, but many others don’t. +For example, CalledProcessError is not pickable (my guess, this is because of stdout, stderr data-members). +This means, that if child process raise CalledProcessError that is not catched, it will propagate to the parent +process, but the propagation will fail, apparently because of bug in Python itself. +This cause *pool.join() to halt forever — and thus memory leak!* +See [https://alex-ber.medium.com/exception-propagation-from-another-process-bb09894ba4ce] for details. + +- `join_files()` function in `files` module - Suppose, that you have some multi-threaded/multi-process application +where each thread/process creates some file (each thread/process create different file) +and you want to join them to one file. +See [https://alex-ber.medium.com/join-files-cc5e38e3c658] for details. + + +#### Changed +- `fixabscwd()` function in `mains` module - minour refactoring - moving out some internal helper function for reuse +in new function. + +- Base docker image version to alexberkovich/alpine-anaconda3:0.2.1-slim. +alexberkovich/alpine-anaconda3:0.1.1 has some minor changes relative to alexberkovich/alpine-anaconda3:0.1.1. +See [https://github.com/alex-ber/alpine-anaconda3/blob/master/CHANGELOG.md] for details. + +#### Updated +### Documentation +- See [https://github.com/alex-ber/AlexBerUtils/issues/8] Config file from another directory is not resolved +(using `argumentParser` with `--general.config.file` can't be passed to `init_app_conf.parse_config()`) + + +## [0.6.4] - 12/12/2020 +#### Changed +- Base docker image version to alexberkovich/alpine-anaconda3:0.1.1-slim. +alexberkovich/alpine-anaconda3:0.1.1 has some minor changes relative to alexberkovich/alpine-anaconda3:0.1.0. +See [https://github.com/alex-ber/alpine-anaconda3/blob/master/CHANGELOG.md] for details. +alexberkovich/alpine-anaconda3:0.1.1-slim is "slim" version of the same docker image, most unused packaged are removed. + +- update versions to pip==20.3.1 setuptools==51.0.0 wheel==0.36.1 + +### Removed +- Script check_d.py + +## [0.6.3] - 18/11/2020 +#### Changed +- Base docker image version to alexberkovich/alpine-anaconda3:0.1.0, it has fix for potential security risk: Git was changed +not to store credential as plain text, but to keep them in memory for 1 hour, +see https://git-scm.com/docs/git-credential-cache + +#### Updated +### Documentation +- My `deploys` module [https://medium.com/analytics-vidhya/my-deploys-module-26c5599f1b15 for documentation] +is updated to contain `fix_retry_env()` function in `mains` module. + +### Added +### Documentation +- `fix_retry_env()` function in `mains` module. [https://alex-ber.medium.com/make-path-to-file-on-windows-works-on-linux-402ed3624f66] + + +## [0.6.2] - 17/11/2020 +### Deprecation +- `method_overloading_test.py` is deprecated and will be removed once AlexBerUtils will support +Python 3.9. It will happen approximately at 01.11.2021. + +This test uses `multidispatch` project that wasn't updated since 2014. +In Python 3.8 it has following warning: + +`multidispatch.py:163: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working + from collections import MutableMapping` + +### Added +- class `OsEnvrionPathRetry`, function `fix_retry_env()` to `mains` module. + +### Changed +- `OsEnvrionPathExpender` - refactored, functionality is preserved. + + +## [0.6.1] - 16/11/2020 +### Added +- optional Dockerfile +- optional .env.docker for reference. +- Support of Python 3.8 is validated, see https://github.com/alex-ber/AlexBerUtils/issues/5 + +- Email formatting changed in Python 3.8, see https://github.com/alex-ber/AlexBerUtils/issues/7 + +Note that it is possible that `7bit` will be replaced with `8bit` as `Content-Transfer-Encoding`, +that I'm considering as ok. + +- `check_all.py` to run all unit test. + +- `check_d.py` for sanity test. + +- .dockerignore + +- requirements*.txt - dependencies version changed, see https://github.com/alex-ber/AlexBerUtils/issues/6 +- Because of pytest upgrade `conftest.py` was changed: +`pytest_configure()` was added to support dynamically used marks. +- In `ymlparsers_test.py` deprecation warning removed (it will be error in Python 3.9) +`collections.Mapping` was changed to `collections.abc.Mapping`. + +### Changed +- README.MD added section about Docker usage. +- setup.py to indicate support of Python 3.8 + + +## [0.5.3] - 10/09/2020 +### Changed +- `alexber.utils.emails.initConfig` is fixed. Before this default variables where ignored. +- 2 Unit tests for `init_app_conf` are fixed. These fix are minors. + + +### Documentation +- `importer` module [https://medium.com/analytics-vidhya/how-to-write-easily-customizable-code-8b00b43406b2] +- `fixabscwd()` function in `mains` module. [https://medium.com/@alex_ber/making-relative-path-to-file-to-work-d5d0f1da67bf] +- `fix_retry_env()` function in `mains` module. [https://alex-ber.medium.com/make-path-to-file-on-windows-works-on-linux-402ed3624f66] +- My `parser` module [https://medium.com/analytics-vidhya/my-parser-module-429ed1457718] +- My `ymlparsers` module [https://medium.com/analytics-vidhya/my-ymlparsers-module-88221edf16a6] +- My major `init_app_conf` module [https://medium.com/analytics-vidhya/my-major-init-app-conf-module-1a5d9fb3998c] +- My `deploys` module [https://medium.com/analytics-vidhya/my-deploys-module-26c5599f1b15] +- My `emails` module [https://medium.com/analytics-vidhya/my-emails-module-3ad36a4861c5] +- My `processinvokes` module [https://medium.com/analytics-vidhya/my-processinvokes-module-de4d301518df] + +## [0.5.2] - 21/06/2020 +### Added +- `path()` function in `mains` module. For older Python version uses +`importlib_resources` module. For newer version built in `importlib.resources`. +- `load_env()` function in `mains` module. Added kwargs forwarding. if dotenv_path or stream is present it will be used. +if ENV_PCK is present, dotenv_path will be constructed from ENV_PCK and ENV_NAME. +Otherwise, kwargs will be forwarded as is to load_dotenv. +- `fix_env()` function in `mains` module. For each key in ENV_KEYS, this method prepends full_prefix to os.environ[key]. +full_prefix is calculated as absolute path of `__init__.py` of ENV_PCK. + + +### Changed +``processinvokes`` function ```run_sub_process``` - documentation typo fixed. +Lower Python version to 3.6. + +## [0.5.1] - 06-05-2020 +### Added + +- `mains` module explanation article https://medium.com/@alex_ber/making-relative-path-to-file-to-work-d5d0f1da67bf is published. + +- `fabs` module. It adds cp method to fabric.Connection. + +This method is Linux-like cp command. It copies single file to remote (Posix) machine. + +- Spited dependency list for setup.py req.txt (inexact versions, direct dependency only) and for +reproducible installation requirements.txt (exact versions, all, including transitive dependencies). + +- Added req-fabric.txt, requirements-fabric.txt - Fabric, used in `fabs` module. + +- Added req-yml.txt, requirements-yml.txt - Yml-related dependencies, used in ymlparsers.py +and in init_app_conf.py, deploys.py; optionally used in ymlparsers_extra.py, emails.py. + +Main dependency is HiYaPyCo. I'm using feature that is availlable in the minimal version. + +HiYaPyCo depends upon PyYAML and Jinja2. Limitations for Jinja2 is from HiYaPyCo project. + +- Added req-env.txt, requirements-env.txt - pydotenv, optionally used in deploys.py. + +- Added `inspects.has_method`(cls, methodName). Check if class cls has method with name methodName directly, +or in one of it's super-classes. + +- Added `pareser.parse_sys_args` function parses command line arguments. + +- Added `ymlparsers` module - `load`/`safe_dump` a Hierarchical Yml files. This is essentially wrapper arround HiYaPyCo project with streamlined +and extended API and couple of work-arrounds. + +Note: this module doesn't use any package-level variables in hiYaPyCo module, including hiYaPyCo.jinja2env. +This module do use Jinja2's `Environment`. + +It also has another defaults for `load`/`safe_dump` methods. +They can be overridden in `initConfig()` function. + +`safe_dump()` method supports simple Python objects like primitive types (str, integer, etc), list, dict, **OrderedDict**. + +`as_str()`- convenient method for getting str representation of the data, +for example of dict. + +`DisableVarSubst` - use of this context manager disables variable substation in the `load()` function. + +`initConfig` - this method reset some defaults. If running from the MainThread, this method is idempotent. + +- Added `init_app_conf` **major** module. + +The main function is `parse_config`. This function parses command line arguments first. +Than it parse yml files. Command line arguments overrides yml files arguments. +Parameters of yml files we always try to convert on best-effort basses. +Parameters of system args we try convert according to `implicit_convert` param. + +If you supply `implicit_convert=True`, than `mask_value()` will be applied to the flat map (first parameter). +Otherwise, `implicit_convert` wiil have the value that was set in `intiConfig()`. By default it is `True`. + +Command line key --general.profiles or appropriate key default yml file is used to find 'profiles'. +Let suppose, that --config_file is resolved to config.yml. +If 'profiles' is not empty, than it will be used to calculate filenames +that will be used to override default yml file. +Let suppose, 'profiles' resolved to ['dev', 'local']. Than first config.yml +will be loaded, than it will be overridden with config-dev.yml, than +it will be overridden with config-local.yml. +At last, it will be overridden with system args. +This entry can be always be overridden with system args. + +`ymlparsers` and `parser` modules serves as Low-Level API for this module. + +`mask_value()` implemented as a wrapper to `parsers.safe_eval()` method with support for boolean +variables. This implementation is used to get type for arguments that we get from system args. +This mechanism can be easily replaced with your own one. + +`to_convex_map()` This method receives dictionary with 'flat keys', it has simple key:value structure +where value can't be another dictionary. +It will return dictionary of dictionaries with natural key mapping, +optionally, entries will be filtered out according to white_list_flat_keys and, +optionally, value will be implicitly converted to appropriate type. + +In order to simulate dictionary of dictionaries 'flat keys' compose key from outer dict with key from inner dict +separated with dot. +For example, 'general.profiles' 'flat key' corresponds to convex map with 'general' key with dictionary as value +that have one of the keys 'profiles' with corresponding value. + +If you supply `implicit_convert=True`, than `mask_value()` will be applied to the values of the received flat dictionary. +Otherwise, `implicit_convert` wiil have the value that was set in `intiConfig()`. By default it is True. + +`merge_list_value_in_dicts` - merges value of 2 dicts. This value represents list of values. +Value from flat_d is roughly obtained by flat_d[main_key+'.'+sub_key]. +Value from d is roughly obtained by d[main_key][sub_key]. + +If you supply `implicit_convert=True`, than `mask_value()` will be applied to the flat map (first parameter). +Otherwise, `implicit_convert` wiil have the value that was set in `intiConfig()`. By default it is `True`. + +`initConfig` - you can set default value of `implicit_convert`. By default it is `True`. +This parameters is used if `implicit_convert` wasn't explicitly supplied. This method is idempotent. + +- Added `deploys` module. +This module is usable in your deployment script. See also `fabs` module. + +This method use `parsers`, ymlparsers`, `init_app_conf` as it's low-level API. `init_app_conf` usage is limited. + +The main function is `load_config()`. It is simplified method for parsing yml configuration file with optionally +overrided profiles only. See `init_app_conf.parse_config()` for another variant. + +`split_path` - Split filename in 2 part parts by split_dirname. first_part will ends with split_dirname. +second_part will start immediately after split_dirname. + +`add_to_zip_copy_function` - Factory method that returns closure that can be used as copy_function param in +`shutil.copytree()`. + +- Added `emails` module. +This module contains extensions of the logging handlers. +This module optionally depends on `ymlparseser` module. +It is better to use `EmailStatus` context manager with configured `emailLogger`. +It is intended to configure first your `emailLogger` with `OneMemoryHandler` (together with `SMTPHandler`). +Than the code block that you want to aggregate log messages from is better to be enclosed with `EmailStatus` +context manager. + +`alexber.utils.emails.SMTPHandler` is customization of `logging.handlers.SMTPHandler`. It's purpose is to connect to +SMTP server and actually send the e-mail. Unlike `logging.handlers.SMTPHandler` this class expects for record.msg to be built EmailMessage. +You can also change use of underline SMTP class to SMTP_SSL, LMTP, etc. +This implementation is *thread-safe*. + +`alexber.utils.emails.OneMemoryHandler` is variant of `logging.handlers.MemoryHandler`. This handler aggregates +log messages until `FINISHED` log-level is received or application is going to terminate abruptly (see docstring +of `calc_abrupt_vars()` method for the details) and we have some log messages in the buffer. On such event +all messages (in the current Thread) are aggregated to the single `EmailMessage`. The subject of the `EmailMessage` +is determined by `get_subject()` method. +If you want to change delimeters used to indicate variable declaration inside template, see docstring of the +`get_subject()` method. +It is better to use `EmailStatus` context manager with configured emailLogger. See docstring of `EmailStatus`. +This implementation is *thread-safe*. + +`alexber.utils.emails.EmailStatus` - if contextmanager exits with exception (it fails), than e-mail with +subject formatted with faildargs and faildkwargs will be send. +Otherwise, e-mail with subject formatted with successargs and successkwargs will be send. +All messages (in the current Thread) will be aggregated to one long e-mail with the subject described in +`OneMemoryHandler.get_subject()` method. + +`alexber.utils.emails.initConfig` - this method reset some defaults. This method is idempotent. +By default, `SMTP` class from `smtplib` is used to send actual e-mail. You can change it to `SMTP_SSL`, `LMTP`, +or another class by specifying default_smpt_cls_name. +You can also specified default port for sending e-mails. + +`processInvokes` module has one primary function - `run_sub_process()` This method run subprocess and logs it's out +to the logger. This method is sophisticated decorator to `subprocess.run()`. It is useful, when your subprocess +run's a lot of time and you're interesting to receive it's `stdout` and `stderr`. By default, it's streamed to log. +You can easily customize this behavior, see `initConig()` method. + +`initConig()` This method can be optionally called prior any call to another function in this module. You can use your +custom class for the logging. For example, FilePipe. + + +### Changed +- Spited dependency list for setup.py req.txt (inexact versions, direct dependency only) and for +reproducible installation requirements.txt (exact versions, all, including transitive dependencies). + +- README.md changed, added section 'Alternatively you install install from requirements file:'. +Some other misc changed done. + +- CHANGELOG.md version 0.4.1 misc changes. + +- Misc improvement in unit tests. + +- Fixed `parser.safe_eval` - safe_eval('%(message)s') was blow up, now it returns value as is. +See https://github.com/alex-ber/AlexBerUtils/issues/2 + +- Enhanced `importer.importer` - added support for PEP 420 (implicit Namespace Packages). +Namespace packages are a mechanism for splitting a single Python package across multiple directories on disk. +When interpreted encounter with non-empty __path__ attribute it adds modules found in those locations +to the current package. +See https://github.com/alex-ber/AlexBerUtils/issues/3 + +- In all documentation refference to `pip3` was changed to `python3 -m pip` + +## [0.4.1] - 2020-04-02 +**BREAKING CHANGE** I highly recommend not to use 0.3.X versions. +### Removed +- module `warns` is droped + +### Changed +- *Limitation:*: + +`mains` module wasn't tested with frozen python script (frozen using py2exe). + +- module `mains` is rewritten. Function `initConf` is dropped entirely. +- module `mains` now works with logger and with warnings (it was wrong decision to work with warnings). + + +## [0.3.4] - 2020-04-02 +### Changed +- CHANGELOG.md fixed +- `warns` module bug fixed, now warnings.warn() works. +- FixabscwdWarning is added to simplify warnings disabling. +- Changing how `mains` module use `warns`. + + +## [0.3.3] - 2020-04-02 +### Changed +- CHANGELOG.md fixed + + +## [0.3.2] - 2020-04-01 +### Changed +- To REAMDE.md add `Installing new version` section +- Fix typo in REAMDE.md (tests, not test). +- Fixing bug: now, you're able to import package in the Python interpreter (`setups.py` fixed) +- Fixing bug: `warns` module now doesn't change log_level in the preconfigured logger in any cases. +- **BREAKING CHANGE**: In`mains` module method `warnsInitConfig()` was renamed to `mainInitConfig()` +Also singature was changed. +- `mains` module minor refactored. + + +### Added +- Unit tests are added for `warns` module +- Unit tests are added for `mains` module + +## [0.3.1] - 2020-04-01 +### Changed +- Tests minor improvements. +- Excluded tests, data from setup.py (from being installed from the sdist.) +- Created MANIFEST.in + + +### Added +- `warns `module is added: + +It provides better integration between warnings and logger. +Unlike `logging._showwarning()` this variant will always go through logger. + +`warns.initConfig()` has optional file parameter (it's file-like object) to redirect warnings. +Default value is `sys.stderr`. + +If logger for `log_name` (default is `py.warnings`) will be configured before call to `showwarning()` method, +than warning will go to the logger's handler with `log_level` (default is `logging.WARNING`). + +If logger for `log_name` (default is `py.warnings`) willn't be configured before call to showwarning() method, +than warning will be done to `file` (default is `sys.stderr`) with `log_level` (default is `logging.WARNING`). + +- `main` module is added: + +`main.fixabscwd()` changes `os.getcwd()` to be the directory of the `__main__` module. + +`main.warnsInitConfig()` reexports `warns.initConfig()` for convenience. + + + +### Added +- Tests for alexber.utils.thread_locals added. + +## [0.2.5] - 2019-05-22 +### Changed +- Fixed bug in UploadCommand, git push should be before git tag. + + +## [0.2.4] - 2019-05-22 +### Changed +- Fixed bug in setup.py, incorrect order between VERSION and UploadCommand (no tag was created on upload) + +## [0.2.1] - 2019-05-22 +### Changed +- setup url fixed. +- Added import of Enum to alexber.utils package. + +## [0.2.0] - 2019-05-22 +### Changed +- setup.py - keywords added. + +## [0.1.1] - 2019-05-22 +### Changed +- README.md fixed typo. + +## [0.1.0] - 2019-05-22 +### Changed +- alexber.utils.UploadCommand - bug fixed, failed on git tag, because VERSION was undefined. + + +## [0.0.1] - 2019-05-22 +### Added +- alexber.utils.StrAsReprMixinEnum - Enum Mixin that has __str__() equal to __repr__(). +- alexber.utils.AutoNameMixinEnum- Enum Mixin that generate value equal to the name. +- alexber.utils.MissingNoneMixinEnum - Enum Mixin will return None if value will not be found. +- alexber.utils.LookUpMixinEnum - Enim Mixin that is designed to be used for lookup by value. + + If lookup fail, None will be return. Also, __str__() will return the same value as __repr__(). +- alexber.utils.threadlocal_var, get_threadlocal_var, del_threadlocal_var. + + Inspired by https://stackoverflow.com/questions/1408171/thread-local-storage-in-python + +- alexber.utils.UploadCommand - Support setup.py upload. + + UploadCommand is intented to be used only from setup.py + + It's builds Source and Wheel distribution. + + It's uploads the package to PyPI via Twine. + + It's pushes the git tags. + +- alexber.utils.uuid1mc is is a hybrid between version 1 & version 4. This is v1 with random MAC ("v1mc"). + + uuid1mc() is deliberately generating v1 UUIDs with a random broadcast MAC address. + + The resulting v1 UUID is time dependant (like regular v1), but lacks all host-specific information (like v4). + + Note: somebody reported that ran into trouble using UUID1 in Amazon EC2 instances. + + +- alexber.utils.importer.importer - Convert str to Python construct that target is represented. +- alexber.utils.importer.new_instance - Convert str to Python construct that target is represented. +args and kwargs will be passed in to appropriate __new__() / __init__() / __init_subclass__() methods. +- alexber.utils.inspects.issetdescriptor - Return true if the object is a method descriptor with setters. + + But not if ismethod() or isclass() or isfunction() are true. +- alexber.utils.inspects.ismethod - Return false if object is not a class and not a function. +Otherwise, return true iff signature has 2 params. +- alexber.utils.parsers.safe_eval - The purpose of this function is convert numbers from str to correct type. + + This function support convertion of built-in Python number to correct type (int, float) + + This function doesn't support decimal.Decimal or datetime.datetime or numpy types. +- alexber.utils.parsers.is_empty - if value is None returns True. + + if value is empty iterable (for example, empty str or emptry list),returns true otherwise false. + + Note: For not iterable values, behaivour is undefined. +- alexber.utils.parsers.parse_boolean - if value is None returns None. + + if value is boolean, it is returned as it is. + if value is str and value is equals ignoring case to "True", True is returned. + if value is str and value is equals ignoring case to "False", False is returned. + + For every other value, the answer is undefined. + + + +- alexber.utils.props.Properties - A Python replacement for java.util.Properties class + + This is modelled as closely as possible to the Java original. + + Created - Anand B Pillai <abpillai@gmail.com>. + + Update to Python 3 by Alex. + + Also there are some tweeks that was done by Alex. + +<!-- +### Changed +### Removed +--> + + + + +%package -n python3-alex-ber-utils +Summary: AlexBerUtils is collection of the small utilities +Provides: python-alex-ber-utils +BuildRequires: python3-devel +BuildRequires: python3-setuptools +BuildRequires: python3-pip +%description -n python3-alex-ber-utils +## AlexBerUtils + +AlexBerUtils is collection of the small utilities. See CHANGELOG.md for detail description. + + + +### Getting Help + + +### QuickStart +```bash +python3 -m pip install -U alex-ber-utils +``` + + +### Installing from Github + +```bash +python3 -m pip install -U https://github.com/alex-ber/AlexBerUtils/archive/master.zip +``` +Optionally installing tests requirements. + +```bash +python3 -m pip install -U https://github.com/alex-ber/AlexBerUtils/archive/master.zip#egg=alex-ber-utils[tests] +``` + +Or explicitly: + +```bash +wget https://github.com/alex-ber/AlexBerUtils/archive/master.zip -O master.zip; unzip master.zip; rm master.zip +``` +And then installing from source (see below). + + +### Installing from source +```bash +python3 -m pip install . # only installs "required" +``` +```bash +python3 -m pip install .[tests] # installs dependencies for tests +``` +```bash +python3 -m pip install .[md] # installs multidispatcher (used in method_overloading_test.py) +``` +```bash +python3 -m pip install .[fabric] # installs fabric (used in fabs.py) +``` +```bash +python3 -m pip install .[yml] # installs Yml related dependencies + # (used in ymlparsers.py, init_app_conf.py, deploys.py; + # optionally used in ymlparsers_extra.py, emails.py) +``` +```bash +python3 -m pip install .[env] # installs pydotenv (optionally used in deploys.py and mains.py) +``` + +#### Alternatively you install install from requirements file: +```bash +python3 -m pip install -r requirements.txt # only installs "required" +``` +```bash +python3 -m pip install -r requirements-tests.txt # installs dependencies for tests +``` +```bash +python3 -m pip install -r requirements-md.txt # installs multidispatcher (used in method_overloading_test.py) +``` +```bash +python3 -m pip install -r requirements-fabric.txt # installs fabric (used in fabs.py) +``` +```bash +python3 -m pip install -r requirements-yml.txt # installs Yml related dependencies + # (used in ymlparsers.py, init_app_conf.py, deploys.py; + # optionally used in ymlparsers_extra.py, emails.py) +``` +```bash +python3 -m pip install -r requirements-env.txt # installs pydotenv (optionally used in deploys.py) +``` + +### Using Docker +`alexberkovich/AlexBerUtils:latest` contains all `AlexBerUtils` dependencies. +This Dockerfile is very simple, you can take relevant part for you and put them into your Dockerfile. + +## +Alternatively, you can use it as base Docker image for your project and add/upgrade +another dependencies as you need. + +For example: + +```Dockerfile +FROM alexberkovich/alex_ber_utils:latest + +COPY requirements.txt etc/requirements.txt + +RUN set -ex && \ + #latest pip,setuptools,wheel + pip install --upgrade pip setuptools wheel && \ + pip install alex_ber_utils + pip install -r etc/requirements.txt + +CMD ["/bin/sh"] +#CMD tail -f /dev/null +``` + +where `requirements.txt` is requirements for your project. + + + +## + +From the directory with setup.py +```bash +python3 setup.py test #run all tests +``` + +or + +```bash + +pytest +``` + +## Installing new version +See https://docs.python.org/3.1/distutils/uploading.html + + +## Installing new version to venv +```bash +python38 -m pip uninstall --yes alex_ber_utils +python38 setup.py clean sdist bdist_wheel +python38 -m pip install --find-links=./dist alex_ber_utils==0.6.5 +``` + +##Manual upload +```bash +#python setup.py clean sdist upload +``` + + +## Requirements + + +AlexBerUtils requires the following modules. + +* Python 3.6+ + +* PyYAML==5.1 + + +# Changelog +All notable changes to this project will be documented in this file. + +\#https://pypi.org/manage/project/alex-ber-utils/releases/ + +## Unreleased + +## [0.6.6] - 13-06-2021 +### Added +- `stdLogging` module. The main function is `initStream()`. This is Thin adapter layer that redirects stdout/stderr +(or any other stream-like object) to standard Python's logger. Based on +https://github.com/fx-kirin/py-stdlogging/blob/master/stdlogging.py See https://github.com/fx-kirin/py-stdlogging/pull/1 +Quote from https://stackoverflow.com/questions/47325506/making-python-loggers-log-all-stdout-and-stderr-messages : +"But be careful to capture stdout because it's very fragile". I decided to focus on redirecting stderr only to the +logger. If you want you can also redirect stdout, by making 2 calls to initStream() package-level method. +But, because of https://unix.stackexchange.com/questions/616616/separate-stdout-and-stderr-for-docker-run it is +sufficient only to do it for stderr for me. +See [https://alex-ber.medium.com/stdlogging-module-d5d69ff7103f] for details. + +#### Changed +- `Doockerfiles` base-image. Now, you can transparentely switch betwee AMD64 to ARM 64 proccessor. +- `cffi` dependency from 1.14.3 to 1.14.5. +- `cryptography` dependency from 3.1.1 to 3.4.7. + + +## [0.6.5] - 12-04-2021 +### Added +- `FixRelCwd` context-manager in `mains` module - This context-manager temporary changes current working directory to +the one where relPackage is installed. What if you have some script or application that use relative path and you +want to invoke in from another directory. To get things more complicated. maybe your “external” code also use +*relative* path, but relative to another directory. +See [https://alex-ber.medium.com/making-more-yo-relative-path-to-file-to-work-fbf6280f9511] for details. + +- `GuardedWorkerException` context-manager in `mains` module - context manager that mitigate exception propogation +from another process. It is very difficult if not impossible to pickle exceptions back to the parent process. +Simple ones work, but many others don’t. +For example, CalledProcessError is not pickable (my guess, this is because of stdout, stderr data-members). +This means, that if child process raise CalledProcessError that is not catched, it will propagate to the parent +process, but the propagation will fail, apparently because of bug in Python itself. +This cause *pool.join() to halt forever — and thus memory leak!* +See [https://alex-ber.medium.com/exception-propagation-from-another-process-bb09894ba4ce] for details. + +- `join_files()` function in `files` module - Suppose, that you have some multi-threaded/multi-process application +where each thread/process creates some file (each thread/process create different file) +and you want to join them to one file. +See [https://alex-ber.medium.com/join-files-cc5e38e3c658] for details. + + +#### Changed +- `fixabscwd()` function in `mains` module - minour refactoring - moving out some internal helper function for reuse +in new function. + +- Base docker image version to alexberkovich/alpine-anaconda3:0.2.1-slim. +alexberkovich/alpine-anaconda3:0.1.1 has some minor changes relative to alexberkovich/alpine-anaconda3:0.1.1. +See [https://github.com/alex-ber/alpine-anaconda3/blob/master/CHANGELOG.md] for details. + +#### Updated +### Documentation +- See [https://github.com/alex-ber/AlexBerUtils/issues/8] Config file from another directory is not resolved +(using `argumentParser` with `--general.config.file` can't be passed to `init_app_conf.parse_config()`) + + +## [0.6.4] - 12/12/2020 +#### Changed +- Base docker image version to alexberkovich/alpine-anaconda3:0.1.1-slim. +alexberkovich/alpine-anaconda3:0.1.1 has some minor changes relative to alexberkovich/alpine-anaconda3:0.1.0. +See [https://github.com/alex-ber/alpine-anaconda3/blob/master/CHANGELOG.md] for details. +alexberkovich/alpine-anaconda3:0.1.1-slim is "slim" version of the same docker image, most unused packaged are removed. + +- update versions to pip==20.3.1 setuptools==51.0.0 wheel==0.36.1 + +### Removed +- Script check_d.py + +## [0.6.3] - 18/11/2020 +#### Changed +- Base docker image version to alexberkovich/alpine-anaconda3:0.1.0, it has fix for potential security risk: Git was changed +not to store credential as plain text, but to keep them in memory for 1 hour, +see https://git-scm.com/docs/git-credential-cache + +#### Updated +### Documentation +- My `deploys` module [https://medium.com/analytics-vidhya/my-deploys-module-26c5599f1b15 for documentation] +is updated to contain `fix_retry_env()` function in `mains` module. + +### Added +### Documentation +- `fix_retry_env()` function in `mains` module. [https://alex-ber.medium.com/make-path-to-file-on-windows-works-on-linux-402ed3624f66] + + +## [0.6.2] - 17/11/2020 +### Deprecation +- `method_overloading_test.py` is deprecated and will be removed once AlexBerUtils will support +Python 3.9. It will happen approximately at 01.11.2021. + +This test uses `multidispatch` project that wasn't updated since 2014. +In Python 3.8 it has following warning: + +`multidispatch.py:163: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working + from collections import MutableMapping` + +### Added +- class `OsEnvrionPathRetry`, function `fix_retry_env()` to `mains` module. + +### Changed +- `OsEnvrionPathExpender` - refactored, functionality is preserved. + + +## [0.6.1] - 16/11/2020 +### Added +- optional Dockerfile +- optional .env.docker for reference. +- Support of Python 3.8 is validated, see https://github.com/alex-ber/AlexBerUtils/issues/5 + +- Email formatting changed in Python 3.8, see https://github.com/alex-ber/AlexBerUtils/issues/7 + +Note that it is possible that `7bit` will be replaced with `8bit` as `Content-Transfer-Encoding`, +that I'm considering as ok. + +- `check_all.py` to run all unit test. + +- `check_d.py` for sanity test. + +- .dockerignore + +- requirements*.txt - dependencies version changed, see https://github.com/alex-ber/AlexBerUtils/issues/6 +- Because of pytest upgrade `conftest.py` was changed: +`pytest_configure()` was added to support dynamically used marks. +- In `ymlparsers_test.py` deprecation warning removed (it will be error in Python 3.9) +`collections.Mapping` was changed to `collections.abc.Mapping`. + +### Changed +- README.MD added section about Docker usage. +- setup.py to indicate support of Python 3.8 + + +## [0.5.3] - 10/09/2020 +### Changed +- `alexber.utils.emails.initConfig` is fixed. Before this default variables where ignored. +- 2 Unit tests for `init_app_conf` are fixed. These fix are minors. + + +### Documentation +- `importer` module [https://medium.com/analytics-vidhya/how-to-write-easily-customizable-code-8b00b43406b2] +- `fixabscwd()` function in `mains` module. [https://medium.com/@alex_ber/making-relative-path-to-file-to-work-d5d0f1da67bf] +- `fix_retry_env()` function in `mains` module. [https://alex-ber.medium.com/make-path-to-file-on-windows-works-on-linux-402ed3624f66] +- My `parser` module [https://medium.com/analytics-vidhya/my-parser-module-429ed1457718] +- My `ymlparsers` module [https://medium.com/analytics-vidhya/my-ymlparsers-module-88221edf16a6] +- My major `init_app_conf` module [https://medium.com/analytics-vidhya/my-major-init-app-conf-module-1a5d9fb3998c] +- My `deploys` module [https://medium.com/analytics-vidhya/my-deploys-module-26c5599f1b15] +- My `emails` module [https://medium.com/analytics-vidhya/my-emails-module-3ad36a4861c5] +- My `processinvokes` module [https://medium.com/analytics-vidhya/my-processinvokes-module-de4d301518df] + +## [0.5.2] - 21/06/2020 +### Added +- `path()` function in `mains` module. For older Python version uses +`importlib_resources` module. For newer version built in `importlib.resources`. +- `load_env()` function in `mains` module. Added kwargs forwarding. if dotenv_path or stream is present it will be used. +if ENV_PCK is present, dotenv_path will be constructed from ENV_PCK and ENV_NAME. +Otherwise, kwargs will be forwarded as is to load_dotenv. +- `fix_env()` function in `mains` module. For each key in ENV_KEYS, this method prepends full_prefix to os.environ[key]. +full_prefix is calculated as absolute path of `__init__.py` of ENV_PCK. + + +### Changed +``processinvokes`` function ```run_sub_process``` - documentation typo fixed. +Lower Python version to 3.6. + +## [0.5.1] - 06-05-2020 +### Added + +- `mains` module explanation article https://medium.com/@alex_ber/making-relative-path-to-file-to-work-d5d0f1da67bf is published. + +- `fabs` module. It adds cp method to fabric.Connection. + +This method is Linux-like cp command. It copies single file to remote (Posix) machine. + +- Spited dependency list for setup.py req.txt (inexact versions, direct dependency only) and for +reproducible installation requirements.txt (exact versions, all, including transitive dependencies). + +- Added req-fabric.txt, requirements-fabric.txt - Fabric, used in `fabs` module. + +- Added req-yml.txt, requirements-yml.txt - Yml-related dependencies, used in ymlparsers.py +and in init_app_conf.py, deploys.py; optionally used in ymlparsers_extra.py, emails.py. + +Main dependency is HiYaPyCo. I'm using feature that is availlable in the minimal version. + +HiYaPyCo depends upon PyYAML and Jinja2. Limitations for Jinja2 is from HiYaPyCo project. + +- Added req-env.txt, requirements-env.txt - pydotenv, optionally used in deploys.py. + +- Added `inspects.has_method`(cls, methodName). Check if class cls has method with name methodName directly, +or in one of it's super-classes. + +- Added `pareser.parse_sys_args` function parses command line arguments. + +- Added `ymlparsers` module - `load`/`safe_dump` a Hierarchical Yml files. This is essentially wrapper arround HiYaPyCo project with streamlined +and extended API and couple of work-arrounds. + +Note: this module doesn't use any package-level variables in hiYaPyCo module, including hiYaPyCo.jinja2env. +This module do use Jinja2's `Environment`. + +It also has another defaults for `load`/`safe_dump` methods. +They can be overridden in `initConfig()` function. + +`safe_dump()` method supports simple Python objects like primitive types (str, integer, etc), list, dict, **OrderedDict**. + +`as_str()`- convenient method for getting str representation of the data, +for example of dict. + +`DisableVarSubst` - use of this context manager disables variable substation in the `load()` function. + +`initConfig` - this method reset some defaults. If running from the MainThread, this method is idempotent. + +- Added `init_app_conf` **major** module. + +The main function is `parse_config`. This function parses command line arguments first. +Than it parse yml files. Command line arguments overrides yml files arguments. +Parameters of yml files we always try to convert on best-effort basses. +Parameters of system args we try convert according to `implicit_convert` param. + +If you supply `implicit_convert=True`, than `mask_value()` will be applied to the flat map (first parameter). +Otherwise, `implicit_convert` wiil have the value that was set in `intiConfig()`. By default it is `True`. + +Command line key --general.profiles or appropriate key default yml file is used to find 'profiles'. +Let suppose, that --config_file is resolved to config.yml. +If 'profiles' is not empty, than it will be used to calculate filenames +that will be used to override default yml file. +Let suppose, 'profiles' resolved to ['dev', 'local']. Than first config.yml +will be loaded, than it will be overridden with config-dev.yml, than +it will be overridden with config-local.yml. +At last, it will be overridden with system args. +This entry can be always be overridden with system args. + +`ymlparsers` and `parser` modules serves as Low-Level API for this module. + +`mask_value()` implemented as a wrapper to `parsers.safe_eval()` method with support for boolean +variables. This implementation is used to get type for arguments that we get from system args. +This mechanism can be easily replaced with your own one. + +`to_convex_map()` This method receives dictionary with 'flat keys', it has simple key:value structure +where value can't be another dictionary. +It will return dictionary of dictionaries with natural key mapping, +optionally, entries will be filtered out according to white_list_flat_keys and, +optionally, value will be implicitly converted to appropriate type. + +In order to simulate dictionary of dictionaries 'flat keys' compose key from outer dict with key from inner dict +separated with dot. +For example, 'general.profiles' 'flat key' corresponds to convex map with 'general' key with dictionary as value +that have one of the keys 'profiles' with corresponding value. + +If you supply `implicit_convert=True`, than `mask_value()` will be applied to the values of the received flat dictionary. +Otherwise, `implicit_convert` wiil have the value that was set in `intiConfig()`. By default it is True. + +`merge_list_value_in_dicts` - merges value of 2 dicts. This value represents list of values. +Value from flat_d is roughly obtained by flat_d[main_key+'.'+sub_key]. +Value from d is roughly obtained by d[main_key][sub_key]. + +If you supply `implicit_convert=True`, than `mask_value()` will be applied to the flat map (first parameter). +Otherwise, `implicit_convert` wiil have the value that was set in `intiConfig()`. By default it is `True`. + +`initConfig` - you can set default value of `implicit_convert`. By default it is `True`. +This parameters is used if `implicit_convert` wasn't explicitly supplied. This method is idempotent. + +- Added `deploys` module. +This module is usable in your deployment script. See also `fabs` module. + +This method use `parsers`, ymlparsers`, `init_app_conf` as it's low-level API. `init_app_conf` usage is limited. + +The main function is `load_config()`. It is simplified method for parsing yml configuration file with optionally +overrided profiles only. See `init_app_conf.parse_config()` for another variant. + +`split_path` - Split filename in 2 part parts by split_dirname. first_part will ends with split_dirname. +second_part will start immediately after split_dirname. + +`add_to_zip_copy_function` - Factory method that returns closure that can be used as copy_function param in +`shutil.copytree()`. + +- Added `emails` module. +This module contains extensions of the logging handlers. +This module optionally depends on `ymlparseser` module. +It is better to use `EmailStatus` context manager with configured `emailLogger`. +It is intended to configure first your `emailLogger` with `OneMemoryHandler` (together with `SMTPHandler`). +Than the code block that you want to aggregate log messages from is better to be enclosed with `EmailStatus` +context manager. + +`alexber.utils.emails.SMTPHandler` is customization of `logging.handlers.SMTPHandler`. It's purpose is to connect to +SMTP server and actually send the e-mail. Unlike `logging.handlers.SMTPHandler` this class expects for record.msg to be built EmailMessage. +You can also change use of underline SMTP class to SMTP_SSL, LMTP, etc. +This implementation is *thread-safe*. + +`alexber.utils.emails.OneMemoryHandler` is variant of `logging.handlers.MemoryHandler`. This handler aggregates +log messages until `FINISHED` log-level is received or application is going to terminate abruptly (see docstring +of `calc_abrupt_vars()` method for the details) and we have some log messages in the buffer. On such event +all messages (in the current Thread) are aggregated to the single `EmailMessage`. The subject of the `EmailMessage` +is determined by `get_subject()` method. +If you want to change delimeters used to indicate variable declaration inside template, see docstring of the +`get_subject()` method. +It is better to use `EmailStatus` context manager with configured emailLogger. See docstring of `EmailStatus`. +This implementation is *thread-safe*. + +`alexber.utils.emails.EmailStatus` - if contextmanager exits with exception (it fails), than e-mail with +subject formatted with faildargs and faildkwargs will be send. +Otherwise, e-mail with subject formatted with successargs and successkwargs will be send. +All messages (in the current Thread) will be aggregated to one long e-mail with the subject described in +`OneMemoryHandler.get_subject()` method. + +`alexber.utils.emails.initConfig` - this method reset some defaults. This method is idempotent. +By default, `SMTP` class from `smtplib` is used to send actual e-mail. You can change it to `SMTP_SSL`, `LMTP`, +or another class by specifying default_smpt_cls_name. +You can also specified default port for sending e-mails. + +`processInvokes` module has one primary function - `run_sub_process()` This method run subprocess and logs it's out +to the logger. This method is sophisticated decorator to `subprocess.run()`. It is useful, when your subprocess +run's a lot of time and you're interesting to receive it's `stdout` and `stderr`. By default, it's streamed to log. +You can easily customize this behavior, see `initConig()` method. + +`initConig()` This method can be optionally called prior any call to another function in this module. You can use your +custom class for the logging. For example, FilePipe. + + +### Changed +- Spited dependency list for setup.py req.txt (inexact versions, direct dependency only) and for +reproducible installation requirements.txt (exact versions, all, including transitive dependencies). + +- README.md changed, added section 'Alternatively you install install from requirements file:'. +Some other misc changed done. + +- CHANGELOG.md version 0.4.1 misc changes. + +- Misc improvement in unit tests. + +- Fixed `parser.safe_eval` - safe_eval('%(message)s') was blow up, now it returns value as is. +See https://github.com/alex-ber/AlexBerUtils/issues/2 + +- Enhanced `importer.importer` - added support for PEP 420 (implicit Namespace Packages). +Namespace packages are a mechanism for splitting a single Python package across multiple directories on disk. +When interpreted encounter with non-empty __path__ attribute it adds modules found in those locations +to the current package. +See https://github.com/alex-ber/AlexBerUtils/issues/3 + +- In all documentation refference to `pip3` was changed to `python3 -m pip` + +## [0.4.1] - 2020-04-02 +**BREAKING CHANGE** I highly recommend not to use 0.3.X versions. +### Removed +- module `warns` is droped + +### Changed +- *Limitation:*: + +`mains` module wasn't tested with frozen python script (frozen using py2exe). + +- module `mains` is rewritten. Function `initConf` is dropped entirely. +- module `mains` now works with logger and with warnings (it was wrong decision to work with warnings). + + +## [0.3.4] - 2020-04-02 +### Changed +- CHANGELOG.md fixed +- `warns` module bug fixed, now warnings.warn() works. +- FixabscwdWarning is added to simplify warnings disabling. +- Changing how `mains` module use `warns`. + + +## [0.3.3] - 2020-04-02 +### Changed +- CHANGELOG.md fixed + + +## [0.3.2] - 2020-04-01 +### Changed +- To REAMDE.md add `Installing new version` section +- Fix typo in REAMDE.md (tests, not test). +- Fixing bug: now, you're able to import package in the Python interpreter (`setups.py` fixed) +- Fixing bug: `warns` module now doesn't change log_level in the preconfigured logger in any cases. +- **BREAKING CHANGE**: In`mains` module method `warnsInitConfig()` was renamed to `mainInitConfig()` +Also singature was changed. +- `mains` module minor refactored. + + +### Added +- Unit tests are added for `warns` module +- Unit tests are added for `mains` module + +## [0.3.1] - 2020-04-01 +### Changed +- Tests minor improvements. +- Excluded tests, data from setup.py (from being installed from the sdist.) +- Created MANIFEST.in + + +### Added +- `warns `module is added: + +It provides better integration between warnings and logger. +Unlike `logging._showwarning()` this variant will always go through logger. + +`warns.initConfig()` has optional file parameter (it's file-like object) to redirect warnings. +Default value is `sys.stderr`. + +If logger for `log_name` (default is `py.warnings`) will be configured before call to `showwarning()` method, +than warning will go to the logger's handler with `log_level` (default is `logging.WARNING`). + +If logger for `log_name` (default is `py.warnings`) willn't be configured before call to showwarning() method, +than warning will be done to `file` (default is `sys.stderr`) with `log_level` (default is `logging.WARNING`). + +- `main` module is added: + +`main.fixabscwd()` changes `os.getcwd()` to be the directory of the `__main__` module. + +`main.warnsInitConfig()` reexports `warns.initConfig()` for convenience. + + + +### Added +- Tests for alexber.utils.thread_locals added. + +## [0.2.5] - 2019-05-22 +### Changed +- Fixed bug in UploadCommand, git push should be before git tag. + + +## [0.2.4] - 2019-05-22 +### Changed +- Fixed bug in setup.py, incorrect order between VERSION and UploadCommand (no tag was created on upload) + +## [0.2.1] - 2019-05-22 +### Changed +- setup url fixed. +- Added import of Enum to alexber.utils package. + +## [0.2.0] - 2019-05-22 +### Changed +- setup.py - keywords added. + +## [0.1.1] - 2019-05-22 +### Changed +- README.md fixed typo. + +## [0.1.0] - 2019-05-22 +### Changed +- alexber.utils.UploadCommand - bug fixed, failed on git tag, because VERSION was undefined. + + +## [0.0.1] - 2019-05-22 +### Added +- alexber.utils.StrAsReprMixinEnum - Enum Mixin that has __str__() equal to __repr__(). +- alexber.utils.AutoNameMixinEnum- Enum Mixin that generate value equal to the name. +- alexber.utils.MissingNoneMixinEnum - Enum Mixin will return None if value will not be found. +- alexber.utils.LookUpMixinEnum - Enim Mixin that is designed to be used for lookup by value. + + If lookup fail, None will be return. Also, __str__() will return the same value as __repr__(). +- alexber.utils.threadlocal_var, get_threadlocal_var, del_threadlocal_var. + + Inspired by https://stackoverflow.com/questions/1408171/thread-local-storage-in-python + +- alexber.utils.UploadCommand - Support setup.py upload. + + UploadCommand is intented to be used only from setup.py + + It's builds Source and Wheel distribution. + + It's uploads the package to PyPI via Twine. + + It's pushes the git tags. + +- alexber.utils.uuid1mc is is a hybrid between version 1 & version 4. This is v1 with random MAC ("v1mc"). + + uuid1mc() is deliberately generating v1 UUIDs with a random broadcast MAC address. + + The resulting v1 UUID is time dependant (like regular v1), but lacks all host-specific information (like v4). + + Note: somebody reported that ran into trouble using UUID1 in Amazon EC2 instances. + + +- alexber.utils.importer.importer - Convert str to Python construct that target is represented. +- alexber.utils.importer.new_instance - Convert str to Python construct that target is represented. +args and kwargs will be passed in to appropriate __new__() / __init__() / __init_subclass__() methods. +- alexber.utils.inspects.issetdescriptor - Return true if the object is a method descriptor with setters. + + But not if ismethod() or isclass() or isfunction() are true. +- alexber.utils.inspects.ismethod - Return false if object is not a class and not a function. +Otherwise, return true iff signature has 2 params. +- alexber.utils.parsers.safe_eval - The purpose of this function is convert numbers from str to correct type. + + This function support convertion of built-in Python number to correct type (int, float) + + This function doesn't support decimal.Decimal or datetime.datetime or numpy types. +- alexber.utils.parsers.is_empty - if value is None returns True. + + if value is empty iterable (for example, empty str or emptry list),returns true otherwise false. + + Note: For not iterable values, behaivour is undefined. +- alexber.utils.parsers.parse_boolean - if value is None returns None. + + if value is boolean, it is returned as it is. + if value is str and value is equals ignoring case to "True", True is returned. + if value is str and value is equals ignoring case to "False", False is returned. + + For every other value, the answer is undefined. + + + +- alexber.utils.props.Properties - A Python replacement for java.util.Properties class + + This is modelled as closely as possible to the Java original. + + Created - Anand B Pillai <abpillai@gmail.com>. + + Update to Python 3 by Alex. + + Also there are some tweeks that was done by Alex. + +<!-- +### Changed +### Removed +--> + + + + +%package help +Summary: Development documents and examples for alex-ber-utils +Provides: python3-alex-ber-utils-doc +%description help +## AlexBerUtils + +AlexBerUtils is collection of the small utilities. See CHANGELOG.md for detail description. + + + +### Getting Help + + +### QuickStart +```bash +python3 -m pip install -U alex-ber-utils +``` + + +### Installing from Github + +```bash +python3 -m pip install -U https://github.com/alex-ber/AlexBerUtils/archive/master.zip +``` +Optionally installing tests requirements. + +```bash +python3 -m pip install -U https://github.com/alex-ber/AlexBerUtils/archive/master.zip#egg=alex-ber-utils[tests] +``` + +Or explicitly: + +```bash +wget https://github.com/alex-ber/AlexBerUtils/archive/master.zip -O master.zip; unzip master.zip; rm master.zip +``` +And then installing from source (see below). + + +### Installing from source +```bash +python3 -m pip install . # only installs "required" +``` +```bash +python3 -m pip install .[tests] # installs dependencies for tests +``` +```bash +python3 -m pip install .[md] # installs multidispatcher (used in method_overloading_test.py) +``` +```bash +python3 -m pip install .[fabric] # installs fabric (used in fabs.py) +``` +```bash +python3 -m pip install .[yml] # installs Yml related dependencies + # (used in ymlparsers.py, init_app_conf.py, deploys.py; + # optionally used in ymlparsers_extra.py, emails.py) +``` +```bash +python3 -m pip install .[env] # installs pydotenv (optionally used in deploys.py and mains.py) +``` + +#### Alternatively you install install from requirements file: +```bash +python3 -m pip install -r requirements.txt # only installs "required" +``` +```bash +python3 -m pip install -r requirements-tests.txt # installs dependencies for tests +``` +```bash +python3 -m pip install -r requirements-md.txt # installs multidispatcher (used in method_overloading_test.py) +``` +```bash +python3 -m pip install -r requirements-fabric.txt # installs fabric (used in fabs.py) +``` +```bash +python3 -m pip install -r requirements-yml.txt # installs Yml related dependencies + # (used in ymlparsers.py, init_app_conf.py, deploys.py; + # optionally used in ymlparsers_extra.py, emails.py) +``` +```bash +python3 -m pip install -r requirements-env.txt # installs pydotenv (optionally used in deploys.py) +``` + +### Using Docker +`alexberkovich/AlexBerUtils:latest` contains all `AlexBerUtils` dependencies. +This Dockerfile is very simple, you can take relevant part for you and put them into your Dockerfile. + +## +Alternatively, you can use it as base Docker image for your project and add/upgrade +another dependencies as you need. + +For example: + +```Dockerfile +FROM alexberkovich/alex_ber_utils:latest + +COPY requirements.txt etc/requirements.txt + +RUN set -ex && \ + #latest pip,setuptools,wheel + pip install --upgrade pip setuptools wheel && \ + pip install alex_ber_utils + pip install -r etc/requirements.txt + +CMD ["/bin/sh"] +#CMD tail -f /dev/null +``` + +where `requirements.txt` is requirements for your project. + + + +## + +From the directory with setup.py +```bash +python3 setup.py test #run all tests +``` + +or + +```bash + +pytest +``` + +## Installing new version +See https://docs.python.org/3.1/distutils/uploading.html + + +## Installing new version to venv +```bash +python38 -m pip uninstall --yes alex_ber_utils +python38 setup.py clean sdist bdist_wheel +python38 -m pip install --find-links=./dist alex_ber_utils==0.6.5 +``` + +##Manual upload +```bash +#python setup.py clean sdist upload +``` + + +## Requirements + + +AlexBerUtils requires the following modules. + +* Python 3.6+ + +* PyYAML==5.1 + + +# Changelog +All notable changes to this project will be documented in this file. + +\#https://pypi.org/manage/project/alex-ber-utils/releases/ + +## Unreleased + +## [0.6.6] - 13-06-2021 +### Added +- `stdLogging` module. The main function is `initStream()`. This is Thin adapter layer that redirects stdout/stderr +(or any other stream-like object) to standard Python's logger. Based on +https://github.com/fx-kirin/py-stdlogging/blob/master/stdlogging.py See https://github.com/fx-kirin/py-stdlogging/pull/1 +Quote from https://stackoverflow.com/questions/47325506/making-python-loggers-log-all-stdout-and-stderr-messages : +"But be careful to capture stdout because it's very fragile". I decided to focus on redirecting stderr only to the +logger. If you want you can also redirect stdout, by making 2 calls to initStream() package-level method. +But, because of https://unix.stackexchange.com/questions/616616/separate-stdout-and-stderr-for-docker-run it is +sufficient only to do it for stderr for me. +See [https://alex-ber.medium.com/stdlogging-module-d5d69ff7103f] for details. + +#### Changed +- `Doockerfiles` base-image. Now, you can transparentely switch betwee AMD64 to ARM 64 proccessor. +- `cffi` dependency from 1.14.3 to 1.14.5. +- `cryptography` dependency from 3.1.1 to 3.4.7. + + +## [0.6.5] - 12-04-2021 +### Added +- `FixRelCwd` context-manager in `mains` module - This context-manager temporary changes current working directory to +the one where relPackage is installed. What if you have some script or application that use relative path and you +want to invoke in from another directory. To get things more complicated. maybe your “external” code also use +*relative* path, but relative to another directory. +See [https://alex-ber.medium.com/making-more-yo-relative-path-to-file-to-work-fbf6280f9511] for details. + +- `GuardedWorkerException` context-manager in `mains` module - context manager that mitigate exception propogation +from another process. It is very difficult if not impossible to pickle exceptions back to the parent process. +Simple ones work, but many others don’t. +For example, CalledProcessError is not pickable (my guess, this is because of stdout, stderr data-members). +This means, that if child process raise CalledProcessError that is not catched, it will propagate to the parent +process, but the propagation will fail, apparently because of bug in Python itself. +This cause *pool.join() to halt forever — and thus memory leak!* +See [https://alex-ber.medium.com/exception-propagation-from-another-process-bb09894ba4ce] for details. + +- `join_files()` function in `files` module - Suppose, that you have some multi-threaded/multi-process application +where each thread/process creates some file (each thread/process create different file) +and you want to join them to one file. +See [https://alex-ber.medium.com/join-files-cc5e38e3c658] for details. + + +#### Changed +- `fixabscwd()` function in `mains` module - minour refactoring - moving out some internal helper function for reuse +in new function. + +- Base docker image version to alexberkovich/alpine-anaconda3:0.2.1-slim. +alexberkovich/alpine-anaconda3:0.1.1 has some minor changes relative to alexberkovich/alpine-anaconda3:0.1.1. +See [https://github.com/alex-ber/alpine-anaconda3/blob/master/CHANGELOG.md] for details. + +#### Updated +### Documentation +- See [https://github.com/alex-ber/AlexBerUtils/issues/8] Config file from another directory is not resolved +(using `argumentParser` with `--general.config.file` can't be passed to `init_app_conf.parse_config()`) + + +## [0.6.4] - 12/12/2020 +#### Changed +- Base docker image version to alexberkovich/alpine-anaconda3:0.1.1-slim. +alexberkovich/alpine-anaconda3:0.1.1 has some minor changes relative to alexberkovich/alpine-anaconda3:0.1.0. +See [https://github.com/alex-ber/alpine-anaconda3/blob/master/CHANGELOG.md] for details. +alexberkovich/alpine-anaconda3:0.1.1-slim is "slim" version of the same docker image, most unused packaged are removed. + +- update versions to pip==20.3.1 setuptools==51.0.0 wheel==0.36.1 + +### Removed +- Script check_d.py + +## [0.6.3] - 18/11/2020 +#### Changed +- Base docker image version to alexberkovich/alpine-anaconda3:0.1.0, it has fix for potential security risk: Git was changed +not to store credential as plain text, but to keep them in memory for 1 hour, +see https://git-scm.com/docs/git-credential-cache + +#### Updated +### Documentation +- My `deploys` module [https://medium.com/analytics-vidhya/my-deploys-module-26c5599f1b15 for documentation] +is updated to contain `fix_retry_env()` function in `mains` module. + +### Added +### Documentation +- `fix_retry_env()` function in `mains` module. [https://alex-ber.medium.com/make-path-to-file-on-windows-works-on-linux-402ed3624f66] + + +## [0.6.2] - 17/11/2020 +### Deprecation +- `method_overloading_test.py` is deprecated and will be removed once AlexBerUtils will support +Python 3.9. It will happen approximately at 01.11.2021. + +This test uses `multidispatch` project that wasn't updated since 2014. +In Python 3.8 it has following warning: + +`multidispatch.py:163: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working + from collections import MutableMapping` + +### Added +- class `OsEnvrionPathRetry`, function `fix_retry_env()` to `mains` module. + +### Changed +- `OsEnvrionPathExpender` - refactored, functionality is preserved. + + +## [0.6.1] - 16/11/2020 +### Added +- optional Dockerfile +- optional .env.docker for reference. +- Support of Python 3.8 is validated, see https://github.com/alex-ber/AlexBerUtils/issues/5 + +- Email formatting changed in Python 3.8, see https://github.com/alex-ber/AlexBerUtils/issues/7 + +Note that it is possible that `7bit` will be replaced with `8bit` as `Content-Transfer-Encoding`, +that I'm considering as ok. + +- `check_all.py` to run all unit test. + +- `check_d.py` for sanity test. + +- .dockerignore + +- requirements*.txt - dependencies version changed, see https://github.com/alex-ber/AlexBerUtils/issues/6 +- Because of pytest upgrade `conftest.py` was changed: +`pytest_configure()` was added to support dynamically used marks. +- In `ymlparsers_test.py` deprecation warning removed (it will be error in Python 3.9) +`collections.Mapping` was changed to `collections.abc.Mapping`. + +### Changed +- README.MD added section about Docker usage. +- setup.py to indicate support of Python 3.8 + + +## [0.5.3] - 10/09/2020 +### Changed +- `alexber.utils.emails.initConfig` is fixed. Before this default variables where ignored. +- 2 Unit tests for `init_app_conf` are fixed. These fix are minors. + + +### Documentation +- `importer` module [https://medium.com/analytics-vidhya/how-to-write-easily-customizable-code-8b00b43406b2] +- `fixabscwd()` function in `mains` module. [https://medium.com/@alex_ber/making-relative-path-to-file-to-work-d5d0f1da67bf] +- `fix_retry_env()` function in `mains` module. [https://alex-ber.medium.com/make-path-to-file-on-windows-works-on-linux-402ed3624f66] +- My `parser` module [https://medium.com/analytics-vidhya/my-parser-module-429ed1457718] +- My `ymlparsers` module [https://medium.com/analytics-vidhya/my-ymlparsers-module-88221edf16a6] +- My major `init_app_conf` module [https://medium.com/analytics-vidhya/my-major-init-app-conf-module-1a5d9fb3998c] +- My `deploys` module [https://medium.com/analytics-vidhya/my-deploys-module-26c5599f1b15] +- My `emails` module [https://medium.com/analytics-vidhya/my-emails-module-3ad36a4861c5] +- My `processinvokes` module [https://medium.com/analytics-vidhya/my-processinvokes-module-de4d301518df] + +## [0.5.2] - 21/06/2020 +### Added +- `path()` function in `mains` module. For older Python version uses +`importlib_resources` module. For newer version built in `importlib.resources`. +- `load_env()` function in `mains` module. Added kwargs forwarding. if dotenv_path or stream is present it will be used. +if ENV_PCK is present, dotenv_path will be constructed from ENV_PCK and ENV_NAME. +Otherwise, kwargs will be forwarded as is to load_dotenv. +- `fix_env()` function in `mains` module. For each key in ENV_KEYS, this method prepends full_prefix to os.environ[key]. +full_prefix is calculated as absolute path of `__init__.py` of ENV_PCK. + + +### Changed +``processinvokes`` function ```run_sub_process``` - documentation typo fixed. +Lower Python version to 3.6. + +## [0.5.1] - 06-05-2020 +### Added + +- `mains` module explanation article https://medium.com/@alex_ber/making-relative-path-to-file-to-work-d5d0f1da67bf is published. + +- `fabs` module. It adds cp method to fabric.Connection. + +This method is Linux-like cp command. It copies single file to remote (Posix) machine. + +- Spited dependency list for setup.py req.txt (inexact versions, direct dependency only) and for +reproducible installation requirements.txt (exact versions, all, including transitive dependencies). + +- Added req-fabric.txt, requirements-fabric.txt - Fabric, used in `fabs` module. + +- Added req-yml.txt, requirements-yml.txt - Yml-related dependencies, used in ymlparsers.py +and in init_app_conf.py, deploys.py; optionally used in ymlparsers_extra.py, emails.py. + +Main dependency is HiYaPyCo. I'm using feature that is availlable in the minimal version. + +HiYaPyCo depends upon PyYAML and Jinja2. Limitations for Jinja2 is from HiYaPyCo project. + +- Added req-env.txt, requirements-env.txt - pydotenv, optionally used in deploys.py. + +- Added `inspects.has_method`(cls, methodName). Check if class cls has method with name methodName directly, +or in one of it's super-classes. + +- Added `pareser.parse_sys_args` function parses command line arguments. + +- Added `ymlparsers` module - `load`/`safe_dump` a Hierarchical Yml files. This is essentially wrapper arround HiYaPyCo project with streamlined +and extended API and couple of work-arrounds. + +Note: this module doesn't use any package-level variables in hiYaPyCo module, including hiYaPyCo.jinja2env. +This module do use Jinja2's `Environment`. + +It also has another defaults for `load`/`safe_dump` methods. +They can be overridden in `initConfig()` function. + +`safe_dump()` method supports simple Python objects like primitive types (str, integer, etc), list, dict, **OrderedDict**. + +`as_str()`- convenient method for getting str representation of the data, +for example of dict. + +`DisableVarSubst` - use of this context manager disables variable substation in the `load()` function. + +`initConfig` - this method reset some defaults. If running from the MainThread, this method is idempotent. + +- Added `init_app_conf` **major** module. + +The main function is `parse_config`. This function parses command line arguments first. +Than it parse yml files. Command line arguments overrides yml files arguments. +Parameters of yml files we always try to convert on best-effort basses. +Parameters of system args we try convert according to `implicit_convert` param. + +If you supply `implicit_convert=True`, than `mask_value()` will be applied to the flat map (first parameter). +Otherwise, `implicit_convert` wiil have the value that was set in `intiConfig()`. By default it is `True`. + +Command line key --general.profiles or appropriate key default yml file is used to find 'profiles'. +Let suppose, that --config_file is resolved to config.yml. +If 'profiles' is not empty, than it will be used to calculate filenames +that will be used to override default yml file. +Let suppose, 'profiles' resolved to ['dev', 'local']. Than first config.yml +will be loaded, than it will be overridden with config-dev.yml, than +it will be overridden with config-local.yml. +At last, it will be overridden with system args. +This entry can be always be overridden with system args. + +`ymlparsers` and `parser` modules serves as Low-Level API for this module. + +`mask_value()` implemented as a wrapper to `parsers.safe_eval()` method with support for boolean +variables. This implementation is used to get type for arguments that we get from system args. +This mechanism can be easily replaced with your own one. + +`to_convex_map()` This method receives dictionary with 'flat keys', it has simple key:value structure +where value can't be another dictionary. +It will return dictionary of dictionaries with natural key mapping, +optionally, entries will be filtered out according to white_list_flat_keys and, +optionally, value will be implicitly converted to appropriate type. + +In order to simulate dictionary of dictionaries 'flat keys' compose key from outer dict with key from inner dict +separated with dot. +For example, 'general.profiles' 'flat key' corresponds to convex map with 'general' key with dictionary as value +that have one of the keys 'profiles' with corresponding value. + +If you supply `implicit_convert=True`, than `mask_value()` will be applied to the values of the received flat dictionary. +Otherwise, `implicit_convert` wiil have the value that was set in `intiConfig()`. By default it is True. + +`merge_list_value_in_dicts` - merges value of 2 dicts. This value represents list of values. +Value from flat_d is roughly obtained by flat_d[main_key+'.'+sub_key]. +Value from d is roughly obtained by d[main_key][sub_key]. + +If you supply `implicit_convert=True`, than `mask_value()` will be applied to the flat map (first parameter). +Otherwise, `implicit_convert` wiil have the value that was set in `intiConfig()`. By default it is `True`. + +`initConfig` - you can set default value of `implicit_convert`. By default it is `True`. +This parameters is used if `implicit_convert` wasn't explicitly supplied. This method is idempotent. + +- Added `deploys` module. +This module is usable in your deployment script. See also `fabs` module. + +This method use `parsers`, ymlparsers`, `init_app_conf` as it's low-level API. `init_app_conf` usage is limited. + +The main function is `load_config()`. It is simplified method for parsing yml configuration file with optionally +overrided profiles only. See `init_app_conf.parse_config()` for another variant. + +`split_path` - Split filename in 2 part parts by split_dirname. first_part will ends with split_dirname. +second_part will start immediately after split_dirname. + +`add_to_zip_copy_function` - Factory method that returns closure that can be used as copy_function param in +`shutil.copytree()`. + +- Added `emails` module. +This module contains extensions of the logging handlers. +This module optionally depends on `ymlparseser` module. +It is better to use `EmailStatus` context manager with configured `emailLogger`. +It is intended to configure first your `emailLogger` with `OneMemoryHandler` (together with `SMTPHandler`). +Than the code block that you want to aggregate log messages from is better to be enclosed with `EmailStatus` +context manager. + +`alexber.utils.emails.SMTPHandler` is customization of `logging.handlers.SMTPHandler`. It's purpose is to connect to +SMTP server and actually send the e-mail. Unlike `logging.handlers.SMTPHandler` this class expects for record.msg to be built EmailMessage. +You can also change use of underline SMTP class to SMTP_SSL, LMTP, etc. +This implementation is *thread-safe*. + +`alexber.utils.emails.OneMemoryHandler` is variant of `logging.handlers.MemoryHandler`. This handler aggregates +log messages until `FINISHED` log-level is received or application is going to terminate abruptly (see docstring +of `calc_abrupt_vars()` method for the details) and we have some log messages in the buffer. On such event +all messages (in the current Thread) are aggregated to the single `EmailMessage`. The subject of the `EmailMessage` +is determined by `get_subject()` method. +If you want to change delimeters used to indicate variable declaration inside template, see docstring of the +`get_subject()` method. +It is better to use `EmailStatus` context manager with configured emailLogger. See docstring of `EmailStatus`. +This implementation is *thread-safe*. + +`alexber.utils.emails.EmailStatus` - if contextmanager exits with exception (it fails), than e-mail with +subject formatted with faildargs and faildkwargs will be send. +Otherwise, e-mail with subject formatted with successargs and successkwargs will be send. +All messages (in the current Thread) will be aggregated to one long e-mail with the subject described in +`OneMemoryHandler.get_subject()` method. + +`alexber.utils.emails.initConfig` - this method reset some defaults. This method is idempotent. +By default, `SMTP` class from `smtplib` is used to send actual e-mail. You can change it to `SMTP_SSL`, `LMTP`, +or another class by specifying default_smpt_cls_name. +You can also specified default port for sending e-mails. + +`processInvokes` module has one primary function - `run_sub_process()` This method run subprocess and logs it's out +to the logger. This method is sophisticated decorator to `subprocess.run()`. It is useful, when your subprocess +run's a lot of time and you're interesting to receive it's `stdout` and `stderr`. By default, it's streamed to log. +You can easily customize this behavior, see `initConig()` method. + +`initConig()` This method can be optionally called prior any call to another function in this module. You can use your +custom class for the logging. For example, FilePipe. + + +### Changed +- Spited dependency list for setup.py req.txt (inexact versions, direct dependency only) and for +reproducible installation requirements.txt (exact versions, all, including transitive dependencies). + +- README.md changed, added section 'Alternatively you install install from requirements file:'. +Some other misc changed done. + +- CHANGELOG.md version 0.4.1 misc changes. + +- Misc improvement in unit tests. + +- Fixed `parser.safe_eval` - safe_eval('%(message)s') was blow up, now it returns value as is. +See https://github.com/alex-ber/AlexBerUtils/issues/2 + +- Enhanced `importer.importer` - added support for PEP 420 (implicit Namespace Packages). +Namespace packages are a mechanism for splitting a single Python package across multiple directories on disk. +When interpreted encounter with non-empty __path__ attribute it adds modules found in those locations +to the current package. +See https://github.com/alex-ber/AlexBerUtils/issues/3 + +- In all documentation refference to `pip3` was changed to `python3 -m pip` + +## [0.4.1] - 2020-04-02 +**BREAKING CHANGE** I highly recommend not to use 0.3.X versions. +### Removed +- module `warns` is droped + +### Changed +- *Limitation:*: + +`mains` module wasn't tested with frozen python script (frozen using py2exe). + +- module `mains` is rewritten. Function `initConf` is dropped entirely. +- module `mains` now works with logger and with warnings (it was wrong decision to work with warnings). + + +## [0.3.4] - 2020-04-02 +### Changed +- CHANGELOG.md fixed +- `warns` module bug fixed, now warnings.warn() works. +- FixabscwdWarning is added to simplify warnings disabling. +- Changing how `mains` module use `warns`. + + +## [0.3.3] - 2020-04-02 +### Changed +- CHANGELOG.md fixed + + +## [0.3.2] - 2020-04-01 +### Changed +- To REAMDE.md add `Installing new version` section +- Fix typo in REAMDE.md (tests, not test). +- Fixing bug: now, you're able to import package in the Python interpreter (`setups.py` fixed) +- Fixing bug: `warns` module now doesn't change log_level in the preconfigured logger in any cases. +- **BREAKING CHANGE**: In`mains` module method `warnsInitConfig()` was renamed to `mainInitConfig()` +Also singature was changed. +- `mains` module minor refactored. + + +### Added +- Unit tests are added for `warns` module +- Unit tests are added for `mains` module + +## [0.3.1] - 2020-04-01 +### Changed +- Tests minor improvements. +- Excluded tests, data from setup.py (from being installed from the sdist.) +- Created MANIFEST.in + + +### Added +- `warns `module is added: + +It provides better integration between warnings and logger. +Unlike `logging._showwarning()` this variant will always go through logger. + +`warns.initConfig()` has optional file parameter (it's file-like object) to redirect warnings. +Default value is `sys.stderr`. + +If logger for `log_name` (default is `py.warnings`) will be configured before call to `showwarning()` method, +than warning will go to the logger's handler with `log_level` (default is `logging.WARNING`). + +If logger for `log_name` (default is `py.warnings`) willn't be configured before call to showwarning() method, +than warning will be done to `file` (default is `sys.stderr`) with `log_level` (default is `logging.WARNING`). + +- `main` module is added: + +`main.fixabscwd()` changes `os.getcwd()` to be the directory of the `__main__` module. + +`main.warnsInitConfig()` reexports `warns.initConfig()` for convenience. + + + +### Added +- Tests for alexber.utils.thread_locals added. + +## [0.2.5] - 2019-05-22 +### Changed +- Fixed bug in UploadCommand, git push should be before git tag. + + +## [0.2.4] - 2019-05-22 +### Changed +- Fixed bug in setup.py, incorrect order between VERSION and UploadCommand (no tag was created on upload) + +## [0.2.1] - 2019-05-22 +### Changed +- setup url fixed. +- Added import of Enum to alexber.utils package. + +## [0.2.0] - 2019-05-22 +### Changed +- setup.py - keywords added. + +## [0.1.1] - 2019-05-22 +### Changed +- README.md fixed typo. + +## [0.1.0] - 2019-05-22 +### Changed +- alexber.utils.UploadCommand - bug fixed, failed on git tag, because VERSION was undefined. + + +## [0.0.1] - 2019-05-22 +### Added +- alexber.utils.StrAsReprMixinEnum - Enum Mixin that has __str__() equal to __repr__(). +- alexber.utils.AutoNameMixinEnum- Enum Mixin that generate value equal to the name. +- alexber.utils.MissingNoneMixinEnum - Enum Mixin will return None if value will not be found. +- alexber.utils.LookUpMixinEnum - Enim Mixin that is designed to be used for lookup by value. + + If lookup fail, None will be return. Also, __str__() will return the same value as __repr__(). +- alexber.utils.threadlocal_var, get_threadlocal_var, del_threadlocal_var. + + Inspired by https://stackoverflow.com/questions/1408171/thread-local-storage-in-python + +- alexber.utils.UploadCommand - Support setup.py upload. + + UploadCommand is intented to be used only from setup.py + + It's builds Source and Wheel distribution. + + It's uploads the package to PyPI via Twine. + + It's pushes the git tags. + +- alexber.utils.uuid1mc is is a hybrid between version 1 & version 4. This is v1 with random MAC ("v1mc"). + + uuid1mc() is deliberately generating v1 UUIDs with a random broadcast MAC address. + + The resulting v1 UUID is time dependant (like regular v1), but lacks all host-specific information (like v4). + + Note: somebody reported that ran into trouble using UUID1 in Amazon EC2 instances. + + +- alexber.utils.importer.importer - Convert str to Python construct that target is represented. +- alexber.utils.importer.new_instance - Convert str to Python construct that target is represented. +args and kwargs will be passed in to appropriate __new__() / __init__() / __init_subclass__() methods. +- alexber.utils.inspects.issetdescriptor - Return true if the object is a method descriptor with setters. + + But not if ismethod() or isclass() or isfunction() are true. +- alexber.utils.inspects.ismethod - Return false if object is not a class and not a function. +Otherwise, return true iff signature has 2 params. +- alexber.utils.parsers.safe_eval - The purpose of this function is convert numbers from str to correct type. + + This function support convertion of built-in Python number to correct type (int, float) + + This function doesn't support decimal.Decimal or datetime.datetime or numpy types. +- alexber.utils.parsers.is_empty - if value is None returns True. + + if value is empty iterable (for example, empty str or emptry list),returns true otherwise false. + + Note: For not iterable values, behaivour is undefined. +- alexber.utils.parsers.parse_boolean - if value is None returns None. + + if value is boolean, it is returned as it is. + if value is str and value is equals ignoring case to "True", True is returned. + if value is str and value is equals ignoring case to "False", False is returned. + + For every other value, the answer is undefined. + + + +- alexber.utils.props.Properties - A Python replacement for java.util.Properties class + + This is modelled as closely as possible to the Java original. + + Created - Anand B Pillai <abpillai@gmail.com>. + + Update to Python 3 by Alex. + + Also there are some tweeks that was done by Alex. + +<!-- +### Changed +### Removed +--> + + + + +%prep +%autosetup -n alex-ber-utils-0.6.6 + +%build +%py3_build + +%install +%py3_install +install -d -m755 %{buildroot}/%{_pkgdocdir} +if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi +if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi +if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi +if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi +pushd %{buildroot} +if [ -d usr/lib ]; then + find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/lib64 ]; then + find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/bin ]; then + find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/sbin ]; then + find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst +fi +touch doclist.lst +if [ -d usr/share/man ]; then + find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst +fi +popd +mv %{buildroot}/filelist.lst . +mv %{buildroot}/doclist.lst . + +%files -n python3-alex-ber-utils -f filelist.lst +%dir %{python3_sitelib}/* + +%files help -f doclist.lst +%{_docdir}/* + +%changelog +* Wed May 10 2023 Python_Bot <Python_Bot@openeuler.org> - 0.6.6-1 +- Package Spec generated |