diff options
author | CoprDistGit <infra@openeuler.org> | 2023-05-10 08:08:14 +0000 |
---|---|---|
committer | CoprDistGit <infra@openeuler.org> | 2023-05-10 08:08:14 +0000 |
commit | 1c80a8608d18da067446978c5ff6c5802d6ca959 (patch) | |
tree | c77f19d84715ea9102d5a978322c4103358ae29c | |
parent | 5b304f674f530e3ef1bcb3b998858bf74b53a5d7 (diff) |
automatic import of python-ts-ids-validatoropeneuler20.03
-rw-r--r-- | .gitignore | 1 | ||||
-rw-r--r-- | python-ts-ids-validator.spec | 822 | ||||
-rw-r--r-- | sources | 1 |
3 files changed, 824 insertions, 0 deletions
@@ -0,0 +1 @@ +/ts_ids_validator-0.9.14.tar.gz diff --git a/python-ts-ids-validator.spec b/python-ts-ids-validator.spec new file mode 100644 index 0000000..9f151d4 --- /dev/null +++ b/python-ts-ids-validator.spec @@ -0,0 +1,822 @@ +%global _empty_manifest_terminate_build 0 +Name: python-ts-ids-validator +Version: 0.9.14 +Release: 1 +Summary: Python utility for validating IDS +License: Apache-2.0 +URL: https://developers.tetrascience.com +Source0: https://mirrors.nju.edu.cn/pypi/web/packages/48/61/3e715408ef5ee9cd27c7f58f6c886a03f6d5553af255d0084259f42c541d/ts_ids_validator-0.9.14.tar.gz +BuildArch: noarch + +Requires: python3-jsonref +Requires: python3-jsonschema +Requires: python3-pydantic +Requires: python3-pydash +Requires: python3-rich +Requires: python3-ts-ids-es-json-generator + +%description +# TetraScience IDS Validator <!-- omit in toc --> + +## Table of Contents <!-- omit in toc --> + +- [Overview](#overview) +- [Usage](#usage) +- [Components](#components) + - [Node](#node) + - [Checker Classes](#checker-classes) + - [Validator](#validator) + - [List of Checker Classes](#list-of-checker-classes) + - [Base Classes](#base-classes) + - [Generic](#generic) + - [V1](#v1) + - [Writing New Checks](#writing-new-checks) + - [Extending Checkers Classes](#extending-checkers-classes) + - [Pattern 1](#pattern-1) + - [Pattern 2](#pattern-2) + - [Running Checks for Specific Nodes](#running-checks-for-specific-nodes) + - [List of Checks for Validator](#list-of-checks-for-validator) +- [Changelog](#changelog) +- [v0.9.14](#v0914) +- [v0.9.13](#v0913) +- [v0.9.12](#v0912) +- [v0.9.11](#v0911) +- [v0.9.10](#v0910) +- [v0.9.9](#v099) +- [v0.9.8](#v098) +- [v0.9.7](#v097) +- [v0.9.6](#v096) + +## Overview + +The TetraScience IDS Validator checks that IDS artifacts follow a set of rules which +make them compatible with the Tetra Data Platform, and optionally that they are +compatible with with additional IDS design conventions. +The validator either passes or fails with a list of the checks which led to the failure. + +The validator checks these files in an IDS folder: + +- schema.json +- elasticsearch.json +- athena.json + +You can find the validation rules in: + +- [IDS Design Conventions - schema.json](https://developers.tetrascience.com/docs/ids-design-conventions-schemajson) +- [IDS Design Conventions - elasticsearch.json](https://developers.tetrascience.com/docs/ids-design-conventions-elasticsearchjson) +- [IDS Design Conventions - athena.json](https://developers.tetrascience.com/docs/ids-design-conventions-athenajson) + +## Version <!-- omit in toc --> + +v0.9.14 + +## Usage + +```bash +poetry run python -m ids_validator --ids_dir=path/to/ids/folder +``` + +This will run the required checks for the `@idsConventionVersion` mentioned in `schema.json`. + +If `@idsConventionVersion` is missing in `schema.json` or if it is not supported by `schema_validator`, only `generic` checks will be run. + +## Components + +### Node + +- `Node: UserDict` class is an abstraction for `dict` in `schema.json` +- When crawling `schema.json`, each `key-value` pair where `value` is a `dict`, is casted into `Node` type. +- For each K_V pair, `Node` has following attributes + - `name (default=root)`: The `key` + - `data`: The `value:dict` + - `path (default=root)`: The fully-qualified path for the `key` in `schema.json` +- File: [ids_node.py](src/ids_node.py) + +### Checker Classes + +- A checker class must implement `AbstractChecker` +- When crawling `schema.json`, its `run()` method will be called for each node. +- `run()` implements the rules/condition to be checked for validating the node. +- `run()` accepts two arguments: + - `node: Node`: `Node` for which we are running the checks + - `context: dict` + - It contains python dicts for `schema.json`, `athena.json` and `convention_version`. + - It is used to supplementary data required for running complex checks. + +### Validator + +- `Validator` class is the one that implements the crawler. +- It has following attributes: + - `ids: dict`: `schema.json` converted to python `dict` + - `athena: dict`: `athena.json` converted to python `dict` + - `checks_list`: A list of instantiated checker classes. + These list of checks will be run for each node +- `Validator.traverse_ids()` crawls from `Node` to `Node` in `ids:dict`, Calling `run()` for each checker in the checks_list on the node + +### List of Checker Classes + +#### Base Classes + +- `AbstractChecker` + + - Every checker class must implement it. + - File: [abstract_checker.py](src/checks/abstract_checker.py) + +- `RuleBasedChecker` + - It is base class that allows validating `Node` against a set of `rules` + - It comes in handy for implementing checks for property Nodes that has predefined template + - The child class inheriting `RulesBasedChecker` must define `rules` + - `rules` is a `dict` that maps `Node.path` to `set of rules:dict` + - The `set of rules` for a `Node.path` may contain following items: + - `"type"`, `Union[List[str], str]`: defines what should be the `type` value for the `Node` + - `"compatible_type"`, `ids_validator.checks.rules_checker.BackwardCompatibleType`: + defines the allowable `type` values for a `Node`, matching either a `preferred` + type, or one of a list of `deprecated` types which will lead to a warning. + - `"min_properties"`, `List[str]`: defines minimum set of property names that must exist for the Node. More properties can exist in addition to `min_properties` + - `"constrained_properties"`, `ids_validator.checks.rules_checker.PropertyConstraints`: defines the minimum properties which must be present in a `Node`'s properties, along with extra properties which may or may not be present. + - `"min_required"`, `List[str]`: The required list of the `Node` must at least contain the values mentioned in `min_required` + - `"required"`, `List[str]`: The required list of the `Node` must only contain values listed in `required` +- Rules based checkers defined for v1 conventions can be found [here](src/checks/v1/nodes_checker.py) + +#### Generic + +- `AdditionalPropertyChecker`: [additional_property.py](src/checks/generic/additional_property.py) +- `RequiredPropertiesChecker`: [required_property.py](src/checks/generic/required_property.py) +- `DatacubesChecker`: [datacubes.py](src/checks/generic/datacubes.py) +- `RootNodeChecker`: [root_node.py](src/checks/generic/root_node.py) +- `TypeChecker`: [type_check.py](src/checks/generic/type_check.py) +- `AthenaChecker`: [athena.py](src/checks/generic/athena.py) + +#### V1 + +- `V1ChildNameChecker`: [child_name.py](src/checks/v1/child_name.py) +- `V1ConventionVersionChecker`: [convention_version_check.py](src/checks/v1/convention_version_check.py) +- `V1SystemNodeChecker`: [nodes_checker.py](src/checks/v1/nodes_checker.py) +- `V1SampleNodeChecker`: [nodes_checker.py](src/checks/v1/nodes_checker.py) +- `V1UserNodeChecker`: [nodes_checker.py](src/checks/v1/nodes_checker.py) +- `V1RootNodeChecker`: [root_node.py](src/checks/v1/root_node.py) +- `V1SnakeCaseChecker`: [snake_case.py](src/checks/v1/snake_case.py) + +### Writing New Checks + +- Checkers must implement `AbstractCheckers` +- `run()` method implement one or more checks for the node +- In case of no failure an empty list must be returned +- In case of failures, it must return a list of one or more tuple +- The tuple will contain two values + - `log message:str`: The message to be logged when check fails + - `criticality`: either `Log.CRITICAL` or `Log.WARNING` + +### Extending Checkers Classes + +#### Pattern 1 + +```python +class ChildChecker(ParentChecker): + def run(node: Node, context: dict): + logs = [] + # Implement new checks and append failure to logs + + # Run Parent checkers and append logs + logs += super().run(node, context) + return logs +``` + +If `check_list` passed to `Validator` contains the `ChildChecker`, then it must not contain `ParentChecker` in the same list. +Doing so will cause ParentCheck to run twice and populate failures logs if any, twice. + +#### Pattern 2 + +```python +class ChildChecker(ParentChecker): + def run(node: Node, context: dict): + logs = [] + # Implement new checks and append failure to logs + # use or override helper function of the parent class + return logs +``` + +### Running Checks for Specific Nodes + +```python +class AdhocChecker(AbstractChecker): + def run(node: Node, context: dict): + logs = [] + paths = [] + # paths is a list of fully qualified path to a key in schema.json + # each path must start form root + # eg: root.samples + # eg: root.samples.items.properties.property_name + if node.path in paths: + # Implement new checks and append failure to logs + logs += perform_new_checks(node, context) + return logs +``` + +### List of Checks for Validator + +- `checks_dict`, defined [here](src/checks/__init__.py), maps the `type of validation` that we want to perform to the `list the of checks` needed to be run for the validation +- The list off checks is actually a list of instantiated checker objects + +## Changelog + +## v0.9.14 + +- Update `samples[*]` check to optionally allow for it to contain a property `pk_samples` of type `"string"`. + +## v0.9.13 + +- `related_files` is no longer checked against annotation fields like "description". + +## v0.9.12 + +- Update check for `samples[*].labels[*].source.name` type: previously the type was + required to be `"string"`, now it is required to be either `["string", "null"]` or + `"string"`, with `"string"` leading to a deprecation warning. This change makes this + `source` definition the same as `samples[*].properties[*].source` in a + backward-compatible way. + +## v0.9.11 + +- Fix bug in `AthenaChecker` to allow root level IDS properties as partition paths. +- Update `TypeChecker` to catch errors related to undefined/misspelled `type` key. +- Update `jsonschema` version to fix package installation error + +## v0.9.10 + +- Modify `V1SnakeCaseChecker` to ignore checks for keys present in `definitions` object. +- Add temporary allowance for `@link` in `*.properties` + +## v0.9.9 + +- Lock `jsonschema` version in requirements.txt + +## v0.9.8 + +- Modify `RulesChecker` to log missing and extra properties + +## v0.9.7 + +- Allow properties with `const` values to have non-nullable `type` + +## v0.9.6 + +- Add checker classes for generic validation +- Add checker classes for v1.0.0 convention validation + + + +%package -n python3-ts-ids-validator +Summary: Python utility for validating IDS +Provides: python-ts-ids-validator +BuildRequires: python3-devel +BuildRequires: python3-setuptools +BuildRequires: python3-pip +%description -n python3-ts-ids-validator +# TetraScience IDS Validator <!-- omit in toc --> + +## Table of Contents <!-- omit in toc --> + +- [Overview](#overview) +- [Usage](#usage) +- [Components](#components) + - [Node](#node) + - [Checker Classes](#checker-classes) + - [Validator](#validator) + - [List of Checker Classes](#list-of-checker-classes) + - [Base Classes](#base-classes) + - [Generic](#generic) + - [V1](#v1) + - [Writing New Checks](#writing-new-checks) + - [Extending Checkers Classes](#extending-checkers-classes) + - [Pattern 1](#pattern-1) + - [Pattern 2](#pattern-2) + - [Running Checks for Specific Nodes](#running-checks-for-specific-nodes) + - [List of Checks for Validator](#list-of-checks-for-validator) +- [Changelog](#changelog) +- [v0.9.14](#v0914) +- [v0.9.13](#v0913) +- [v0.9.12](#v0912) +- [v0.9.11](#v0911) +- [v0.9.10](#v0910) +- [v0.9.9](#v099) +- [v0.9.8](#v098) +- [v0.9.7](#v097) +- [v0.9.6](#v096) + +## Overview + +The TetraScience IDS Validator checks that IDS artifacts follow a set of rules which +make them compatible with the Tetra Data Platform, and optionally that they are +compatible with with additional IDS design conventions. +The validator either passes or fails with a list of the checks which led to the failure. + +The validator checks these files in an IDS folder: + +- schema.json +- elasticsearch.json +- athena.json + +You can find the validation rules in: + +- [IDS Design Conventions - schema.json](https://developers.tetrascience.com/docs/ids-design-conventions-schemajson) +- [IDS Design Conventions - elasticsearch.json](https://developers.tetrascience.com/docs/ids-design-conventions-elasticsearchjson) +- [IDS Design Conventions - athena.json](https://developers.tetrascience.com/docs/ids-design-conventions-athenajson) + +## Version <!-- omit in toc --> + +v0.9.14 + +## Usage + +```bash +poetry run python -m ids_validator --ids_dir=path/to/ids/folder +``` + +This will run the required checks for the `@idsConventionVersion` mentioned in `schema.json`. + +If `@idsConventionVersion` is missing in `schema.json` or if it is not supported by `schema_validator`, only `generic` checks will be run. + +## Components + +### Node + +- `Node: UserDict` class is an abstraction for `dict` in `schema.json` +- When crawling `schema.json`, each `key-value` pair where `value` is a `dict`, is casted into `Node` type. +- For each K_V pair, `Node` has following attributes + - `name (default=root)`: The `key` + - `data`: The `value:dict` + - `path (default=root)`: The fully-qualified path for the `key` in `schema.json` +- File: [ids_node.py](src/ids_node.py) + +### Checker Classes + +- A checker class must implement `AbstractChecker` +- When crawling `schema.json`, its `run()` method will be called for each node. +- `run()` implements the rules/condition to be checked for validating the node. +- `run()` accepts two arguments: + - `node: Node`: `Node` for which we are running the checks + - `context: dict` + - It contains python dicts for `schema.json`, `athena.json` and `convention_version`. + - It is used to supplementary data required for running complex checks. + +### Validator + +- `Validator` class is the one that implements the crawler. +- It has following attributes: + - `ids: dict`: `schema.json` converted to python `dict` + - `athena: dict`: `athena.json` converted to python `dict` + - `checks_list`: A list of instantiated checker classes. + These list of checks will be run for each node +- `Validator.traverse_ids()` crawls from `Node` to `Node` in `ids:dict`, Calling `run()` for each checker in the checks_list on the node + +### List of Checker Classes + +#### Base Classes + +- `AbstractChecker` + + - Every checker class must implement it. + - File: [abstract_checker.py](src/checks/abstract_checker.py) + +- `RuleBasedChecker` + - It is base class that allows validating `Node` against a set of `rules` + - It comes in handy for implementing checks for property Nodes that has predefined template + - The child class inheriting `RulesBasedChecker` must define `rules` + - `rules` is a `dict` that maps `Node.path` to `set of rules:dict` + - The `set of rules` for a `Node.path` may contain following items: + - `"type"`, `Union[List[str], str]`: defines what should be the `type` value for the `Node` + - `"compatible_type"`, `ids_validator.checks.rules_checker.BackwardCompatibleType`: + defines the allowable `type` values for a `Node`, matching either a `preferred` + type, or one of a list of `deprecated` types which will lead to a warning. + - `"min_properties"`, `List[str]`: defines minimum set of property names that must exist for the Node. More properties can exist in addition to `min_properties` + - `"constrained_properties"`, `ids_validator.checks.rules_checker.PropertyConstraints`: defines the minimum properties which must be present in a `Node`'s properties, along with extra properties which may or may not be present. + - `"min_required"`, `List[str]`: The required list of the `Node` must at least contain the values mentioned in `min_required` + - `"required"`, `List[str]`: The required list of the `Node` must only contain values listed in `required` +- Rules based checkers defined for v1 conventions can be found [here](src/checks/v1/nodes_checker.py) + +#### Generic + +- `AdditionalPropertyChecker`: [additional_property.py](src/checks/generic/additional_property.py) +- `RequiredPropertiesChecker`: [required_property.py](src/checks/generic/required_property.py) +- `DatacubesChecker`: [datacubes.py](src/checks/generic/datacubes.py) +- `RootNodeChecker`: [root_node.py](src/checks/generic/root_node.py) +- `TypeChecker`: [type_check.py](src/checks/generic/type_check.py) +- `AthenaChecker`: [athena.py](src/checks/generic/athena.py) + +#### V1 + +- `V1ChildNameChecker`: [child_name.py](src/checks/v1/child_name.py) +- `V1ConventionVersionChecker`: [convention_version_check.py](src/checks/v1/convention_version_check.py) +- `V1SystemNodeChecker`: [nodes_checker.py](src/checks/v1/nodes_checker.py) +- `V1SampleNodeChecker`: [nodes_checker.py](src/checks/v1/nodes_checker.py) +- `V1UserNodeChecker`: [nodes_checker.py](src/checks/v1/nodes_checker.py) +- `V1RootNodeChecker`: [root_node.py](src/checks/v1/root_node.py) +- `V1SnakeCaseChecker`: [snake_case.py](src/checks/v1/snake_case.py) + +### Writing New Checks + +- Checkers must implement `AbstractCheckers` +- `run()` method implement one or more checks for the node +- In case of no failure an empty list must be returned +- In case of failures, it must return a list of one or more tuple +- The tuple will contain two values + - `log message:str`: The message to be logged when check fails + - `criticality`: either `Log.CRITICAL` or `Log.WARNING` + +### Extending Checkers Classes + +#### Pattern 1 + +```python +class ChildChecker(ParentChecker): + def run(node: Node, context: dict): + logs = [] + # Implement new checks and append failure to logs + + # Run Parent checkers and append logs + logs += super().run(node, context) + return logs +``` + +If `check_list` passed to `Validator` contains the `ChildChecker`, then it must not contain `ParentChecker` in the same list. +Doing so will cause ParentCheck to run twice and populate failures logs if any, twice. + +#### Pattern 2 + +```python +class ChildChecker(ParentChecker): + def run(node: Node, context: dict): + logs = [] + # Implement new checks and append failure to logs + # use or override helper function of the parent class + return logs +``` + +### Running Checks for Specific Nodes + +```python +class AdhocChecker(AbstractChecker): + def run(node: Node, context: dict): + logs = [] + paths = [] + # paths is a list of fully qualified path to a key in schema.json + # each path must start form root + # eg: root.samples + # eg: root.samples.items.properties.property_name + if node.path in paths: + # Implement new checks and append failure to logs + logs += perform_new_checks(node, context) + return logs +``` + +### List of Checks for Validator + +- `checks_dict`, defined [here](src/checks/__init__.py), maps the `type of validation` that we want to perform to the `list the of checks` needed to be run for the validation +- The list off checks is actually a list of instantiated checker objects + +## Changelog + +## v0.9.14 + +- Update `samples[*]` check to optionally allow for it to contain a property `pk_samples` of type `"string"`. + +## v0.9.13 + +- `related_files` is no longer checked against annotation fields like "description". + +## v0.9.12 + +- Update check for `samples[*].labels[*].source.name` type: previously the type was + required to be `"string"`, now it is required to be either `["string", "null"]` or + `"string"`, with `"string"` leading to a deprecation warning. This change makes this + `source` definition the same as `samples[*].properties[*].source` in a + backward-compatible way. + +## v0.9.11 + +- Fix bug in `AthenaChecker` to allow root level IDS properties as partition paths. +- Update `TypeChecker` to catch errors related to undefined/misspelled `type` key. +- Update `jsonschema` version to fix package installation error + +## v0.9.10 + +- Modify `V1SnakeCaseChecker` to ignore checks for keys present in `definitions` object. +- Add temporary allowance for `@link` in `*.properties` + +## v0.9.9 + +- Lock `jsonschema` version in requirements.txt + +## v0.9.8 + +- Modify `RulesChecker` to log missing and extra properties + +## v0.9.7 + +- Allow properties with `const` values to have non-nullable `type` + +## v0.9.6 + +- Add checker classes for generic validation +- Add checker classes for v1.0.0 convention validation + + + +%package help +Summary: Development documents and examples for ts-ids-validator +Provides: python3-ts-ids-validator-doc +%description help +# TetraScience IDS Validator <!-- omit in toc --> + +## Table of Contents <!-- omit in toc --> + +- [Overview](#overview) +- [Usage](#usage) +- [Components](#components) + - [Node](#node) + - [Checker Classes](#checker-classes) + - [Validator](#validator) + - [List of Checker Classes](#list-of-checker-classes) + - [Base Classes](#base-classes) + - [Generic](#generic) + - [V1](#v1) + - [Writing New Checks](#writing-new-checks) + - [Extending Checkers Classes](#extending-checkers-classes) + - [Pattern 1](#pattern-1) + - [Pattern 2](#pattern-2) + - [Running Checks for Specific Nodes](#running-checks-for-specific-nodes) + - [List of Checks for Validator](#list-of-checks-for-validator) +- [Changelog](#changelog) +- [v0.9.14](#v0914) +- [v0.9.13](#v0913) +- [v0.9.12](#v0912) +- [v0.9.11](#v0911) +- [v0.9.10](#v0910) +- [v0.9.9](#v099) +- [v0.9.8](#v098) +- [v0.9.7](#v097) +- [v0.9.6](#v096) + +## Overview + +The TetraScience IDS Validator checks that IDS artifacts follow a set of rules which +make them compatible with the Tetra Data Platform, and optionally that they are +compatible with with additional IDS design conventions. +The validator either passes or fails with a list of the checks which led to the failure. + +The validator checks these files in an IDS folder: + +- schema.json +- elasticsearch.json +- athena.json + +You can find the validation rules in: + +- [IDS Design Conventions - schema.json](https://developers.tetrascience.com/docs/ids-design-conventions-schemajson) +- [IDS Design Conventions - elasticsearch.json](https://developers.tetrascience.com/docs/ids-design-conventions-elasticsearchjson) +- [IDS Design Conventions - athena.json](https://developers.tetrascience.com/docs/ids-design-conventions-athenajson) + +## Version <!-- omit in toc --> + +v0.9.14 + +## Usage + +```bash +poetry run python -m ids_validator --ids_dir=path/to/ids/folder +``` + +This will run the required checks for the `@idsConventionVersion` mentioned in `schema.json`. + +If `@idsConventionVersion` is missing in `schema.json` or if it is not supported by `schema_validator`, only `generic` checks will be run. + +## Components + +### Node + +- `Node: UserDict` class is an abstraction for `dict` in `schema.json` +- When crawling `schema.json`, each `key-value` pair where `value` is a `dict`, is casted into `Node` type. +- For each K_V pair, `Node` has following attributes + - `name (default=root)`: The `key` + - `data`: The `value:dict` + - `path (default=root)`: The fully-qualified path for the `key` in `schema.json` +- File: [ids_node.py](src/ids_node.py) + +### Checker Classes + +- A checker class must implement `AbstractChecker` +- When crawling `schema.json`, its `run()` method will be called for each node. +- `run()` implements the rules/condition to be checked for validating the node. +- `run()` accepts two arguments: + - `node: Node`: `Node` for which we are running the checks + - `context: dict` + - It contains python dicts for `schema.json`, `athena.json` and `convention_version`. + - It is used to supplementary data required for running complex checks. + +### Validator + +- `Validator` class is the one that implements the crawler. +- It has following attributes: + - `ids: dict`: `schema.json` converted to python `dict` + - `athena: dict`: `athena.json` converted to python `dict` + - `checks_list`: A list of instantiated checker classes. + These list of checks will be run for each node +- `Validator.traverse_ids()` crawls from `Node` to `Node` in `ids:dict`, Calling `run()` for each checker in the checks_list on the node + +### List of Checker Classes + +#### Base Classes + +- `AbstractChecker` + + - Every checker class must implement it. + - File: [abstract_checker.py](src/checks/abstract_checker.py) + +- `RuleBasedChecker` + - It is base class that allows validating `Node` against a set of `rules` + - It comes in handy for implementing checks for property Nodes that has predefined template + - The child class inheriting `RulesBasedChecker` must define `rules` + - `rules` is a `dict` that maps `Node.path` to `set of rules:dict` + - The `set of rules` for a `Node.path` may contain following items: + - `"type"`, `Union[List[str], str]`: defines what should be the `type` value for the `Node` + - `"compatible_type"`, `ids_validator.checks.rules_checker.BackwardCompatibleType`: + defines the allowable `type` values for a `Node`, matching either a `preferred` + type, or one of a list of `deprecated` types which will lead to a warning. + - `"min_properties"`, `List[str]`: defines minimum set of property names that must exist for the Node. More properties can exist in addition to `min_properties` + - `"constrained_properties"`, `ids_validator.checks.rules_checker.PropertyConstraints`: defines the minimum properties which must be present in a `Node`'s properties, along with extra properties which may or may not be present. + - `"min_required"`, `List[str]`: The required list of the `Node` must at least contain the values mentioned in `min_required` + - `"required"`, `List[str]`: The required list of the `Node` must only contain values listed in `required` +- Rules based checkers defined for v1 conventions can be found [here](src/checks/v1/nodes_checker.py) + +#### Generic + +- `AdditionalPropertyChecker`: [additional_property.py](src/checks/generic/additional_property.py) +- `RequiredPropertiesChecker`: [required_property.py](src/checks/generic/required_property.py) +- `DatacubesChecker`: [datacubes.py](src/checks/generic/datacubes.py) +- `RootNodeChecker`: [root_node.py](src/checks/generic/root_node.py) +- `TypeChecker`: [type_check.py](src/checks/generic/type_check.py) +- `AthenaChecker`: [athena.py](src/checks/generic/athena.py) + +#### V1 + +- `V1ChildNameChecker`: [child_name.py](src/checks/v1/child_name.py) +- `V1ConventionVersionChecker`: [convention_version_check.py](src/checks/v1/convention_version_check.py) +- `V1SystemNodeChecker`: [nodes_checker.py](src/checks/v1/nodes_checker.py) +- `V1SampleNodeChecker`: [nodes_checker.py](src/checks/v1/nodes_checker.py) +- `V1UserNodeChecker`: [nodes_checker.py](src/checks/v1/nodes_checker.py) +- `V1RootNodeChecker`: [root_node.py](src/checks/v1/root_node.py) +- `V1SnakeCaseChecker`: [snake_case.py](src/checks/v1/snake_case.py) + +### Writing New Checks + +- Checkers must implement `AbstractCheckers` +- `run()` method implement one or more checks for the node +- In case of no failure an empty list must be returned +- In case of failures, it must return a list of one or more tuple +- The tuple will contain two values + - `log message:str`: The message to be logged when check fails + - `criticality`: either `Log.CRITICAL` or `Log.WARNING` + +### Extending Checkers Classes + +#### Pattern 1 + +```python +class ChildChecker(ParentChecker): + def run(node: Node, context: dict): + logs = [] + # Implement new checks and append failure to logs + + # Run Parent checkers and append logs + logs += super().run(node, context) + return logs +``` + +If `check_list` passed to `Validator` contains the `ChildChecker`, then it must not contain `ParentChecker` in the same list. +Doing so will cause ParentCheck to run twice and populate failures logs if any, twice. + +#### Pattern 2 + +```python +class ChildChecker(ParentChecker): + def run(node: Node, context: dict): + logs = [] + # Implement new checks and append failure to logs + # use or override helper function of the parent class + return logs +``` + +### Running Checks for Specific Nodes + +```python +class AdhocChecker(AbstractChecker): + def run(node: Node, context: dict): + logs = [] + paths = [] + # paths is a list of fully qualified path to a key in schema.json + # each path must start form root + # eg: root.samples + # eg: root.samples.items.properties.property_name + if node.path in paths: + # Implement new checks and append failure to logs + logs += perform_new_checks(node, context) + return logs +``` + +### List of Checks for Validator + +- `checks_dict`, defined [here](src/checks/__init__.py), maps the `type of validation` that we want to perform to the `list the of checks` needed to be run for the validation +- The list off checks is actually a list of instantiated checker objects + +## Changelog + +## v0.9.14 + +- Update `samples[*]` check to optionally allow for it to contain a property `pk_samples` of type `"string"`. + +## v0.9.13 + +- `related_files` is no longer checked against annotation fields like "description". + +## v0.9.12 + +- Update check for `samples[*].labels[*].source.name` type: previously the type was + required to be `"string"`, now it is required to be either `["string", "null"]` or + `"string"`, with `"string"` leading to a deprecation warning. This change makes this + `source` definition the same as `samples[*].properties[*].source` in a + backward-compatible way. + +## v0.9.11 + +- Fix bug in `AthenaChecker` to allow root level IDS properties as partition paths. +- Update `TypeChecker` to catch errors related to undefined/misspelled `type` key. +- Update `jsonschema` version to fix package installation error + +## v0.9.10 + +- Modify `V1SnakeCaseChecker` to ignore checks for keys present in `definitions` object. +- Add temporary allowance for `@link` in `*.properties` + +## v0.9.9 + +- Lock `jsonschema` version in requirements.txt + +## v0.9.8 + +- Modify `RulesChecker` to log missing and extra properties + +## v0.9.7 + +- Allow properties with `const` values to have non-nullable `type` + +## v0.9.6 + +- Add checker classes for generic validation +- Add checker classes for v1.0.0 convention validation + + + +%prep +%autosetup -n ts-ids-validator-0.9.14 + +%build +%py3_build + +%install +%py3_install +install -d -m755 %{buildroot}/%{_pkgdocdir} +if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi +if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi +if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi +if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi +pushd %{buildroot} +if [ -d usr/lib ]; then + find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/lib64 ]; then + find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/bin ]; then + find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/sbin ]; then + find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst +fi +touch doclist.lst +if [ -d usr/share/man ]; then + find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst +fi +popd +mv %{buildroot}/filelist.lst . +mv %{buildroot}/doclist.lst . + +%files -n python3-ts-ids-validator -f filelist.lst +%dir %{python3_sitelib}/* + +%files help -f doclist.lst +%{_docdir}/* + +%changelog +* Wed May 10 2023 Python_Bot <Python_Bot@openeuler.org> - 0.9.14-1 +- Package Spec generated @@ -0,0 +1 @@ +7820a5ed097396d4e7b30316b63a51f6 ts_ids_validator-0.9.14.tar.gz |