diff options
| -rw-r--r-- | .gitignore | 1 | ||||
| -rw-r--r-- | python-ayeaye.spec | 370 | ||||
| -rw-r--r-- | sources | 1 |
3 files changed, 372 insertions, 0 deletions
@@ -0,0 +1 @@ +/ayeaye-0.0.45.tar.gz diff --git a/python-ayeaye.spec b/python-ayeaye.spec new file mode 100644 index 0000000..5340619 --- /dev/null +++ b/python-ayeaye.spec @@ -0,0 +1,370 @@ +%global _empty_manifest_terminate_build 0 +Name: python-ayeaye +Version: 0.0.45 +Release: 1 +Summary: An ETL (Extract, Transform, Load) framework +License: Apache Software License +URL: https://github.com/Aye-Aye-Dev/AyeAye +Source0: https://mirrors.aliyun.com/pypi/web/packages/12/d4/36db2468a349a551797f7eae9d1dc2727ab8d0074aa4e70a7486f2f25b12/ayeaye-0.0.45.tar.gz +BuildArch: noarch + +Requires: python3-requests +Requires: python3-boto3 +Requires: python3-smart-open +Requires: python3-smart-open + +%description +# Aye Aye + +An ETL (Extract, Transform, Load) framework. + +## Quick install + +In the virtual environment for the project you’d like to use Aye Aye in, run:- + +```shell +pip install ayeaye +``` + +## Quick start + +Use [Pipenv](https://pipenv.pypa.io/en/latest/) to manage a python virtual environment and package management0 + +```shell +pipenv shell +pipenv install ayeaye +``` + +Within the environment created by pipenv above, run one of the examples:- + +```shell +curl "https://raw.githubusercontent.com/Aye-Aye-Dev/AyeAye/master/examples/poisonous_animals.py" \ + --output poisonous_animals.py +mkdir data +curl https://raw.githubusercontent.com/Aye-Aye-Dev/AyeAye/master/examples/data/poisonous_animals.json \ + --output data/poisonous_animals.json +python poisonous_animals.py +``` + +This model takes a small input dataset of animals and collates them by the country they are found. It doesn't write to a dataset, it just outputs a log. The log for this example contains the name of the country and the animals found there. + +There are more examples in the [Aye-Aye-Recipes](https://github.com/Aye-Aye-Dev/Aye-Aye-Recipes) git repo. + + +## Overview + +An Aye Aye ETL *model* inherits from `ayeaye.model` and uses class level variables to declare *connectors* to the data it acts on. + +Example:- + +```python +import ayeaye + +class PoisonousAnimals(ayeaye.Model): + poisonous_animals = ayeaye.Connect(engine_url='json://data/poisonous_animals.json') +``` + +When instantiated, `self.poisonous_animals` will be a *dataset* that ETL operations can be done with. + +The `engine_url` parameter passed to `ayeaye.Connect` is specifying the dataset type JSON in this case) and exact location for the data (`data/poisonous_animals.json` is a relative file path). + +Instead of `engine_url` you could also specify a `ref` and this uses the data catalogue to lookup the `engine_url`. (TODO this feature is coming soon!). When used this way, `ayeaye.Connect` is responsible for resolving the `ref` to an `engine_url` and passing this to a subclass of `ayeaye.connectors.base.DataConnector` which can read and maybe write this data type. + + +## Unit tests + +Ensure the working directory is the base Aye Aye directory (i.e. the same directory as the Pipfile): +```shell +pipenv install --dev +export PYTHONPATH=`pwd`/lib +pipenv run python -m unittest discover +``` + +## Development version + +To use the latest code in editable mode- + +```shell +pipenv install -e git+https://github.com/Aye-Aye-Dev/AyeAye#egg=ayeaye +``` + +When `venv` is being used, add this line to `requirements.txt`- + +``` +git+https://github.com/Aye-Aye-Dev/AyeAye#egg=ayeaye +``` + +## Optional extras + +Extra dependencies for API usage within Aye-aye models can be installed like this: + +```shell +pipenv install "ayeaye[api]" +``` + +| Label | Functionality | +| --- | --- | +| api | Restful JSON via http(s) | +| aws | File based connectors can use Amazon Web Service S3 file storage | +| compression | On the fly compression for file based connectors | + + +## License + +Aye Aye is distributed under the terms of the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0.html) and Copyright Progressive Logic Limit 2021 and onwards. + + +%package -n python3-ayeaye +Summary: An ETL (Extract, Transform, Load) framework +Provides: python-ayeaye +BuildRequires: python3-devel +BuildRequires: python3-setuptools +BuildRequires: python3-pip +%description -n python3-ayeaye +# Aye Aye + +An ETL (Extract, Transform, Load) framework. + +## Quick install + +In the virtual environment for the project you’d like to use Aye Aye in, run:- + +```shell +pip install ayeaye +``` + +## Quick start + +Use [Pipenv](https://pipenv.pypa.io/en/latest/) to manage a python virtual environment and package management0 + +```shell +pipenv shell +pipenv install ayeaye +``` + +Within the environment created by pipenv above, run one of the examples:- + +```shell +curl "https://raw.githubusercontent.com/Aye-Aye-Dev/AyeAye/master/examples/poisonous_animals.py" \ + --output poisonous_animals.py +mkdir data +curl https://raw.githubusercontent.com/Aye-Aye-Dev/AyeAye/master/examples/data/poisonous_animals.json \ + --output data/poisonous_animals.json +python poisonous_animals.py +``` + +This model takes a small input dataset of animals and collates them by the country they are found. It doesn't write to a dataset, it just outputs a log. The log for this example contains the name of the country and the animals found there. + +There are more examples in the [Aye-Aye-Recipes](https://github.com/Aye-Aye-Dev/Aye-Aye-Recipes) git repo. + + +## Overview + +An Aye Aye ETL *model* inherits from `ayeaye.model` and uses class level variables to declare *connectors* to the data it acts on. + +Example:- + +```python +import ayeaye + +class PoisonousAnimals(ayeaye.Model): + poisonous_animals = ayeaye.Connect(engine_url='json://data/poisonous_animals.json') +``` + +When instantiated, `self.poisonous_animals` will be a *dataset* that ETL operations can be done with. + +The `engine_url` parameter passed to `ayeaye.Connect` is specifying the dataset type JSON in this case) and exact location for the data (`data/poisonous_animals.json` is a relative file path). + +Instead of `engine_url` you could also specify a `ref` and this uses the data catalogue to lookup the `engine_url`. (TODO this feature is coming soon!). When used this way, `ayeaye.Connect` is responsible for resolving the `ref` to an `engine_url` and passing this to a subclass of `ayeaye.connectors.base.DataConnector` which can read and maybe write this data type. + + +## Unit tests + +Ensure the working directory is the base Aye Aye directory (i.e. the same directory as the Pipfile): +```shell +pipenv install --dev +export PYTHONPATH=`pwd`/lib +pipenv run python -m unittest discover +``` + +## Development version + +To use the latest code in editable mode- + +```shell +pipenv install -e git+https://github.com/Aye-Aye-Dev/AyeAye#egg=ayeaye +``` + +When `venv` is being used, add this line to `requirements.txt`- + +``` +git+https://github.com/Aye-Aye-Dev/AyeAye#egg=ayeaye +``` + +## Optional extras + +Extra dependencies for API usage within Aye-aye models can be installed like this: + +```shell +pipenv install "ayeaye[api]" +``` + +| Label | Functionality | +| --- | --- | +| api | Restful JSON via http(s) | +| aws | File based connectors can use Amazon Web Service S3 file storage | +| compression | On the fly compression for file based connectors | + + +## License + +Aye Aye is distributed under the terms of the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0.html) and Copyright Progressive Logic Limit 2021 and onwards. + + +%package help +Summary: Development documents and examples for ayeaye +Provides: python3-ayeaye-doc +%description help +# Aye Aye + +An ETL (Extract, Transform, Load) framework. + +## Quick install + +In the virtual environment for the project you’d like to use Aye Aye in, run:- + +```shell +pip install ayeaye +``` + +## Quick start + +Use [Pipenv](https://pipenv.pypa.io/en/latest/) to manage a python virtual environment and package management0 + +```shell +pipenv shell +pipenv install ayeaye +``` + +Within the environment created by pipenv above, run one of the examples:- + +```shell +curl "https://raw.githubusercontent.com/Aye-Aye-Dev/AyeAye/master/examples/poisonous_animals.py" \ + --output poisonous_animals.py +mkdir data +curl https://raw.githubusercontent.com/Aye-Aye-Dev/AyeAye/master/examples/data/poisonous_animals.json \ + --output data/poisonous_animals.json +python poisonous_animals.py +``` + +This model takes a small input dataset of animals and collates them by the country they are found. It doesn't write to a dataset, it just outputs a log. The log for this example contains the name of the country and the animals found there. + +There are more examples in the [Aye-Aye-Recipes](https://github.com/Aye-Aye-Dev/Aye-Aye-Recipes) git repo. + + +## Overview + +An Aye Aye ETL *model* inherits from `ayeaye.model` and uses class level variables to declare *connectors* to the data it acts on. + +Example:- + +```python +import ayeaye + +class PoisonousAnimals(ayeaye.Model): + poisonous_animals = ayeaye.Connect(engine_url='json://data/poisonous_animals.json') +``` + +When instantiated, `self.poisonous_animals` will be a *dataset* that ETL operations can be done with. + +The `engine_url` parameter passed to `ayeaye.Connect` is specifying the dataset type JSON in this case) and exact location for the data (`data/poisonous_animals.json` is a relative file path). + +Instead of `engine_url` you could also specify a `ref` and this uses the data catalogue to lookup the `engine_url`. (TODO this feature is coming soon!). When used this way, `ayeaye.Connect` is responsible for resolving the `ref` to an `engine_url` and passing this to a subclass of `ayeaye.connectors.base.DataConnector` which can read and maybe write this data type. + + +## Unit tests + +Ensure the working directory is the base Aye Aye directory (i.e. the same directory as the Pipfile): +```shell +pipenv install --dev +export PYTHONPATH=`pwd`/lib +pipenv run python -m unittest discover +``` + +## Development version + +To use the latest code in editable mode- + +```shell +pipenv install -e git+https://github.com/Aye-Aye-Dev/AyeAye#egg=ayeaye +``` + +When `venv` is being used, add this line to `requirements.txt`- + +``` +git+https://github.com/Aye-Aye-Dev/AyeAye#egg=ayeaye +``` + +## Optional extras + +Extra dependencies for API usage within Aye-aye models can be installed like this: + +```shell +pipenv install "ayeaye[api]" +``` + +| Label | Functionality | +| --- | --- | +| api | Restful JSON via http(s) | +| aws | File based connectors can use Amazon Web Service S3 file storage | +| compression | On the fly compression for file based connectors | + + +## License + +Aye Aye is distributed under the terms of the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0.html) and Copyright Progressive Logic Limit 2021 and onwards. + + +%prep +%autosetup -n ayeaye-0.0.45 + +%build +%py3_build + +%install +%py3_install +install -d -m755 %{buildroot}/%{_pkgdocdir} +if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi +if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi +if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi +if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi +pushd %{buildroot} +if [ -d usr/lib ]; then + find usr/lib -type f -printf "\"/%h/%f\"\n" >> filelist.lst +fi +if [ -d usr/lib64 ]; then + find usr/lib64 -type f -printf "\"/%h/%f\"\n" >> filelist.lst +fi +if [ -d usr/bin ]; then + find usr/bin -type f -printf "\"/%h/%f\"\n" >> filelist.lst +fi +if [ -d usr/sbin ]; then + find usr/sbin -type f -printf "\"/%h/%f\"\n" >> filelist.lst +fi +touch doclist.lst +if [ -d usr/share/man ]; then + find usr/share/man -type f -printf "\"/%h/%f.gz\"\n" >> doclist.lst +fi +popd +mv %{buildroot}/filelist.lst . +mv %{buildroot}/doclist.lst . + +%files -n python3-ayeaye -f filelist.lst +%dir %{python3_sitelib}/* + +%files help -f doclist.lst +%{_docdir}/* + +%changelog +* Tue Jun 20 2023 Python_Bot <Python_Bot@openeuler.org> - 0.0.45-1 +- Package Spec generated @@ -0,0 +1 @@ +eee16e53fa54d4e4d278992c7a28bfd2 ayeaye-0.0.45.tar.gz |