%global _empty_manifest_terminate_build 0 Name: python-ayeaye Version: 0.0.45 Release: 1 Summary: An ETL (Extract, Transform, Load) framework License: Apache Software License URL: https://github.com/Aye-Aye-Dev/AyeAye Source0: https://mirrors.aliyun.com/pypi/web/packages/12/d4/36db2468a349a551797f7eae9d1dc2727ab8d0074aa4e70a7486f2f25b12/ayeaye-0.0.45.tar.gz BuildArch: noarch Requires: python3-requests Requires: python3-boto3 Requires: python3-smart-open Requires: python3-smart-open %description # Aye Aye An ETL (Extract, Transform, Load) framework. ## Quick install In the virtual environment for the project you’d like to use Aye Aye in, run:- ```shell pip install ayeaye ``` ## Quick start Use [Pipenv](https://pipenv.pypa.io/en/latest/) to manage a python virtual environment and package management0 ```shell pipenv shell pipenv install ayeaye ``` Within the environment created by pipenv above, run one of the examples:- ```shell curl "https://raw.githubusercontent.com/Aye-Aye-Dev/AyeAye/master/examples/poisonous_animals.py" \ --output poisonous_animals.py mkdir data curl https://raw.githubusercontent.com/Aye-Aye-Dev/AyeAye/master/examples/data/poisonous_animals.json \ --output data/poisonous_animals.json python poisonous_animals.py ``` This model takes a small input dataset of animals and collates them by the country they are found. It doesn't write to a dataset, it just outputs a log. The log for this example contains the name of the country and the animals found there. There are more examples in the [Aye-Aye-Recipes](https://github.com/Aye-Aye-Dev/Aye-Aye-Recipes) git repo. ## Overview An Aye Aye ETL *model* inherits from `ayeaye.model` and uses class level variables to declare *connectors* to the data it acts on. Example:- ```python import ayeaye class PoisonousAnimals(ayeaye.Model): poisonous_animals = ayeaye.Connect(engine_url='json://data/poisonous_animals.json') ``` When instantiated, `self.poisonous_animals` will be a *dataset* that ETL operations can be done with. The `engine_url` parameter passed to `ayeaye.Connect` is specifying the dataset type JSON in this case) and exact location for the data (`data/poisonous_animals.json` is a relative file path). Instead of `engine_url` you could also specify a `ref` and this uses the data catalogue to lookup the `engine_url`. (TODO this feature is coming soon!). When used this way, `ayeaye.Connect` is responsible for resolving the `ref` to an `engine_url` and passing this to a subclass of `ayeaye.connectors.base.DataConnector` which can read and maybe write this data type. ## Unit tests Ensure the working directory is the base Aye Aye directory (i.e. the same directory as the Pipfile): ```shell pipenv install --dev export PYTHONPATH=`pwd`/lib pipenv run python -m unittest discover ``` ## Development version To use the latest code in editable mode- ```shell pipenv install -e git+https://github.com/Aye-Aye-Dev/AyeAye#egg=ayeaye ``` When `venv` is being used, add this line to `requirements.txt`- ``` git+https://github.com/Aye-Aye-Dev/AyeAye#egg=ayeaye ``` ## Optional extras Extra dependencies for API usage within Aye-aye models can be installed like this: ```shell pipenv install "ayeaye[api]" ``` | Label | Functionality | | --- | --- | | api | Restful JSON via http(s) | | aws | File based connectors can use Amazon Web Service S3 file storage | | compression | On the fly compression for file based connectors | ## License Aye Aye is distributed under the terms of the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0.html) and Copyright Progressive Logic Limit 2021 and onwards. %package -n python3-ayeaye Summary: An ETL (Extract, Transform, Load) framework Provides: python-ayeaye BuildRequires: python3-devel BuildRequires: python3-setuptools BuildRequires: python3-pip %description -n python3-ayeaye # Aye Aye An ETL (Extract, Transform, Load) framework. ## Quick install In the virtual environment for the project you’d like to use Aye Aye in, run:- ```shell pip install ayeaye ``` ## Quick start Use [Pipenv](https://pipenv.pypa.io/en/latest/) to manage a python virtual environment and package management0 ```shell pipenv shell pipenv install ayeaye ``` Within the environment created by pipenv above, run one of the examples:- ```shell curl "https://raw.githubusercontent.com/Aye-Aye-Dev/AyeAye/master/examples/poisonous_animals.py" \ --output poisonous_animals.py mkdir data curl https://raw.githubusercontent.com/Aye-Aye-Dev/AyeAye/master/examples/data/poisonous_animals.json \ --output data/poisonous_animals.json python poisonous_animals.py ``` This model takes a small input dataset of animals and collates them by the country they are found. It doesn't write to a dataset, it just outputs a log. The log for this example contains the name of the country and the animals found there. There are more examples in the [Aye-Aye-Recipes](https://github.com/Aye-Aye-Dev/Aye-Aye-Recipes) git repo. ## Overview An Aye Aye ETL *model* inherits from `ayeaye.model` and uses class level variables to declare *connectors* to the data it acts on. Example:- ```python import ayeaye class PoisonousAnimals(ayeaye.Model): poisonous_animals = ayeaye.Connect(engine_url='json://data/poisonous_animals.json') ``` When instantiated, `self.poisonous_animals` will be a *dataset* that ETL operations can be done with. The `engine_url` parameter passed to `ayeaye.Connect` is specifying the dataset type JSON in this case) and exact location for the data (`data/poisonous_animals.json` is a relative file path). Instead of `engine_url` you could also specify a `ref` and this uses the data catalogue to lookup the `engine_url`. (TODO this feature is coming soon!). When used this way, `ayeaye.Connect` is responsible for resolving the `ref` to an `engine_url` and passing this to a subclass of `ayeaye.connectors.base.DataConnector` which can read and maybe write this data type. ## Unit tests Ensure the working directory is the base Aye Aye directory (i.e. the same directory as the Pipfile): ```shell pipenv install --dev export PYTHONPATH=`pwd`/lib pipenv run python -m unittest discover ``` ## Development version To use the latest code in editable mode- ```shell pipenv install -e git+https://github.com/Aye-Aye-Dev/AyeAye#egg=ayeaye ``` When `venv` is being used, add this line to `requirements.txt`- ``` git+https://github.com/Aye-Aye-Dev/AyeAye#egg=ayeaye ``` ## Optional extras Extra dependencies for API usage within Aye-aye models can be installed like this: ```shell pipenv install "ayeaye[api]" ``` | Label | Functionality | | --- | --- | | api | Restful JSON via http(s) | | aws | File based connectors can use Amazon Web Service S3 file storage | | compression | On the fly compression for file based connectors | ## License Aye Aye is distributed under the terms of the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0.html) and Copyright Progressive Logic Limit 2021 and onwards. %package help Summary: Development documents and examples for ayeaye Provides: python3-ayeaye-doc %description help # Aye Aye An ETL (Extract, Transform, Load) framework. ## Quick install In the virtual environment for the project you’d like to use Aye Aye in, run:- ```shell pip install ayeaye ``` ## Quick start Use [Pipenv](https://pipenv.pypa.io/en/latest/) to manage a python virtual environment and package management0 ```shell pipenv shell pipenv install ayeaye ``` Within the environment created by pipenv above, run one of the examples:- ```shell curl "https://raw.githubusercontent.com/Aye-Aye-Dev/AyeAye/master/examples/poisonous_animals.py" \ --output poisonous_animals.py mkdir data curl https://raw.githubusercontent.com/Aye-Aye-Dev/AyeAye/master/examples/data/poisonous_animals.json \ --output data/poisonous_animals.json python poisonous_animals.py ``` This model takes a small input dataset of animals and collates them by the country they are found. It doesn't write to a dataset, it just outputs a log. The log for this example contains the name of the country and the animals found there. There are more examples in the [Aye-Aye-Recipes](https://github.com/Aye-Aye-Dev/Aye-Aye-Recipes) git repo. ## Overview An Aye Aye ETL *model* inherits from `ayeaye.model` and uses class level variables to declare *connectors* to the data it acts on. Example:- ```python import ayeaye class PoisonousAnimals(ayeaye.Model): poisonous_animals = ayeaye.Connect(engine_url='json://data/poisonous_animals.json') ``` When instantiated, `self.poisonous_animals` will be a *dataset* that ETL operations can be done with. The `engine_url` parameter passed to `ayeaye.Connect` is specifying the dataset type JSON in this case) and exact location for the data (`data/poisonous_animals.json` is a relative file path). Instead of `engine_url` you could also specify a `ref` and this uses the data catalogue to lookup the `engine_url`. (TODO this feature is coming soon!). When used this way, `ayeaye.Connect` is responsible for resolving the `ref` to an `engine_url` and passing this to a subclass of `ayeaye.connectors.base.DataConnector` which can read and maybe write this data type. ## Unit tests Ensure the working directory is the base Aye Aye directory (i.e. the same directory as the Pipfile): ```shell pipenv install --dev export PYTHONPATH=`pwd`/lib pipenv run python -m unittest discover ``` ## Development version To use the latest code in editable mode- ```shell pipenv install -e git+https://github.com/Aye-Aye-Dev/AyeAye#egg=ayeaye ``` When `venv` is being used, add this line to `requirements.txt`- ``` git+https://github.com/Aye-Aye-Dev/AyeAye#egg=ayeaye ``` ## Optional extras Extra dependencies for API usage within Aye-aye models can be installed like this: ```shell pipenv install "ayeaye[api]" ``` | Label | Functionality | | --- | --- | | api | Restful JSON via http(s) | | aws | File based connectors can use Amazon Web Service S3 file storage | | compression | On the fly compression for file based connectors | ## License Aye Aye is distributed under the terms of the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0.html) and Copyright Progressive Logic Limit 2021 and onwards. %prep %autosetup -n ayeaye-0.0.45 %build %py3_build %install %py3_install install -d -m755 %{buildroot}/%{_pkgdocdir} if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi pushd %{buildroot} if [ -d usr/lib ]; then find usr/lib -type f -printf "\"/%h/%f\"\n" >> filelist.lst fi if [ -d usr/lib64 ]; then find usr/lib64 -type f -printf "\"/%h/%f\"\n" >> filelist.lst fi if [ -d usr/bin ]; then find usr/bin -type f -printf "\"/%h/%f\"\n" >> filelist.lst fi if [ -d usr/sbin ]; then find usr/sbin -type f -printf "\"/%h/%f\"\n" >> filelist.lst fi touch doclist.lst if [ -d usr/share/man ]; then find usr/share/man -type f -printf "\"/%h/%f.gz\"\n" >> doclist.lst fi popd mv %{buildroot}/filelist.lst . mv %{buildroot}/doclist.lst . %files -n python3-ayeaye -f filelist.lst %dir %{python3_sitelib}/* %files help -f doclist.lst %{_docdir}/* %changelog * Tue Jun 20 2023 Python_Bot - 0.0.45-1 - Package Spec generated