diff options
| -rw-r--r-- | .gitignore | 1 | ||||
| -rw-r--r-- | python-pygyver.spec | 335 | ||||
| -rw-r--r-- | sources | 1 |
3 files changed, 337 insertions, 0 deletions
@@ -0,0 +1 @@ +/pygyver-0.1.1.42.tar.gz diff --git a/python-pygyver.spec b/python-pygyver.spec new file mode 100644 index 0000000..55e8568 --- /dev/null +++ b/python-pygyver.spec @@ -0,0 +1,335 @@ +%global _empty_manifest_terminate_build 0 +Name: python-pygyver +Version: 0.1.1.42 +Release: 1 +Summary: Data engineering & Data science Pipeline Framework +License: MIT License +URL: https://github.com/madedotcom/pygyver +Source0: https://mirrors.nju.edu.cn/pypi/web/packages/e6/c0/cb05b7eef1faeda980138bf515390099bff1ac59fbc9c8e1a819da00d2f1/pygyver-0.1.1.42.tar.gz +BuildArch: noarch + +Requires: python3-boto3 +Requires: python3-codecov +Requires: python3-facebook-business +Requires: python3-google-cloud-bigquery +Requires: python3-google-cloud-bigquery-datatransfer +Requires: python3-google-cloud-storage +Requires: python3-gspread +Requires: python3-gspread-dataframe +Requires: python3-moto +Requires: python3-nltk +Requires: python3-numpy +Requires: python3-pandas +Requires: python3-pandas-gbq +Requires: python3-pg8000 +Requires: python3-pyarrow +Requires: python3-pymysql +Requires: python3-pytest +Requires: python3-PyYAML +Requires: python3-slack-sdk +Requires: python3-sqlalchemy +Requires: python3-tox +Requires: python3-versioneer +Requires: python3-wheel + +%description +# PyGyver + +> PyGyver is a user-friendly python package for data integration and manipulation. + +> Named after MacGyver, title character in the TV series MacGyver, and Python, the main language used in the repository. + +## Installation + +### PyPi + +PyGyver is available on [PyPi](https://pypi.org/project/pygyver/). + +```python +pip install pygyver +``` + +### Setup + +Most APIs requires access token files to authentificate and perform tasks such as creating or deleting objects. Those files need to be generated prior to using `pygyver` and stored in the environment you are executing your code against. The package make use of environment variables, and some of the below might need be supplied in your environment: + +``` +# Access token path +GOOGLE_APPLICATION_CREDENTIALS=path_to_google_access_token.json +FACEBOOK_APPLICATION_CREDENTIALS=path_to_facebook_access_token.json + +# Default values +BIGQUERY_PROJECT=your-gcs-project +GCS_PROJECT=your-gcs-project +GCS_BUCKET=your-gcs-bucket + +# Optional +PROJECT_ROOT=path_to_where_your_code_lives +``` + +## Modules + +PyGyver is structured around several modules available in the `etl` folder. Here is a summary table of those modules: + +| Module name | Descrition | Documentation | +| ------------- |-------------|-------------| +| `dw` | Perform task against the Google Cloud BigQuery API | [dw.md](docs/dw.md) | +| `facebook` | Perform task against the Facebook Marketing API | [facebook.md](docs/facebook.md) | +| `gooddata` | Perform task against the GoodData API | - | +| `gs` | Perform task against the Google Sheet API | - | +| `lib` | Store utilities used by other modules | - | +| `pipeline` | Utility to build data pipelines via YAML definition | [pipeline.md](docs/pipeline.md) | +| `prep` | Data transformation - ML pipelines | - | +| `storage` | Perform task against the AWS S3 and Google Cloud Storage API | [storage.md](docs/storage.md) | +| `toolkit` | Sets of tools for data manipulation | - | + +In order to load `BigQueryExecutor` from the `dw` module, you can run: + +``` +from pygyver.etl.dw import BigQueryExecutor +``` + +## Contributing + +> To get started... + +### Step 1 + +- 👯 Clone this repo to your local machine using `git@github.com:madedotcom/pygyver.git` + +### Step 2 + +- **HACK AWAY!** 🔨🔨🔨 + +The team follows TDD to develop new features on `pygyver`. +Tests can be found in `pygyver/tests`. + +### Step 3 + +- 🔃 Create a new pull request and request review from team members. Where applicable, a test should be added with the code change. + +## FAQ + +- **How to release a new version to PyPi?** + 1. Merge your changes to `master` branch + 2. Create a new release using `https://github.com/madedotcom/pygyver/releases` + + +%package -n python3-pygyver +Summary: Data engineering & Data science Pipeline Framework +Provides: python-pygyver +BuildRequires: python3-devel +BuildRequires: python3-setuptools +BuildRequires: python3-pip +%description -n python3-pygyver +# PyGyver + +> PyGyver is a user-friendly python package for data integration and manipulation. + +> Named after MacGyver, title character in the TV series MacGyver, and Python, the main language used in the repository. + +## Installation + +### PyPi + +PyGyver is available on [PyPi](https://pypi.org/project/pygyver/). + +```python +pip install pygyver +``` + +### Setup + +Most APIs requires access token files to authentificate and perform tasks such as creating or deleting objects. Those files need to be generated prior to using `pygyver` and stored in the environment you are executing your code against. The package make use of environment variables, and some of the below might need be supplied in your environment: + +``` +# Access token path +GOOGLE_APPLICATION_CREDENTIALS=path_to_google_access_token.json +FACEBOOK_APPLICATION_CREDENTIALS=path_to_facebook_access_token.json + +# Default values +BIGQUERY_PROJECT=your-gcs-project +GCS_PROJECT=your-gcs-project +GCS_BUCKET=your-gcs-bucket + +# Optional +PROJECT_ROOT=path_to_where_your_code_lives +``` + +## Modules + +PyGyver is structured around several modules available in the `etl` folder. Here is a summary table of those modules: + +| Module name | Descrition | Documentation | +| ------------- |-------------|-------------| +| `dw` | Perform task against the Google Cloud BigQuery API | [dw.md](docs/dw.md) | +| `facebook` | Perform task against the Facebook Marketing API | [facebook.md](docs/facebook.md) | +| `gooddata` | Perform task against the GoodData API | - | +| `gs` | Perform task against the Google Sheet API | - | +| `lib` | Store utilities used by other modules | - | +| `pipeline` | Utility to build data pipelines via YAML definition | [pipeline.md](docs/pipeline.md) | +| `prep` | Data transformation - ML pipelines | - | +| `storage` | Perform task against the AWS S3 and Google Cloud Storage API | [storage.md](docs/storage.md) | +| `toolkit` | Sets of tools for data manipulation | - | + +In order to load `BigQueryExecutor` from the `dw` module, you can run: + +``` +from pygyver.etl.dw import BigQueryExecutor +``` + +## Contributing + +> To get started... + +### Step 1 + +- 👯 Clone this repo to your local machine using `git@github.com:madedotcom/pygyver.git` + +### Step 2 + +- **HACK AWAY!** 🔨🔨🔨 + +The team follows TDD to develop new features on `pygyver`. +Tests can be found in `pygyver/tests`. + +### Step 3 + +- 🔃 Create a new pull request and request review from team members. Where applicable, a test should be added with the code change. + +## FAQ + +- **How to release a new version to PyPi?** + 1. Merge your changes to `master` branch + 2. Create a new release using `https://github.com/madedotcom/pygyver/releases` + + +%package help +Summary: Development documents and examples for pygyver +Provides: python3-pygyver-doc +%description help +# PyGyver + +> PyGyver is a user-friendly python package for data integration and manipulation. + +> Named after MacGyver, title character in the TV series MacGyver, and Python, the main language used in the repository. + +## Installation + +### PyPi + +PyGyver is available on [PyPi](https://pypi.org/project/pygyver/). + +```python +pip install pygyver +``` + +### Setup + +Most APIs requires access token files to authentificate and perform tasks such as creating or deleting objects. Those files need to be generated prior to using `pygyver` and stored in the environment you are executing your code against. The package make use of environment variables, and some of the below might need be supplied in your environment: + +``` +# Access token path +GOOGLE_APPLICATION_CREDENTIALS=path_to_google_access_token.json +FACEBOOK_APPLICATION_CREDENTIALS=path_to_facebook_access_token.json + +# Default values +BIGQUERY_PROJECT=your-gcs-project +GCS_PROJECT=your-gcs-project +GCS_BUCKET=your-gcs-bucket + +# Optional +PROJECT_ROOT=path_to_where_your_code_lives +``` + +## Modules + +PyGyver is structured around several modules available in the `etl` folder. Here is a summary table of those modules: + +| Module name | Descrition | Documentation | +| ------------- |-------------|-------------| +| `dw` | Perform task against the Google Cloud BigQuery API | [dw.md](docs/dw.md) | +| `facebook` | Perform task against the Facebook Marketing API | [facebook.md](docs/facebook.md) | +| `gooddata` | Perform task against the GoodData API | - | +| `gs` | Perform task against the Google Sheet API | - | +| `lib` | Store utilities used by other modules | - | +| `pipeline` | Utility to build data pipelines via YAML definition | [pipeline.md](docs/pipeline.md) | +| `prep` | Data transformation - ML pipelines | - | +| `storage` | Perform task against the AWS S3 and Google Cloud Storage API | [storage.md](docs/storage.md) | +| `toolkit` | Sets of tools for data manipulation | - | + +In order to load `BigQueryExecutor` from the `dw` module, you can run: + +``` +from pygyver.etl.dw import BigQueryExecutor +``` + +## Contributing + +> To get started... + +### Step 1 + +- 👯 Clone this repo to your local machine using `git@github.com:madedotcom/pygyver.git` + +### Step 2 + +- **HACK AWAY!** 🔨🔨🔨 + +The team follows TDD to develop new features on `pygyver`. +Tests can be found in `pygyver/tests`. + +### Step 3 + +- 🔃 Create a new pull request and request review from team members. Where applicable, a test should be added with the code change. + +## FAQ + +- **How to release a new version to PyPi?** + 1. Merge your changes to `master` branch + 2. Create a new release using `https://github.com/madedotcom/pygyver/releases` + + +%prep +%autosetup -n pygyver-0.1.1.42 + +%build +%py3_build + +%install +%py3_install +install -d -m755 %{buildroot}/%{_pkgdocdir} +if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi +if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi +if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi +if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi +pushd %{buildroot} +if [ -d usr/lib ]; then + find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/lib64 ]; then + find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/bin ]; then + find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/sbin ]; then + find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst +fi +touch doclist.lst +if [ -d usr/share/man ]; then + find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst +fi +popd +mv %{buildroot}/filelist.lst . +mv %{buildroot}/doclist.lst . + +%files -n python3-pygyver -f filelist.lst +%dir %{python3_sitelib}/* + +%files help -f doclist.lst +%{_docdir}/* + +%changelog +* Mon May 29 2023 Python_Bot <Python_Bot@openeuler.org> - 0.1.1.42-1 +- Package Spec generated @@ -0,0 +1 @@ +69fb21ff05c89925e59aadd2dea000d1 pygyver-0.1.1.42.tar.gz |