summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorCoprDistGit <infra@openeuler.org>2023-04-10 08:46:46 +0000
committerCoprDistGit <infra@openeuler.org>2023-04-10 08:46:46 +0000
commit1b3fcfd570bf3a2d1104839034a4e0a1084c3a70 (patch)
treeda82084bba5e56c2bfba14ee9497eb715a225ddf
parentaa88d0c97d514f6ea2907249760f33b7caf53e01 (diff)
automatic import of python-awswrangler
-rw-r--r--.gitignore1
-rw-r--r--python-awswrangler.spec815
-rw-r--r--sources1
3 files changed, 817 insertions, 0 deletions
diff --git a/.gitignore b/.gitignore
index e69de29..9412773 100644
--- a/.gitignore
+++ b/.gitignore
@@ -0,0 +1 @@
+/awswrangler-2.20.1.tar.gz
diff --git a/python-awswrangler.spec b/python-awswrangler.spec
new file mode 100644
index 0000000..ea30698
--- /dev/null
+++ b/python-awswrangler.spec
@@ -0,0 +1,815 @@
+%global _empty_manifest_terminate_build 0
+Name: python-awswrangler
+Version: 2.20.1
+Release: 1
+Summary: Pandas on AWS.
+License: Apache-2.0
+URL: https://aws-sdk-pandas.readthedocs.io/
+Source0: https://mirrors.nju.edu.cn/pypi/web/packages/8b/72/f76290dffdf1a991277702d584dbfd913c1ddb0c4787b1d9e76890da8125/awswrangler-2.20.1.tar.gz
+BuildArch: noarch
+
+Requires: python3-boto3
+Requires: python3-botocore
+Requires: python3-pandas
+Requires: python3-numpy
+Requires: python3-numpy
+Requires: python3-pyarrow
+Requires: python3-redshift-connector
+Requires: python3-pymysql
+Requires: python3-pg8000
+Requires: python3-openpyxl
+Requires: python3-requests-aws4auth
+Requires: python3-jsonpath-ng
+Requires: python3-progressbar2
+Requires: python3-opensearch-py
+Requires: python3-gremlinpython
+Requires: python3-backoff
+Requires: python3-SPARQLWrapper
+Requires: python3-pyodbc
+Requires: python3-oracledb
+Requires: python3-deltalake
+
+%description
+# AWS SDK for pandas (awswrangler)
+
+AWS Data Wrangler is now **AWS SDK for pandas (awswrangler)**. We’re changing the name we use when we talk about the library, but everything else will stay the same. You’ll still be able to install using `pip install awswrangler` and you won’t need to change any of your code. As part of this change, we’ve moved the library from AWS Labs to the main AWS GitHub organisation but, thanks to the GitHub’s redirect feature, you’ll still be able to access the project by its old URLs until you update your bookmarks. Our documentation has also moved to [aws-sdk-pandas.readthedocs.io](https://aws-sdk-pandas.readthedocs.io), but old bookmarks will redirect to the new site.
+
+*Pandas on AWS*
+
+Easy integration with Athena, Glue, Redshift, Timestream, OpenSearch, Neptune, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
+
+![AWS SDK for pandas](docs/source/_static/logo2.png?raw=true "AWS SDK for pandas")
+![tracker](https://d3tiqpr4kkkomd.cloudfront.net/img/pixel.png?asset=GVOYN2BOOQ573LTVIHEW)
+
+> An [AWS Professional Service](https://aws.amazon.com/professional-services/) open source initiative | aws-proserve-opensource@amazon.com
+
+[![Release](https://img.shields.io/badge/release-2.20.1-brightgreen.svg)](https://pypi.org/project/awswrangler/)
+[![Python Version](https://img.shields.io/badge/python-3.7%20%7C%203.8%20%7C%203.9%20%7C%203.10-brightgreen.svg)](https://anaconda.org/conda-forge/awswrangler)
+[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
+[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
+
+[![Checked with mypy](http://www.mypy-lang.org/static/mypy_badge.svg)](http://mypy-lang.org/)
+[![Coverage](https://img.shields.io/badge/coverage-91%25-brightgreen.svg)](https://pypi.org/project/awswrangler/)
+![Static Checking](https://github.com/aws/aws-sdk-pandas/workflows/Static%20Checking/badge.svg?branch=main)
+[![Documentation Status](https://readthedocs.org/projects/aws-sdk-pandas/badge/?version=latest)](https://aws-sdk-pandas.readthedocs.io/?badge=latest)
+
+| Source | Downloads | Installation Command |
+|--------|-----------|----------------------|
+| **[PyPi](https://pypi.org/project/awswrangler/)** | [![PyPI Downloads](https://pepy.tech/badge/awswrangler)](https://pypi.org/project/awswrangler/) | `pip install awswrangler` |
+| **[Conda](https://anaconda.org/conda-forge/awswrangler)** | [![Conda Downloads](https://img.shields.io/conda/dn/conda-forge/awswrangler.svg)](https://anaconda.org/conda-forge/awswrangler) | `conda install -c conda-forge awswrangler` |
+
+> ⚠️ **For platforms without PyArrow 3 support (e.g. [EMR](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#emr-cluster), [Glue PySpark Job](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#aws-glue-pyspark-jobs), MWAA):**<br>
+➡️ `pip install pyarrow==2 awswrangler`
+
+Powered By [<img src="https://arrow.apache.org/img/arrow.png" width="200">](https://arrow.apache.org/powered_by/)
+
+## Table of contents
+
+- [Quick Start](#quick-start)
+- [Read The Docs](#read-the-docs)
+- [Getting Help](#getting-help)
+- [Community Resources](#community-resources)
+- [Logging](#logging)
+- [Who uses AWS SDK for pandas?](#who-uses-aws-sdk-pandas)
+
+## Quick Start
+
+Installation command: `pip install awswrangler`
+
+> ⚠️ **For platforms without PyArrow 3 support (e.g. [EMR](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#emr-cluster), [Glue PySpark Job](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#aws-glue-pyspark-jobs), MWAA):**<br>
+➡️`pip install pyarrow==2 awswrangler`
+
+```py3
+import awswrangler as wr
+import pandas as pd
+from datetime import datetime
+
+df = pd.DataFrame({"id": [1, 2], "value": ["foo", "boo"]})
+
+# Storing data on Data Lake
+wr.s3.to_parquet(
+ df=df,
+ path="s3://bucket/dataset/",
+ dataset=True,
+ database="my_db",
+ table="my_table"
+)
+
+# Retrieving the data directly from Amazon S3
+df = wr.s3.read_parquet("s3://bucket/dataset/", dataset=True)
+
+# Retrieving the data from Amazon Athena
+df = wr.athena.read_sql_query("SELECT * FROM my_table", database="my_db")
+
+# Get a Redshift connection from Glue Catalog and retrieving data from Redshift Spectrum
+con = wr.redshift.connect("my-glue-connection")
+df = wr.redshift.read_sql_query("SELECT * FROM external_schema.my_table", con=con)
+con.close()
+
+# Amazon Timestream Write
+df = pd.DataFrame({
+ "time": [datetime.now(), datetime.now()],
+ "my_dimension": ["foo", "boo"],
+ "measure": [1.0, 1.1],
+})
+rejected_records = wr.timestream.write(df,
+ database="sampleDB",
+ table="sampleTable",
+ time_col="time",
+ measure_col="measure",
+ dimensions_cols=["my_dimension"],
+)
+
+# Amazon Timestream Query
+wr.timestream.query("""
+SELECT time, measure_value::double, my_dimension
+FROM "sampleDB"."sampleTable" ORDER BY time DESC LIMIT 3
+""")
+
+```
+
+## [Read The Docs](https://aws-sdk-pandas.readthedocs.io/)
+
+- [**What is AWS SDK for pandas?**](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/about.html)
+- [**Install**](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html)
+ - [PyPi (pip)](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#pypi-pip)
+ - [Conda](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#conda)
+ - [AWS Lambda Layer](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#aws-lambda-layer)
+ - [AWS Glue Python Shell Jobs](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#aws-glue-python-shell-jobs)
+ - [AWS Glue PySpark Jobs](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#aws-glue-pyspark-jobs)
+ - [Amazon SageMaker Notebook](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#amazon-sagemaker-notebook)
+ - [Amazon SageMaker Notebook Lifecycle](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#amazon-sagemaker-notebook-lifecycle)
+ - [EMR](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#emr)
+ - [From source](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#from-source)
+- [**Tutorials**](https://github.com/aws/aws-sdk-pandas/tree/main/tutorials)
+ - [001 - Introduction](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/001%20-%20Introduction.ipynb)
+ - [002 - Sessions](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/002%20-%20Sessions.ipynb)
+ - [003 - Amazon S3](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/003%20-%20Amazon%20S3.ipynb)
+ - [004 - Parquet Datasets](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/004%20-%20Parquet%20Datasets.ipynb)
+ - [005 - Glue Catalog](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/005%20-%20Glue%20Catalog.ipynb)
+ - [006 - Amazon Athena](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/006%20-%20Amazon%20Athena.ipynb)
+ - [007 - Databases (Redshift, MySQL, PostgreSQL, SQL Server and Oracle)](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/007%20-%20Redshift%2C%20MySQL%2C%20PostgreSQL%2C%20SQL%20Server%2C%20Oracle.ipynb)
+ - [008 - Redshift - Copy & Unload.ipynb](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/008%20-%20Redshift%20-%20Copy%20%26%20Unload.ipynb)
+ - [009 - Redshift - Append, Overwrite and Upsert](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/009%20-%20Redshift%20-%20Append%2C%20Overwrite%2C%20Upsert.ipynb)
+ - [010 - Parquet Crawler](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/010%20-%20Parquet%20Crawler.ipynb)
+ - [011 - CSV Datasets](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/011%20-%20CSV%20Datasets.ipynb)
+ - [012 - CSV Crawler](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/012%20-%20CSV%20Crawler.ipynb)
+ - [013 - Merging Datasets on S3](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/013%20-%20Merging%20Datasets%20on%20S3.ipynb)
+ - [014 - Schema Evolution](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/014%20-%20Schema%20Evolution.ipynb)
+ - [015 - EMR](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/015%20-%20EMR.ipynb)
+ - [016 - EMR & Docker](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/016%20-%20EMR%20%26%20Docker.ipynb)
+ - [017 - Partition Projection](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/017%20-%20Partition%20Projection.ipynb)
+ - [018 - QuickSight](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/018%20-%20QuickSight.ipynb)
+ - [019 - Athena Cache](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/019%20-%20Athena%20Cache.ipynb)
+ - [020 - Spark Table Interoperability](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/020%20-%20Spark%20Table%20Interoperability.ipynb)
+ - [021 - Global Configurations](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/021%20-%20Global%20Configurations.ipynb)
+ - [022 - Writing Partitions Concurrently](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/022%20-%20Writing%20Partitions%20Concurrently.ipynb)
+ - [023 - Flexible Partitions Filter](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/023%20-%20Flexible%20Partitions%20Filter.ipynb)
+ - [024 - Athena Query Metadata](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/024%20-%20Athena%20Query%20Metadata.ipynb)
+ - [025 - Redshift - Loading Parquet files with Spectrum](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/025%20-%20Redshift%20-%20Loading%20Parquet%20files%20with%20Spectrum.ipynb)
+ - [026 - Amazon Timestream](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/026%20-%20Amazon%20Timestream.ipynb)
+ - [027 - Amazon Timestream 2](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/027%20-%20Amazon%20Timestream%202.ipynb)
+ - [028 - Amazon DynamoDB](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/028%20-%20DynamoDB.ipynb)
+ - [029 - S3 Select](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/029%20-%20S3%20Select.ipynb)
+ - [030 - Data Api](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/030%20-%20Data%20Api.ipynb)
+ - [031 - OpenSearch](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/031%20-%20OpenSearch.ipynb)
+ - [032 - Lake Formation Governed Tables](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/032%20-%20Lake%20Formation%20Governed%20Tables.ipynb)
+ - [033 - Amazon Neptune](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/033%20-%20Amazon%20Neptune.ipynb)
+ - [034 - Glue Data Quality](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/034%20-Glue%20%20Data%20Quality.ipynb)
+- [**API Reference**](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html)
+ - [Amazon S3](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-s3)
+ - [AWS Glue Catalog](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#aws-glue-catalog)
+ - [Amazon Athena](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-athena)
+ - [AWS Lake Formation](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#aws-lake-formation)
+ - [Amazon Redshift](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-redshift)
+ - [PostgreSQL](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#postgresql)
+ - [MySQL](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#mysql)
+ - [SQL Server](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#sqlserver)
+ - [Oracle](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#oracle)
+ - [Data API Redshift](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#data-api-redshift)
+ - [Data API RDS](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#data-api-rds)
+ - [OpenSearch](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#opensearch)
+ - [Amazon Neptune](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-neptune)
+ - [DynamoDB](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#dynamodb)
+ - [Amazon Timestream](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-timestream)
+ - [Amazon EMR](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-emr)
+ - [Amazon CloudWatch Logs](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-cloudwatch-logs)
+ - [Amazon Chime](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-chime)
+ - [Amazon QuickSight](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-quicksight)
+ - [AWS STS](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#aws-sts)
+ - [AWS Secrets Manager](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#aws-secrets-manager)
+ - [Global Configurations](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#global-configurations)
+- [**License**](https://github.com/aws/aws-sdk-pandas/blob/main/LICENSE.txt)
+- [**Contributing**](https://github.com/aws/aws-sdk-pandas/blob/main/CONTRIBUTING.md)
+
+## Getting Help
+
+The best way to interact with our team is through GitHub. You can open an [issue](https://github.com/aws/aws-sdk-pandas/issues/new/choose) and choose from one of our templates for bug reports, feature requests...
+You may also find help on these community resources:
+* The #aws-sdk-pandas Slack [channel](https://join.slack.com/t/aws-sdk-pandas/shared_invite/zt-sxdx38sl-E0coRfAds8WdpxXD2Nzfrg)
+* Ask a question on [Stack Overflow](https://stackoverflow.com/questions/tagged/awswrangler)
+ and tag it with `awswrangler`
+
+## Community Resources
+
+Please [send a Pull Request](https://github.com/aws/aws-sdk-pandas/edit/main/README.md) with your resource reference and @githubhandle.
+
+- [Optimize Python ETL by extending Pandas with AWS SDK for pandas](https://aws.amazon.com/blogs/big-data/optimize-python-etl-by-extending-pandas-with-aws-data-wrangler/) [[@igorborgest](https://github.com/igorborgest)]
+- [Reading Parquet Files With AWS Lambda](https://aprakash.wordpress.com/2020/04/14/reading-parquet-files-with-aws-lambda/) [[@anand086](https://github.com/anand086)]
+- [Transform AWS CloudTrail data using AWS SDK for pandas](https://aprakash.wordpress.com/2020/09/17/transform-aws-cloudtrail-data-using-aws-data-wrangler/) [[@anand086](https://github.com/anand086)]
+- [Rename Glue Tables using AWS SDK for pandas](https://ananddatastories.com/rename-glue-tables-using-aws-sdk-pandas/) [[@anand086](https://github.com/anand086)]
+- [Getting started on AWS SDK for pandas and Athena](https://medium.com/@dheerajsharmainampudi/getting-started-on-aws-sdk-pandas-and-athena-7b446c834076) [[@dheerajsharma21](https://github.com/dheerajsharma21)]
+- [Simplifying Pandas integration with AWS data related services](https://medium.com/@bv_subhash/aws-sdk-pandas-simplifying-pandas-integration-with-aws-data-related-services-2b3325c12188) [[@bvsubhash](https://github.com/bvsubhash)]
+- [Build an ETL pipeline using AWS S3, Glue and Athena](https://www.linkedin.com/pulse/build-etl-pipeline-using-aws-s3-glue-athena-data-wrangler-tom-reid/) [[@taupirho](https://github.com/taupirho)]
+
+## Logging
+
+Enabling internal logging examples:
+
+```py3
+import logging
+logging.basicConfig(level=logging.INFO, format="[%(name)s][%(funcName)s] %(message)s")
+logging.getLogger("awswrangler").setLevel(logging.DEBUG)
+logging.getLogger("botocore.credentials").setLevel(logging.CRITICAL)
+```
+
+Into AWS lambda:
+
+```py3
+import logging
+logging.getLogger("awswrangler").setLevel(logging.DEBUG)
+```
+
+## Who uses AWS SDK for pandas?
+
+Knowing which companies are using this library is important to help prioritize the project internally.
+If you would like us to include your company’s name and/or logo in the README file to indicate that your company is using the AWS SDK for pandas, please raise a "Support Us" issue. If you would like us to display your company’s logo, please raise a linked pull request to provide an image file for the logo. Note that by raising a Support Us issue (and related pull request), you are granting AWS permission to use your company’s name (and logo) for the limited purpose described here and you are confirming that you have authority to grant such permission.
+
+- [Amazon](https://www.amazon.com/)
+- [AWS](https://aws.amazon.com/)
+- [Cepsa](https://cepsa.com) [[@alvaropc](https://github.com/alvaropc)]
+- [Cognitivo](https://www.cognitivo.ai/) [[@msantino](https://github.com/msantino)]
+- [Digio](https://www.digio.com.br/) [[@afonsomy](https://github.com/afonsomy)]
+- [DNX](https://www.dnx.solutions/) [[@DNXLabs](https://github.com/DNXLabs)]
+- [Fortescue Future Industries](https://ffi.com.au/) [[@spencervoorend](https://github.com/spencervoorend)]
+- [Funcional Health Tech](https://www.funcionalcorp.com.br/) [[@webysther](https://github.com/webysther)]
+- [Funding Circle](https://www.fundingcircle.com/) [[@pfig](https://github.com/pfig)]
+- [Infomach](https://www.infomach.com.br/)
+- [Informa Markets](https://www.informamarkets.com/en/home.html) [[@mateusmorato]](http://github.com/mateusmorato)
+- [LINE TV](https://www.linetv.tw/) [[@bryanyang0528](https://github.com/bryanyang0528)]
+- [Magnataur](https://magnataur.com) [[@brianmingus2](https://github.com/brianmingus2)]
+- [M4U](https://www.m4u.com.br/) [[@Thiago-Dantas](https://github.com/Thiago-Dantas)]
+- [NBCUniversal](https://www.nbcuniversal.com/) [[@vibe](https://github.com/vibe)]
+- [nrd.io](https://nrd.io/) [[@mrtns](https://github.com/mrtns)]
+- [OKRA Technologies](https://okra.ai) [[@JPFrancoia](https://github.com/JPFrancoia), [@schot](https://github.com/schot)]
+- [Pier](https://www.pier.digital/) [[@flaviomax](https://github.com/flaviomax)]
+- [Pismo](https://www.pismo.io/) [[@msantino](https://github.com/msantino)]
+- [ringDNA](https://www.ringdna.com/) [[@msropp](https://github.com/msropp)]
+- [Serasa Experian](https://www.serasaexperian.com.br/) [[@andre-marcos-perez](https://github.com/andre-marcos-perez)]
+- [Shipwell](https://shipwell.com/) [[@zacharycarter](https://github.com/zacharycarter)]
+- [strongDM](https://www.strongdm.com/) [[@mrtns](https://github.com/mrtns)]
+- [Thinkbumblebee](https://www.thinkbumblebee.com/) [[@dheerajsharma21]](https://github.com/dheerajsharma21)
+- [VTEX](https://vtex.com/us-en/) [[@igorborgest]](https://github.com/igorborgest)
+- [Zillow](https://www.zillow.com/) [[@nicholas-miles]](https://github.com/nicholas-miles)
+
+
+%package -n python3-awswrangler
+Summary: Pandas on AWS.
+Provides: python-awswrangler
+BuildRequires: python3-devel
+BuildRequires: python3-setuptools
+BuildRequires: python3-pip
+%description -n python3-awswrangler
+# AWS SDK for pandas (awswrangler)
+
+AWS Data Wrangler is now **AWS SDK for pandas (awswrangler)**. We’re changing the name we use when we talk about the library, but everything else will stay the same. You’ll still be able to install using `pip install awswrangler` and you won’t need to change any of your code. As part of this change, we’ve moved the library from AWS Labs to the main AWS GitHub organisation but, thanks to the GitHub’s redirect feature, you’ll still be able to access the project by its old URLs until you update your bookmarks. Our documentation has also moved to [aws-sdk-pandas.readthedocs.io](https://aws-sdk-pandas.readthedocs.io), but old bookmarks will redirect to the new site.
+
+*Pandas on AWS*
+
+Easy integration with Athena, Glue, Redshift, Timestream, OpenSearch, Neptune, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
+
+![AWS SDK for pandas](docs/source/_static/logo2.png?raw=true "AWS SDK for pandas")
+![tracker](https://d3tiqpr4kkkomd.cloudfront.net/img/pixel.png?asset=GVOYN2BOOQ573LTVIHEW)
+
+> An [AWS Professional Service](https://aws.amazon.com/professional-services/) open source initiative | aws-proserve-opensource@amazon.com
+
+[![Release](https://img.shields.io/badge/release-2.20.1-brightgreen.svg)](https://pypi.org/project/awswrangler/)
+[![Python Version](https://img.shields.io/badge/python-3.7%20%7C%203.8%20%7C%203.9%20%7C%203.10-brightgreen.svg)](https://anaconda.org/conda-forge/awswrangler)
+[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
+[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
+
+[![Checked with mypy](http://www.mypy-lang.org/static/mypy_badge.svg)](http://mypy-lang.org/)
+[![Coverage](https://img.shields.io/badge/coverage-91%25-brightgreen.svg)](https://pypi.org/project/awswrangler/)
+![Static Checking](https://github.com/aws/aws-sdk-pandas/workflows/Static%20Checking/badge.svg?branch=main)
+[![Documentation Status](https://readthedocs.org/projects/aws-sdk-pandas/badge/?version=latest)](https://aws-sdk-pandas.readthedocs.io/?badge=latest)
+
+| Source | Downloads | Installation Command |
+|--------|-----------|----------------------|
+| **[PyPi](https://pypi.org/project/awswrangler/)** | [![PyPI Downloads](https://pepy.tech/badge/awswrangler)](https://pypi.org/project/awswrangler/) | `pip install awswrangler` |
+| **[Conda](https://anaconda.org/conda-forge/awswrangler)** | [![Conda Downloads](https://img.shields.io/conda/dn/conda-forge/awswrangler.svg)](https://anaconda.org/conda-forge/awswrangler) | `conda install -c conda-forge awswrangler` |
+
+> ⚠️ **For platforms without PyArrow 3 support (e.g. [EMR](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#emr-cluster), [Glue PySpark Job](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#aws-glue-pyspark-jobs), MWAA):**<br>
+➡️ `pip install pyarrow==2 awswrangler`
+
+Powered By [<img src="https://arrow.apache.org/img/arrow.png" width="200">](https://arrow.apache.org/powered_by/)
+
+## Table of contents
+
+- [Quick Start](#quick-start)
+- [Read The Docs](#read-the-docs)
+- [Getting Help](#getting-help)
+- [Community Resources](#community-resources)
+- [Logging](#logging)
+- [Who uses AWS SDK for pandas?](#who-uses-aws-sdk-pandas)
+
+## Quick Start
+
+Installation command: `pip install awswrangler`
+
+> ⚠️ **For platforms without PyArrow 3 support (e.g. [EMR](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#emr-cluster), [Glue PySpark Job](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#aws-glue-pyspark-jobs), MWAA):**<br>
+➡️`pip install pyarrow==2 awswrangler`
+
+```py3
+import awswrangler as wr
+import pandas as pd
+from datetime import datetime
+
+df = pd.DataFrame({"id": [1, 2], "value": ["foo", "boo"]})
+
+# Storing data on Data Lake
+wr.s3.to_parquet(
+ df=df,
+ path="s3://bucket/dataset/",
+ dataset=True,
+ database="my_db",
+ table="my_table"
+)
+
+# Retrieving the data directly from Amazon S3
+df = wr.s3.read_parquet("s3://bucket/dataset/", dataset=True)
+
+# Retrieving the data from Amazon Athena
+df = wr.athena.read_sql_query("SELECT * FROM my_table", database="my_db")
+
+# Get a Redshift connection from Glue Catalog and retrieving data from Redshift Spectrum
+con = wr.redshift.connect("my-glue-connection")
+df = wr.redshift.read_sql_query("SELECT * FROM external_schema.my_table", con=con)
+con.close()
+
+# Amazon Timestream Write
+df = pd.DataFrame({
+ "time": [datetime.now(), datetime.now()],
+ "my_dimension": ["foo", "boo"],
+ "measure": [1.0, 1.1],
+})
+rejected_records = wr.timestream.write(df,
+ database="sampleDB",
+ table="sampleTable",
+ time_col="time",
+ measure_col="measure",
+ dimensions_cols=["my_dimension"],
+)
+
+# Amazon Timestream Query
+wr.timestream.query("""
+SELECT time, measure_value::double, my_dimension
+FROM "sampleDB"."sampleTable" ORDER BY time DESC LIMIT 3
+""")
+
+```
+
+## [Read The Docs](https://aws-sdk-pandas.readthedocs.io/)
+
+- [**What is AWS SDK for pandas?**](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/about.html)
+- [**Install**](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html)
+ - [PyPi (pip)](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#pypi-pip)
+ - [Conda](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#conda)
+ - [AWS Lambda Layer](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#aws-lambda-layer)
+ - [AWS Glue Python Shell Jobs](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#aws-glue-python-shell-jobs)
+ - [AWS Glue PySpark Jobs](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#aws-glue-pyspark-jobs)
+ - [Amazon SageMaker Notebook](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#amazon-sagemaker-notebook)
+ - [Amazon SageMaker Notebook Lifecycle](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#amazon-sagemaker-notebook-lifecycle)
+ - [EMR](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#emr)
+ - [From source](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#from-source)
+- [**Tutorials**](https://github.com/aws/aws-sdk-pandas/tree/main/tutorials)
+ - [001 - Introduction](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/001%20-%20Introduction.ipynb)
+ - [002 - Sessions](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/002%20-%20Sessions.ipynb)
+ - [003 - Amazon S3](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/003%20-%20Amazon%20S3.ipynb)
+ - [004 - Parquet Datasets](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/004%20-%20Parquet%20Datasets.ipynb)
+ - [005 - Glue Catalog](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/005%20-%20Glue%20Catalog.ipynb)
+ - [006 - Amazon Athena](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/006%20-%20Amazon%20Athena.ipynb)
+ - [007 - Databases (Redshift, MySQL, PostgreSQL, SQL Server and Oracle)](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/007%20-%20Redshift%2C%20MySQL%2C%20PostgreSQL%2C%20SQL%20Server%2C%20Oracle.ipynb)
+ - [008 - Redshift - Copy & Unload.ipynb](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/008%20-%20Redshift%20-%20Copy%20%26%20Unload.ipynb)
+ - [009 - Redshift - Append, Overwrite and Upsert](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/009%20-%20Redshift%20-%20Append%2C%20Overwrite%2C%20Upsert.ipynb)
+ - [010 - Parquet Crawler](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/010%20-%20Parquet%20Crawler.ipynb)
+ - [011 - CSV Datasets](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/011%20-%20CSV%20Datasets.ipynb)
+ - [012 - CSV Crawler](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/012%20-%20CSV%20Crawler.ipynb)
+ - [013 - Merging Datasets on S3](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/013%20-%20Merging%20Datasets%20on%20S3.ipynb)
+ - [014 - Schema Evolution](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/014%20-%20Schema%20Evolution.ipynb)
+ - [015 - EMR](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/015%20-%20EMR.ipynb)
+ - [016 - EMR & Docker](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/016%20-%20EMR%20%26%20Docker.ipynb)
+ - [017 - Partition Projection](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/017%20-%20Partition%20Projection.ipynb)
+ - [018 - QuickSight](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/018%20-%20QuickSight.ipynb)
+ - [019 - Athena Cache](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/019%20-%20Athena%20Cache.ipynb)
+ - [020 - Spark Table Interoperability](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/020%20-%20Spark%20Table%20Interoperability.ipynb)
+ - [021 - Global Configurations](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/021%20-%20Global%20Configurations.ipynb)
+ - [022 - Writing Partitions Concurrently](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/022%20-%20Writing%20Partitions%20Concurrently.ipynb)
+ - [023 - Flexible Partitions Filter](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/023%20-%20Flexible%20Partitions%20Filter.ipynb)
+ - [024 - Athena Query Metadata](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/024%20-%20Athena%20Query%20Metadata.ipynb)
+ - [025 - Redshift - Loading Parquet files with Spectrum](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/025%20-%20Redshift%20-%20Loading%20Parquet%20files%20with%20Spectrum.ipynb)
+ - [026 - Amazon Timestream](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/026%20-%20Amazon%20Timestream.ipynb)
+ - [027 - Amazon Timestream 2](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/027%20-%20Amazon%20Timestream%202.ipynb)
+ - [028 - Amazon DynamoDB](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/028%20-%20DynamoDB.ipynb)
+ - [029 - S3 Select](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/029%20-%20S3%20Select.ipynb)
+ - [030 - Data Api](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/030%20-%20Data%20Api.ipynb)
+ - [031 - OpenSearch](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/031%20-%20OpenSearch.ipynb)
+ - [032 - Lake Formation Governed Tables](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/032%20-%20Lake%20Formation%20Governed%20Tables.ipynb)
+ - [033 - Amazon Neptune](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/033%20-%20Amazon%20Neptune.ipynb)
+ - [034 - Glue Data Quality](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/034%20-Glue%20%20Data%20Quality.ipynb)
+- [**API Reference**](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html)
+ - [Amazon S3](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-s3)
+ - [AWS Glue Catalog](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#aws-glue-catalog)
+ - [Amazon Athena](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-athena)
+ - [AWS Lake Formation](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#aws-lake-formation)
+ - [Amazon Redshift](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-redshift)
+ - [PostgreSQL](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#postgresql)
+ - [MySQL](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#mysql)
+ - [SQL Server](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#sqlserver)
+ - [Oracle](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#oracle)
+ - [Data API Redshift](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#data-api-redshift)
+ - [Data API RDS](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#data-api-rds)
+ - [OpenSearch](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#opensearch)
+ - [Amazon Neptune](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-neptune)
+ - [DynamoDB](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#dynamodb)
+ - [Amazon Timestream](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-timestream)
+ - [Amazon EMR](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-emr)
+ - [Amazon CloudWatch Logs](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-cloudwatch-logs)
+ - [Amazon Chime](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-chime)
+ - [Amazon QuickSight](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-quicksight)
+ - [AWS STS](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#aws-sts)
+ - [AWS Secrets Manager](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#aws-secrets-manager)
+ - [Global Configurations](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#global-configurations)
+- [**License**](https://github.com/aws/aws-sdk-pandas/blob/main/LICENSE.txt)
+- [**Contributing**](https://github.com/aws/aws-sdk-pandas/blob/main/CONTRIBUTING.md)
+
+## Getting Help
+
+The best way to interact with our team is through GitHub. You can open an [issue](https://github.com/aws/aws-sdk-pandas/issues/new/choose) and choose from one of our templates for bug reports, feature requests...
+You may also find help on these community resources:
+* The #aws-sdk-pandas Slack [channel](https://join.slack.com/t/aws-sdk-pandas/shared_invite/zt-sxdx38sl-E0coRfAds8WdpxXD2Nzfrg)
+* Ask a question on [Stack Overflow](https://stackoverflow.com/questions/tagged/awswrangler)
+ and tag it with `awswrangler`
+
+## Community Resources
+
+Please [send a Pull Request](https://github.com/aws/aws-sdk-pandas/edit/main/README.md) with your resource reference and @githubhandle.
+
+- [Optimize Python ETL by extending Pandas with AWS SDK for pandas](https://aws.amazon.com/blogs/big-data/optimize-python-etl-by-extending-pandas-with-aws-data-wrangler/) [[@igorborgest](https://github.com/igorborgest)]
+- [Reading Parquet Files With AWS Lambda](https://aprakash.wordpress.com/2020/04/14/reading-parquet-files-with-aws-lambda/) [[@anand086](https://github.com/anand086)]
+- [Transform AWS CloudTrail data using AWS SDK for pandas](https://aprakash.wordpress.com/2020/09/17/transform-aws-cloudtrail-data-using-aws-data-wrangler/) [[@anand086](https://github.com/anand086)]
+- [Rename Glue Tables using AWS SDK for pandas](https://ananddatastories.com/rename-glue-tables-using-aws-sdk-pandas/) [[@anand086](https://github.com/anand086)]
+- [Getting started on AWS SDK for pandas and Athena](https://medium.com/@dheerajsharmainampudi/getting-started-on-aws-sdk-pandas-and-athena-7b446c834076) [[@dheerajsharma21](https://github.com/dheerajsharma21)]
+- [Simplifying Pandas integration with AWS data related services](https://medium.com/@bv_subhash/aws-sdk-pandas-simplifying-pandas-integration-with-aws-data-related-services-2b3325c12188) [[@bvsubhash](https://github.com/bvsubhash)]
+- [Build an ETL pipeline using AWS S3, Glue and Athena](https://www.linkedin.com/pulse/build-etl-pipeline-using-aws-s3-glue-athena-data-wrangler-tom-reid/) [[@taupirho](https://github.com/taupirho)]
+
+## Logging
+
+Enabling internal logging examples:
+
+```py3
+import logging
+logging.basicConfig(level=logging.INFO, format="[%(name)s][%(funcName)s] %(message)s")
+logging.getLogger("awswrangler").setLevel(logging.DEBUG)
+logging.getLogger("botocore.credentials").setLevel(logging.CRITICAL)
+```
+
+Into AWS lambda:
+
+```py3
+import logging
+logging.getLogger("awswrangler").setLevel(logging.DEBUG)
+```
+
+## Who uses AWS SDK for pandas?
+
+Knowing which companies are using this library is important to help prioritize the project internally.
+If you would like us to include your company’s name and/or logo in the README file to indicate that your company is using the AWS SDK for pandas, please raise a "Support Us" issue. If you would like us to display your company’s logo, please raise a linked pull request to provide an image file for the logo. Note that by raising a Support Us issue (and related pull request), you are granting AWS permission to use your company’s name (and logo) for the limited purpose described here and you are confirming that you have authority to grant such permission.
+
+- [Amazon](https://www.amazon.com/)
+- [AWS](https://aws.amazon.com/)
+- [Cepsa](https://cepsa.com) [[@alvaropc](https://github.com/alvaropc)]
+- [Cognitivo](https://www.cognitivo.ai/) [[@msantino](https://github.com/msantino)]
+- [Digio](https://www.digio.com.br/) [[@afonsomy](https://github.com/afonsomy)]
+- [DNX](https://www.dnx.solutions/) [[@DNXLabs](https://github.com/DNXLabs)]
+- [Fortescue Future Industries](https://ffi.com.au/) [[@spencervoorend](https://github.com/spencervoorend)]
+- [Funcional Health Tech](https://www.funcionalcorp.com.br/) [[@webysther](https://github.com/webysther)]
+- [Funding Circle](https://www.fundingcircle.com/) [[@pfig](https://github.com/pfig)]
+- [Infomach](https://www.infomach.com.br/)
+- [Informa Markets](https://www.informamarkets.com/en/home.html) [[@mateusmorato]](http://github.com/mateusmorato)
+- [LINE TV](https://www.linetv.tw/) [[@bryanyang0528](https://github.com/bryanyang0528)]
+- [Magnataur](https://magnataur.com) [[@brianmingus2](https://github.com/brianmingus2)]
+- [M4U](https://www.m4u.com.br/) [[@Thiago-Dantas](https://github.com/Thiago-Dantas)]
+- [NBCUniversal](https://www.nbcuniversal.com/) [[@vibe](https://github.com/vibe)]
+- [nrd.io](https://nrd.io/) [[@mrtns](https://github.com/mrtns)]
+- [OKRA Technologies](https://okra.ai) [[@JPFrancoia](https://github.com/JPFrancoia), [@schot](https://github.com/schot)]
+- [Pier](https://www.pier.digital/) [[@flaviomax](https://github.com/flaviomax)]
+- [Pismo](https://www.pismo.io/) [[@msantino](https://github.com/msantino)]
+- [ringDNA](https://www.ringdna.com/) [[@msropp](https://github.com/msropp)]
+- [Serasa Experian](https://www.serasaexperian.com.br/) [[@andre-marcos-perez](https://github.com/andre-marcos-perez)]
+- [Shipwell](https://shipwell.com/) [[@zacharycarter](https://github.com/zacharycarter)]
+- [strongDM](https://www.strongdm.com/) [[@mrtns](https://github.com/mrtns)]
+- [Thinkbumblebee](https://www.thinkbumblebee.com/) [[@dheerajsharma21]](https://github.com/dheerajsharma21)
+- [VTEX](https://vtex.com/us-en/) [[@igorborgest]](https://github.com/igorborgest)
+- [Zillow](https://www.zillow.com/) [[@nicholas-miles]](https://github.com/nicholas-miles)
+
+
+%package help
+Summary: Development documents and examples for awswrangler
+Provides: python3-awswrangler-doc
+%description help
+# AWS SDK for pandas (awswrangler)
+
+AWS Data Wrangler is now **AWS SDK for pandas (awswrangler)**. We’re changing the name we use when we talk about the library, but everything else will stay the same. You’ll still be able to install using `pip install awswrangler` and you won’t need to change any of your code. As part of this change, we’ve moved the library from AWS Labs to the main AWS GitHub organisation but, thanks to the GitHub’s redirect feature, you’ll still be able to access the project by its old URLs until you update your bookmarks. Our documentation has also moved to [aws-sdk-pandas.readthedocs.io](https://aws-sdk-pandas.readthedocs.io), but old bookmarks will redirect to the new site.
+
+*Pandas on AWS*
+
+Easy integration with Athena, Glue, Redshift, Timestream, OpenSearch, Neptune, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
+
+![AWS SDK for pandas](docs/source/_static/logo2.png?raw=true "AWS SDK for pandas")
+![tracker](https://d3tiqpr4kkkomd.cloudfront.net/img/pixel.png?asset=GVOYN2BOOQ573LTVIHEW)
+
+> An [AWS Professional Service](https://aws.amazon.com/professional-services/) open source initiative | aws-proserve-opensource@amazon.com
+
+[![Release](https://img.shields.io/badge/release-2.20.1-brightgreen.svg)](https://pypi.org/project/awswrangler/)
+[![Python Version](https://img.shields.io/badge/python-3.7%20%7C%203.8%20%7C%203.9%20%7C%203.10-brightgreen.svg)](https://anaconda.org/conda-forge/awswrangler)
+[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
+[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
+
+[![Checked with mypy](http://www.mypy-lang.org/static/mypy_badge.svg)](http://mypy-lang.org/)
+[![Coverage](https://img.shields.io/badge/coverage-91%25-brightgreen.svg)](https://pypi.org/project/awswrangler/)
+![Static Checking](https://github.com/aws/aws-sdk-pandas/workflows/Static%20Checking/badge.svg?branch=main)
+[![Documentation Status](https://readthedocs.org/projects/aws-sdk-pandas/badge/?version=latest)](https://aws-sdk-pandas.readthedocs.io/?badge=latest)
+
+| Source | Downloads | Installation Command |
+|--------|-----------|----------------------|
+| **[PyPi](https://pypi.org/project/awswrangler/)** | [![PyPI Downloads](https://pepy.tech/badge/awswrangler)](https://pypi.org/project/awswrangler/) | `pip install awswrangler` |
+| **[Conda](https://anaconda.org/conda-forge/awswrangler)** | [![Conda Downloads](https://img.shields.io/conda/dn/conda-forge/awswrangler.svg)](https://anaconda.org/conda-forge/awswrangler) | `conda install -c conda-forge awswrangler` |
+
+> ⚠️ **For platforms without PyArrow 3 support (e.g. [EMR](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#emr-cluster), [Glue PySpark Job](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#aws-glue-pyspark-jobs), MWAA):**<br>
+➡️ `pip install pyarrow==2 awswrangler`
+
+Powered By [<img src="https://arrow.apache.org/img/arrow.png" width="200">](https://arrow.apache.org/powered_by/)
+
+## Table of contents
+
+- [Quick Start](#quick-start)
+- [Read The Docs](#read-the-docs)
+- [Getting Help](#getting-help)
+- [Community Resources](#community-resources)
+- [Logging](#logging)
+- [Who uses AWS SDK for pandas?](#who-uses-aws-sdk-pandas)
+
+## Quick Start
+
+Installation command: `pip install awswrangler`
+
+> ⚠️ **For platforms without PyArrow 3 support (e.g. [EMR](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#emr-cluster), [Glue PySpark Job](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#aws-glue-pyspark-jobs), MWAA):**<br>
+➡️`pip install pyarrow==2 awswrangler`
+
+```py3
+import awswrangler as wr
+import pandas as pd
+from datetime import datetime
+
+df = pd.DataFrame({"id": [1, 2], "value": ["foo", "boo"]})
+
+# Storing data on Data Lake
+wr.s3.to_parquet(
+ df=df,
+ path="s3://bucket/dataset/",
+ dataset=True,
+ database="my_db",
+ table="my_table"
+)
+
+# Retrieving the data directly from Amazon S3
+df = wr.s3.read_parquet("s3://bucket/dataset/", dataset=True)
+
+# Retrieving the data from Amazon Athena
+df = wr.athena.read_sql_query("SELECT * FROM my_table", database="my_db")
+
+# Get a Redshift connection from Glue Catalog and retrieving data from Redshift Spectrum
+con = wr.redshift.connect("my-glue-connection")
+df = wr.redshift.read_sql_query("SELECT * FROM external_schema.my_table", con=con)
+con.close()
+
+# Amazon Timestream Write
+df = pd.DataFrame({
+ "time": [datetime.now(), datetime.now()],
+ "my_dimension": ["foo", "boo"],
+ "measure": [1.0, 1.1],
+})
+rejected_records = wr.timestream.write(df,
+ database="sampleDB",
+ table="sampleTable",
+ time_col="time",
+ measure_col="measure",
+ dimensions_cols=["my_dimension"],
+)
+
+# Amazon Timestream Query
+wr.timestream.query("""
+SELECT time, measure_value::double, my_dimension
+FROM "sampleDB"."sampleTable" ORDER BY time DESC LIMIT 3
+""")
+
+```
+
+## [Read The Docs](https://aws-sdk-pandas.readthedocs.io/)
+
+- [**What is AWS SDK for pandas?**](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/about.html)
+- [**Install**](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html)
+ - [PyPi (pip)](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#pypi-pip)
+ - [Conda](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#conda)
+ - [AWS Lambda Layer](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#aws-lambda-layer)
+ - [AWS Glue Python Shell Jobs](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#aws-glue-python-shell-jobs)
+ - [AWS Glue PySpark Jobs](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#aws-glue-pyspark-jobs)
+ - [Amazon SageMaker Notebook](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#amazon-sagemaker-notebook)
+ - [Amazon SageMaker Notebook Lifecycle](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#amazon-sagemaker-notebook-lifecycle)
+ - [EMR](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#emr)
+ - [From source](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/install.html#from-source)
+- [**Tutorials**](https://github.com/aws/aws-sdk-pandas/tree/main/tutorials)
+ - [001 - Introduction](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/001%20-%20Introduction.ipynb)
+ - [002 - Sessions](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/002%20-%20Sessions.ipynb)
+ - [003 - Amazon S3](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/003%20-%20Amazon%20S3.ipynb)
+ - [004 - Parquet Datasets](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/004%20-%20Parquet%20Datasets.ipynb)
+ - [005 - Glue Catalog](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/005%20-%20Glue%20Catalog.ipynb)
+ - [006 - Amazon Athena](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/006%20-%20Amazon%20Athena.ipynb)
+ - [007 - Databases (Redshift, MySQL, PostgreSQL, SQL Server and Oracle)](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/007%20-%20Redshift%2C%20MySQL%2C%20PostgreSQL%2C%20SQL%20Server%2C%20Oracle.ipynb)
+ - [008 - Redshift - Copy & Unload.ipynb](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/008%20-%20Redshift%20-%20Copy%20%26%20Unload.ipynb)
+ - [009 - Redshift - Append, Overwrite and Upsert](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/009%20-%20Redshift%20-%20Append%2C%20Overwrite%2C%20Upsert.ipynb)
+ - [010 - Parquet Crawler](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/010%20-%20Parquet%20Crawler.ipynb)
+ - [011 - CSV Datasets](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/011%20-%20CSV%20Datasets.ipynb)
+ - [012 - CSV Crawler](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/012%20-%20CSV%20Crawler.ipynb)
+ - [013 - Merging Datasets on S3](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/013%20-%20Merging%20Datasets%20on%20S3.ipynb)
+ - [014 - Schema Evolution](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/014%20-%20Schema%20Evolution.ipynb)
+ - [015 - EMR](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/015%20-%20EMR.ipynb)
+ - [016 - EMR & Docker](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/016%20-%20EMR%20%26%20Docker.ipynb)
+ - [017 - Partition Projection](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/017%20-%20Partition%20Projection.ipynb)
+ - [018 - QuickSight](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/018%20-%20QuickSight.ipynb)
+ - [019 - Athena Cache](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/019%20-%20Athena%20Cache.ipynb)
+ - [020 - Spark Table Interoperability](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/020%20-%20Spark%20Table%20Interoperability.ipynb)
+ - [021 - Global Configurations](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/021%20-%20Global%20Configurations.ipynb)
+ - [022 - Writing Partitions Concurrently](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/022%20-%20Writing%20Partitions%20Concurrently.ipynb)
+ - [023 - Flexible Partitions Filter](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/023%20-%20Flexible%20Partitions%20Filter.ipynb)
+ - [024 - Athena Query Metadata](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/024%20-%20Athena%20Query%20Metadata.ipynb)
+ - [025 - Redshift - Loading Parquet files with Spectrum](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/025%20-%20Redshift%20-%20Loading%20Parquet%20files%20with%20Spectrum.ipynb)
+ - [026 - Amazon Timestream](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/026%20-%20Amazon%20Timestream.ipynb)
+ - [027 - Amazon Timestream 2](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/027%20-%20Amazon%20Timestream%202.ipynb)
+ - [028 - Amazon DynamoDB](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/028%20-%20DynamoDB.ipynb)
+ - [029 - S3 Select](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/029%20-%20S3%20Select.ipynb)
+ - [030 - Data Api](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/030%20-%20Data%20Api.ipynb)
+ - [031 - OpenSearch](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/031%20-%20OpenSearch.ipynb)
+ - [032 - Lake Formation Governed Tables](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/032%20-%20Lake%20Formation%20Governed%20Tables.ipynb)
+ - [033 - Amazon Neptune](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/033%20-%20Amazon%20Neptune.ipynb)
+ - [034 - Glue Data Quality](https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/034%20-Glue%20%20Data%20Quality.ipynb)
+- [**API Reference**](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html)
+ - [Amazon S3](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-s3)
+ - [AWS Glue Catalog](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#aws-glue-catalog)
+ - [Amazon Athena](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-athena)
+ - [AWS Lake Formation](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#aws-lake-formation)
+ - [Amazon Redshift](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-redshift)
+ - [PostgreSQL](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#postgresql)
+ - [MySQL](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#mysql)
+ - [SQL Server](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#sqlserver)
+ - [Oracle](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#oracle)
+ - [Data API Redshift](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#data-api-redshift)
+ - [Data API RDS](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#data-api-rds)
+ - [OpenSearch](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#opensearch)
+ - [Amazon Neptune](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-neptune)
+ - [DynamoDB](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#dynamodb)
+ - [Amazon Timestream](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-timestream)
+ - [Amazon EMR](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-emr)
+ - [Amazon CloudWatch Logs](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-cloudwatch-logs)
+ - [Amazon Chime](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-chime)
+ - [Amazon QuickSight](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#amazon-quicksight)
+ - [AWS STS](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#aws-sts)
+ - [AWS Secrets Manager](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#aws-secrets-manager)
+ - [Global Configurations](https://aws-sdk-pandas.readthedocs.io/en/2.20.1/api.html#global-configurations)
+- [**License**](https://github.com/aws/aws-sdk-pandas/blob/main/LICENSE.txt)
+- [**Contributing**](https://github.com/aws/aws-sdk-pandas/blob/main/CONTRIBUTING.md)
+
+## Getting Help
+
+The best way to interact with our team is through GitHub. You can open an [issue](https://github.com/aws/aws-sdk-pandas/issues/new/choose) and choose from one of our templates for bug reports, feature requests...
+You may also find help on these community resources:
+* The #aws-sdk-pandas Slack [channel](https://join.slack.com/t/aws-sdk-pandas/shared_invite/zt-sxdx38sl-E0coRfAds8WdpxXD2Nzfrg)
+* Ask a question on [Stack Overflow](https://stackoverflow.com/questions/tagged/awswrangler)
+ and tag it with `awswrangler`
+
+## Community Resources
+
+Please [send a Pull Request](https://github.com/aws/aws-sdk-pandas/edit/main/README.md) with your resource reference and @githubhandle.
+
+- [Optimize Python ETL by extending Pandas with AWS SDK for pandas](https://aws.amazon.com/blogs/big-data/optimize-python-etl-by-extending-pandas-with-aws-data-wrangler/) [[@igorborgest](https://github.com/igorborgest)]
+- [Reading Parquet Files With AWS Lambda](https://aprakash.wordpress.com/2020/04/14/reading-parquet-files-with-aws-lambda/) [[@anand086](https://github.com/anand086)]
+- [Transform AWS CloudTrail data using AWS SDK for pandas](https://aprakash.wordpress.com/2020/09/17/transform-aws-cloudtrail-data-using-aws-data-wrangler/) [[@anand086](https://github.com/anand086)]
+- [Rename Glue Tables using AWS SDK for pandas](https://ananddatastories.com/rename-glue-tables-using-aws-sdk-pandas/) [[@anand086](https://github.com/anand086)]
+- [Getting started on AWS SDK for pandas and Athena](https://medium.com/@dheerajsharmainampudi/getting-started-on-aws-sdk-pandas-and-athena-7b446c834076) [[@dheerajsharma21](https://github.com/dheerajsharma21)]
+- [Simplifying Pandas integration with AWS data related services](https://medium.com/@bv_subhash/aws-sdk-pandas-simplifying-pandas-integration-with-aws-data-related-services-2b3325c12188) [[@bvsubhash](https://github.com/bvsubhash)]
+- [Build an ETL pipeline using AWS S3, Glue and Athena](https://www.linkedin.com/pulse/build-etl-pipeline-using-aws-s3-glue-athena-data-wrangler-tom-reid/) [[@taupirho](https://github.com/taupirho)]
+
+## Logging
+
+Enabling internal logging examples:
+
+```py3
+import logging
+logging.basicConfig(level=logging.INFO, format="[%(name)s][%(funcName)s] %(message)s")
+logging.getLogger("awswrangler").setLevel(logging.DEBUG)
+logging.getLogger("botocore.credentials").setLevel(logging.CRITICAL)
+```
+
+Into AWS lambda:
+
+```py3
+import logging
+logging.getLogger("awswrangler").setLevel(logging.DEBUG)
+```
+
+## Who uses AWS SDK for pandas?
+
+Knowing which companies are using this library is important to help prioritize the project internally.
+If you would like us to include your company’s name and/or logo in the README file to indicate that your company is using the AWS SDK for pandas, please raise a "Support Us" issue. If you would like us to display your company’s logo, please raise a linked pull request to provide an image file for the logo. Note that by raising a Support Us issue (and related pull request), you are granting AWS permission to use your company’s name (and logo) for the limited purpose described here and you are confirming that you have authority to grant such permission.
+
+- [Amazon](https://www.amazon.com/)
+- [AWS](https://aws.amazon.com/)
+- [Cepsa](https://cepsa.com) [[@alvaropc](https://github.com/alvaropc)]
+- [Cognitivo](https://www.cognitivo.ai/) [[@msantino](https://github.com/msantino)]
+- [Digio](https://www.digio.com.br/) [[@afonsomy](https://github.com/afonsomy)]
+- [DNX](https://www.dnx.solutions/) [[@DNXLabs](https://github.com/DNXLabs)]
+- [Fortescue Future Industries](https://ffi.com.au/) [[@spencervoorend](https://github.com/spencervoorend)]
+- [Funcional Health Tech](https://www.funcionalcorp.com.br/) [[@webysther](https://github.com/webysther)]
+- [Funding Circle](https://www.fundingcircle.com/) [[@pfig](https://github.com/pfig)]
+- [Infomach](https://www.infomach.com.br/)
+- [Informa Markets](https://www.informamarkets.com/en/home.html) [[@mateusmorato]](http://github.com/mateusmorato)
+- [LINE TV](https://www.linetv.tw/) [[@bryanyang0528](https://github.com/bryanyang0528)]
+- [Magnataur](https://magnataur.com) [[@brianmingus2](https://github.com/brianmingus2)]
+- [M4U](https://www.m4u.com.br/) [[@Thiago-Dantas](https://github.com/Thiago-Dantas)]
+- [NBCUniversal](https://www.nbcuniversal.com/) [[@vibe](https://github.com/vibe)]
+- [nrd.io](https://nrd.io/) [[@mrtns](https://github.com/mrtns)]
+- [OKRA Technologies](https://okra.ai) [[@JPFrancoia](https://github.com/JPFrancoia), [@schot](https://github.com/schot)]
+- [Pier](https://www.pier.digital/) [[@flaviomax](https://github.com/flaviomax)]
+- [Pismo](https://www.pismo.io/) [[@msantino](https://github.com/msantino)]
+- [ringDNA](https://www.ringdna.com/) [[@msropp](https://github.com/msropp)]
+- [Serasa Experian](https://www.serasaexperian.com.br/) [[@andre-marcos-perez](https://github.com/andre-marcos-perez)]
+- [Shipwell](https://shipwell.com/) [[@zacharycarter](https://github.com/zacharycarter)]
+- [strongDM](https://www.strongdm.com/) [[@mrtns](https://github.com/mrtns)]
+- [Thinkbumblebee](https://www.thinkbumblebee.com/) [[@dheerajsharma21]](https://github.com/dheerajsharma21)
+- [VTEX](https://vtex.com/us-en/) [[@igorborgest]](https://github.com/igorborgest)
+- [Zillow](https://www.zillow.com/) [[@nicholas-miles]](https://github.com/nicholas-miles)
+
+
+%prep
+%autosetup -n awswrangler-2.20.1
+
+%build
+%py3_build
+
+%install
+%py3_install
+install -d -m755 %{buildroot}/%{_pkgdocdir}
+if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi
+if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi
+if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi
+if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi
+pushd %{buildroot}
+if [ -d usr/lib ]; then
+ find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/lib64 ]; then
+ find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/bin ]; then
+ find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/sbin ]; then
+ find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+touch doclist.lst
+if [ -d usr/share/man ]; then
+ find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst
+fi
+popd
+mv %{buildroot}/filelist.lst .
+mv %{buildroot}/doclist.lst .
+
+%files -n python3-awswrangler -f filelist.lst
+%dir %{python3_sitelib}/*
+
+%files help -f doclist.lst
+%{_docdir}/*
+
+%changelog
+* Mon Apr 10 2023 Python_Bot <Python_Bot@openeuler.org> - 2.20.1-1
+- Package Spec generated
diff --git a/sources b/sources
new file mode 100644
index 0000000..f93c928
--- /dev/null
+++ b/sources
@@ -0,0 +1 @@
+e7f848b8677bc6fe97e2cb7cfc504b97 awswrangler-2.20.1.tar.gz