%global _empty_manifest_terminate_build 0 Name: python-es-pandas Version: 0.0.22 Release: 1 Summary: Read, write and update large scale pandas DataFrame with ElasticSearch License: MIT License URL: https://github.com/fuyb1992/es_pandas Source0: https://mirrors.nju.edu.cn/pypi/web/packages/8d/fb/d0323116dd231002763468a3f7fb03482a47a8d1f8dd359225b0c9e3b02a/es_pandas-0.0.22.tar.gz BuildArch: noarch Requires: python3-pandas Requires: python3-elasticsearch Requires: python3-tqdm Requires: python3-requests %description # es_pandas [![Build Status](https://travis-ci.org/fuyb1992/es_pandas.svg?branch=master)](https://travis-ci.org/fuyb1992/es_pandas) 996.icu [![LICENSE](https://img.shields.io/badge/license-Anti%20996-blue.svg)](https://github.com/996icu/996.ICU/blob/master/LICENSE) [![PyPi version](https://img.shields.io/pypi/v/es_pandas)](https://pypi.org/project/es-pandas/) [![Downloads](https://pepy.tech/badge/es-pandas/month)](https://pepy.tech/project/es-pandas) Read, write and update large scale [pandas](http://pandas.pydata.org/) DataFrame with [ElasticSearch](https://www.elastic.co/). ## Requirements This package should work on Python3(>=3.4) and ElasticSearch should be version 5.x, 6.x or 7.x. Installation The package is hosted on PyPi and can be installed with pip: ``` pip install es_pandas ``` #### Deprecation Notice Supporting of ElasticSearch 5.x will by deprecated in future version. ## Usage ``` import time import pandas as pd from es_pandas import es_pandas # Information of es cluseter es_host = 'localhost:9200' index = 'demo' # crete es_pandas instance ep = es_pandas(es_host) # Example data frame df = pd.DataFrame({'Num': [x for x in range(100000)]}) df['Alpha'] = 'Hello' df['Date'] = pd.datetime.now() # init template if you want doc_type = 'demo' ep.init_es_tmpl(df, doc_type) # Example of write data to es, use the template you create ep.to_es(df, index, doc_type=doc_type, thread_count=2, chunk_size=10000) # set use_index=True if you want to use DataFrame index as records' _id ep.to_es(df, index, doc_type=doc_type, use_index=True, thread_count=2, chunk_size=10000) # delete records from es ep.to_es(df.iloc[5000:], index, doc_type=doc_type, _op_type='delete', thread_count=2, chunk_size=10000) # Update doc by doc _id df.iloc[:1000, 1] = 'Bye' df.iloc[:1000, 2] = pd.datetime.now() ep.to_es(df.iloc[:1000, 1:], index, doc_type=doc_type, _op_type='update') # Example of read data from es df = ep.to_pandas(index) print(df.head()) # return certain fields in es heads = ['Num', 'Date'] df = ep.to_pandas(index, heads=heads) print(df.head()) # set certain columns dtype dtype = {'Num': 'float', 'Alpha': object} df = ep.to_pandas(index, dtype=dtype) print(df.dtypes) # infer dtype from es template df = ep.to_pandas(index, infer_dtype=True) print(df.dtypes) # use query_sql parameter if you want to do query in sql # Example of write data to es with pandas.io.json ep.to_es(df, index, doc_type=doc_type, use_pandas_json=True, thread_count=2, chunk_size=10000) print('write es doc with pandas.io.json finished') ``` %package -n python3-es-pandas Summary: Read, write and update large scale pandas DataFrame with ElasticSearch Provides: python-es-pandas BuildRequires: python3-devel BuildRequires: python3-setuptools BuildRequires: python3-pip %description -n python3-es-pandas # es_pandas [![Build Status](https://travis-ci.org/fuyb1992/es_pandas.svg?branch=master)](https://travis-ci.org/fuyb1992/es_pandas) 996.icu [![LICENSE](https://img.shields.io/badge/license-Anti%20996-blue.svg)](https://github.com/996icu/996.ICU/blob/master/LICENSE) [![PyPi version](https://img.shields.io/pypi/v/es_pandas)](https://pypi.org/project/es-pandas/) [![Downloads](https://pepy.tech/badge/es-pandas/month)](https://pepy.tech/project/es-pandas) Read, write and update large scale [pandas](http://pandas.pydata.org/) DataFrame with [ElasticSearch](https://www.elastic.co/). ## Requirements This package should work on Python3(>=3.4) and ElasticSearch should be version 5.x, 6.x or 7.x. Installation The package is hosted on PyPi and can be installed with pip: ``` pip install es_pandas ``` #### Deprecation Notice Supporting of ElasticSearch 5.x will by deprecated in future version. ## Usage ``` import time import pandas as pd from es_pandas import es_pandas # Information of es cluseter es_host = 'localhost:9200' index = 'demo' # crete es_pandas instance ep = es_pandas(es_host) # Example data frame df = pd.DataFrame({'Num': [x for x in range(100000)]}) df['Alpha'] = 'Hello' df['Date'] = pd.datetime.now() # init template if you want doc_type = 'demo' ep.init_es_tmpl(df, doc_type) # Example of write data to es, use the template you create ep.to_es(df, index, doc_type=doc_type, thread_count=2, chunk_size=10000) # set use_index=True if you want to use DataFrame index as records' _id ep.to_es(df, index, doc_type=doc_type, use_index=True, thread_count=2, chunk_size=10000) # delete records from es ep.to_es(df.iloc[5000:], index, doc_type=doc_type, _op_type='delete', thread_count=2, chunk_size=10000) # Update doc by doc _id df.iloc[:1000, 1] = 'Bye' df.iloc[:1000, 2] = pd.datetime.now() ep.to_es(df.iloc[:1000, 1:], index, doc_type=doc_type, _op_type='update') # Example of read data from es df = ep.to_pandas(index) print(df.head()) # return certain fields in es heads = ['Num', 'Date'] df = ep.to_pandas(index, heads=heads) print(df.head()) # set certain columns dtype dtype = {'Num': 'float', 'Alpha': object} df = ep.to_pandas(index, dtype=dtype) print(df.dtypes) # infer dtype from es template df = ep.to_pandas(index, infer_dtype=True) print(df.dtypes) # use query_sql parameter if you want to do query in sql # Example of write data to es with pandas.io.json ep.to_es(df, index, doc_type=doc_type, use_pandas_json=True, thread_count=2, chunk_size=10000) print('write es doc with pandas.io.json finished') ``` %package help Summary: Development documents and examples for es-pandas Provides: python3-es-pandas-doc %description help # es_pandas [![Build Status](https://travis-ci.org/fuyb1992/es_pandas.svg?branch=master)](https://travis-ci.org/fuyb1992/es_pandas) 996.icu [![LICENSE](https://img.shields.io/badge/license-Anti%20996-blue.svg)](https://github.com/996icu/996.ICU/blob/master/LICENSE) [![PyPi version](https://img.shields.io/pypi/v/es_pandas)](https://pypi.org/project/es-pandas/) [![Downloads](https://pepy.tech/badge/es-pandas/month)](https://pepy.tech/project/es-pandas) Read, write and update large scale [pandas](http://pandas.pydata.org/) DataFrame with [ElasticSearch](https://www.elastic.co/). ## Requirements This package should work on Python3(>=3.4) and ElasticSearch should be version 5.x, 6.x or 7.x. Installation The package is hosted on PyPi and can be installed with pip: ``` pip install es_pandas ``` #### Deprecation Notice Supporting of ElasticSearch 5.x will by deprecated in future version. ## Usage ``` import time import pandas as pd from es_pandas import es_pandas # Information of es cluseter es_host = 'localhost:9200' index = 'demo' # crete es_pandas instance ep = es_pandas(es_host) # Example data frame df = pd.DataFrame({'Num': [x for x in range(100000)]}) df['Alpha'] = 'Hello' df['Date'] = pd.datetime.now() # init template if you want doc_type = 'demo' ep.init_es_tmpl(df, doc_type) # Example of write data to es, use the template you create ep.to_es(df, index, doc_type=doc_type, thread_count=2, chunk_size=10000) # set use_index=True if you want to use DataFrame index as records' _id ep.to_es(df, index, doc_type=doc_type, use_index=True, thread_count=2, chunk_size=10000) # delete records from es ep.to_es(df.iloc[5000:], index, doc_type=doc_type, _op_type='delete', thread_count=2, chunk_size=10000) # Update doc by doc _id df.iloc[:1000, 1] = 'Bye' df.iloc[:1000, 2] = pd.datetime.now() ep.to_es(df.iloc[:1000, 1:], index, doc_type=doc_type, _op_type='update') # Example of read data from es df = ep.to_pandas(index) print(df.head()) # return certain fields in es heads = ['Num', 'Date'] df = ep.to_pandas(index, heads=heads) print(df.head()) # set certain columns dtype dtype = {'Num': 'float', 'Alpha': object} df = ep.to_pandas(index, dtype=dtype) print(df.dtypes) # infer dtype from es template df = ep.to_pandas(index, infer_dtype=True) print(df.dtypes) # use query_sql parameter if you want to do query in sql # Example of write data to es with pandas.io.json ep.to_es(df, index, doc_type=doc_type, use_pandas_json=True, thread_count=2, chunk_size=10000) print('write es doc with pandas.io.json finished') ``` %prep %autosetup -n es-pandas-0.0.22 %build %py3_build %install %py3_install install -d -m755 %{buildroot}/%{_pkgdocdir} if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi pushd %{buildroot} if [ -d usr/lib ]; then find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/lib64 ]; then find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/bin ]; then find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst fi if [ -d usr/sbin ]; then find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst fi touch doclist.lst if [ -d usr/share/man ]; then find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst fi popd mv %{buildroot}/filelist.lst . mv %{buildroot}/doclist.lst . %files -n python3-es-pandas -f filelist.lst %dir %{python3_sitelib}/* %files help -f doclist.lst %{_docdir}/* %changelog * Wed Apr 12 2023 Python_Bot - 0.0.22-1 - Package Spec generated