diff options
author | CoprDistGit <infra@openeuler.org> | 2023-06-20 08:35:45 +0000 |
---|---|---|
committer | CoprDistGit <infra@openeuler.org> | 2023-06-20 08:35:45 +0000 |
commit | f43ebca3cd0198311b66f27ddbbf13992a6bc3ef (patch) | |
tree | 20c68d06ebd3a4c15489fa123f0d475a6d086dc2 | |
parent | d16a131e32477db92f4cd10a9a90fe951d0b05cb (diff) |
automatic import of python-dbrequestsopeneuler20.03
-rw-r--r-- | .gitignore | 1 | ||||
-rw-r--r-- | python-dbrequests.spec | 699 | ||||
-rw-r--r-- | sources | 1 |
3 files changed, 701 insertions, 0 deletions
@@ -0,0 +1 @@ +/dbrequests-1.4.8.tar.gz diff --git a/python-dbrequests.spec b/python-dbrequests.spec new file mode 100644 index 0000000..fb5e89a --- /dev/null +++ b/python-dbrequests.spec @@ -0,0 +1,699 @@ +%global _empty_manifest_terminate_build 0 +Name: python-dbrequests +Version: 1.4.8 +Release: 1 +Summary: Python package for querying and connecting to databases. +License: MIT +URL: https://github.com/INWTlab/dbrequests +Source0: https://mirrors.aliyun.com/pypi/web/packages/1a/ff/41d609ce2f84e62ec5f80b0e8ac48e88fdf24f14952695fea09e43f96883/dbrequests-1.4.8.tar.gz +BuildArch: noarch + +Requires: python3-pandas +Requires: python3-SQLAlchemy +Requires: python3-datatable +Requires: python3-psycopg2 +Requires: python3-sqlalchemy-redshift +Requires: python3-psycopg2 + +%description +# dbrequests + +[](https://travis-ci.org/INWTlab/dbrequests) + +**dbrequests is a python package built for easy use of raw SQL within python and pandas projects.** + +It uses ideas from [records](https://github.com/kennethreitz/records/) and is built using [sqlalchemy-engines](https://www.sqlalchemy.org/), but is more heavily integrated with pandas. It aims to reproduce the pilosophy behind the R-package [dbtools](https://github.com/INWT/dbtools/). + + +_Database support includes RedShift, Postgres, MySQL, SQLite, Oracle, and MS-SQL (drivers not included)._ + +## Usage + +### Send queries and bulk queries + +Easy sending of raw sql and output as pandas DataFrames, with credentials given as dictionary (for an example see creds_example.json) or the url of the database: + +```python +from dbrequests import Database + +db = Database(creds=creds) +df = db.send_query(""" + SELECT * FROM test; + """) +df # table test as pandas DataFrame +``` + +You can put the sql query in a file and direct `send_query` to the file. The sql-file may be parametrized: + +```sql +SELECT {col1}, {col2} FROM test; + +``` + +```python +from dbrequests import Database + +db = Database(creds=creds, sql_dir = '/path/to/dir/') +db.send_query('select', col1='id', col2='name') +df # table test, including columns 'id', 'name' as pandas DataFrame +``` + +You can also pass arguments to pandas `read_sql`-Function: + +```python +from dbrequests import Database + +db = Database(creds=creds, sql_dir = '/path/to/dir/') +db.send_query('select', col1='id', col2='name', index_col='id') +df # table test, including column 'name' as pandas DataFrame with index 'id' +``` + +You may also send queries with no table as output to the database via `send_bulk_query`, which exhibits the same behavior as `send_query`: + +```python +db.send_bulk_query('drop test from test;') +``` + +### Send data + +Easy sending of pandas Dataframes in multiple modes: + +```python +db.send_data(df, 'table', mode='insert') +``` + +Supported modes are: + - 'insert': Appending new records. Duplicate primary keys will result in errors (sql insert into). + - 'truncate': Delete the table and completely rewrite it (sql truncate and insert into). + - 'replace': Replace records with duplicate primary keys (sql replace into). + - 'update': Update records with duplicate primary keys (sql insert into duplicate key update). + +### Utilities + +- Comments can be automatically removed from SQL code by adding `remove_comments=True` either to the Database defintion or send_query. This is especially useful if outcommenting code blocks including parametized variables and thus `{}`. The default of this behavior is `False`. +- Percentage signs can be transfered to a Python readable way by adding `escape_percentage=True` either to the Database definition or send_query. This means percentage signs dont have to be escaped manually when sending them via Python. The default is `False`. +- Database.get_table_names will give existing tables in the database +- Parameters such es `chunksize` for `pandas.to_sql` may be given to the wrapper function `send_data` and are handed over to pandas. The same is true for `send_query`. +- For transactions the context manager `transaction` may be of use. + +## Installation + +The package can be installed via pip: + +``` +pip install dbrequests +``` + +## Extensibility +dbrequests is designed to easily accommodate different needs in the form of drivers / dialects. For examples of how to extend the capabilities of the Connection class, see connection_subclass.py under examples. + +### Existing extensions +- MySQL / MariaDB: use +``` +from dbrequests.mysql import Database +``` +for using the MySQL specific extension as Database connector. The extension provides MySQL specific functionalities, like using `load data infile` for writing data to tables. + + +## Version 1.0 + - implemented send_data and send_query wrappers for pandas sql-functionality + +## Version 1.0.8 + - renamed to dbrequests + +## Version 1.1 + - added handling of percentage signs + - added possibility of automated comment removal + - ensured the compatibility of raw SQL code with code sent via dbrequests + +## Version 1.2 + - changed static loading of the connection class to a flexible model, allowing to override connections for different SQL drivers + +## Version 1.3 + - added mysql backend for specific mysql and mariadb support + - added possibility for send_data to use infile for mysql databases + +## Version 1.3.8 + - dbrequests.mysql module: + - see #11 for motivation on moving dialect specific implementation into extras + - see #15 for mysql specific send_data via load data local infile + +## Version 1.3.9 + - dbrequests.mysql: + - see #20 for memory efficient send_data using datatable + +## Version 1.3.10 + - dbrequests.mysql: + - see #24 for bugfix while writing to csv + +## Version 1.3.11 + - dbrequests.mysql: + - see #28 for bugfix when sending escape sequences + +## Version 1.3.12 + - dbrequests.mysql: + - see #22 for performance improvements for send_query when reading large datasets + - support for pymysql and mysqldb + +## Version 1.3.13 + - dbrequests.mysql + - see #30: fixes handling of None/NULL values in columns + +## Version 1.3.14 + - dbrequests.mysql: + - see #32 for bugfix in send_query for empty result sets + +## Version 1.3.15 + - dbrequests: + - see #27 for bugfix in Database class when specifiying a port in a + credentials object. + - the argument 'creds' in the init method of a database class is now + deprecated + - the argument 'db_url' can now handle str and dict type; str is a + sqlalchemy url; a dict a credentials object + - credential objects can now have additional fields which will be used as + elements in connect_args for sqlalchemies create_engine: see #12 + - dbrequests.mysql + - see #36 for bugfix while sending an empty frame + +## Version 1.3.16 + - dbrequests.mysql + - see #35: New send_data modes: update_diffs, insert_diffs, replace_diffs + +## Version 1.3.17 + - dbrequests.mysql + - new function in database: send_delete for deleting rows in a database. + - send_data in mode 'update' now allows to send only partial subset of + columns. + +## Version 1.4.0 + - dbrequests.mysql + - bugfix in send_data with mode [update|insert|replace]_diffs: same as #36 + +## Version 1.4.1 + - dbrequests.mysql + - bugfix for upstream bug in mariadb: sending diffs needs persisten tables + instead of temporary. + +## Version 1.4.2 + - dbrequests.mysql + - creating temorary removing partitions and system versioned from temporary + tables. + - new mode for send data: sync_diffs: update differences and delete + deprecated rows. + - new mode for delete data: in_delete_col: mark rows to delete, then delete. + - bugfix for temporary tables: now properly removes tables. + +## Version 1.4.3-5 + - dbrequests.mysql + - More stable and reliable Version of sync_diffs mode for sending data. + Respects scarce resources on the mysql server. + +## Version 1.4.6 + - dbrequests.mysql + - HOTFIX #51: Bug with latest datatable version + +## Version 1.4.7 + - dbrequests.mysql + - bugfix of #52 + +## Version 1.4.8 + - dbrequests.mysql + - fix for creating temporary files on Windows. + + + + + +%package -n python3-dbrequests +Summary: Python package for querying and connecting to databases. +Provides: python-dbrequests +BuildRequires: python3-devel +BuildRequires: python3-setuptools +BuildRequires: python3-pip +%description -n python3-dbrequests +# dbrequests + +[](https://travis-ci.org/INWTlab/dbrequests) + +**dbrequests is a python package built for easy use of raw SQL within python and pandas projects.** + +It uses ideas from [records](https://github.com/kennethreitz/records/) and is built using [sqlalchemy-engines](https://www.sqlalchemy.org/), but is more heavily integrated with pandas. It aims to reproduce the pilosophy behind the R-package [dbtools](https://github.com/INWT/dbtools/). + + +_Database support includes RedShift, Postgres, MySQL, SQLite, Oracle, and MS-SQL (drivers not included)._ + +## Usage + +### Send queries and bulk queries + +Easy sending of raw sql and output as pandas DataFrames, with credentials given as dictionary (for an example see creds_example.json) or the url of the database: + +```python +from dbrequests import Database + +db = Database(creds=creds) +df = db.send_query(""" + SELECT * FROM test; + """) +df # table test as pandas DataFrame +``` + +You can put the sql query in a file and direct `send_query` to the file. The sql-file may be parametrized: + +```sql +SELECT {col1}, {col2} FROM test; + +``` + +```python +from dbrequests import Database + +db = Database(creds=creds, sql_dir = '/path/to/dir/') +db.send_query('select', col1='id', col2='name') +df # table test, including columns 'id', 'name' as pandas DataFrame +``` + +You can also pass arguments to pandas `read_sql`-Function: + +```python +from dbrequests import Database + +db = Database(creds=creds, sql_dir = '/path/to/dir/') +db.send_query('select', col1='id', col2='name', index_col='id') +df # table test, including column 'name' as pandas DataFrame with index 'id' +``` + +You may also send queries with no table as output to the database via `send_bulk_query`, which exhibits the same behavior as `send_query`: + +```python +db.send_bulk_query('drop test from test;') +``` + +### Send data + +Easy sending of pandas Dataframes in multiple modes: + +```python +db.send_data(df, 'table', mode='insert') +``` + +Supported modes are: + - 'insert': Appending new records. Duplicate primary keys will result in errors (sql insert into). + - 'truncate': Delete the table and completely rewrite it (sql truncate and insert into). + - 'replace': Replace records with duplicate primary keys (sql replace into). + - 'update': Update records with duplicate primary keys (sql insert into duplicate key update). + +### Utilities + +- Comments can be automatically removed from SQL code by adding `remove_comments=True` either to the Database defintion or send_query. This is especially useful if outcommenting code blocks including parametized variables and thus `{}`. The default of this behavior is `False`. +- Percentage signs can be transfered to a Python readable way by adding `escape_percentage=True` either to the Database definition or send_query. This means percentage signs dont have to be escaped manually when sending them via Python. The default is `False`. +- Database.get_table_names will give existing tables in the database +- Parameters such es `chunksize` for `pandas.to_sql` may be given to the wrapper function `send_data` and are handed over to pandas. The same is true for `send_query`. +- For transactions the context manager `transaction` may be of use. + +## Installation + +The package can be installed via pip: + +``` +pip install dbrequests +``` + +## Extensibility +dbrequests is designed to easily accommodate different needs in the form of drivers / dialects. For examples of how to extend the capabilities of the Connection class, see connection_subclass.py under examples. + +### Existing extensions +- MySQL / MariaDB: use +``` +from dbrequests.mysql import Database +``` +for using the MySQL specific extension as Database connector. The extension provides MySQL specific functionalities, like using `load data infile` for writing data to tables. + + +## Version 1.0 + - implemented send_data and send_query wrappers for pandas sql-functionality + +## Version 1.0.8 + - renamed to dbrequests + +## Version 1.1 + - added handling of percentage signs + - added possibility of automated comment removal + - ensured the compatibility of raw SQL code with code sent via dbrequests + +## Version 1.2 + - changed static loading of the connection class to a flexible model, allowing to override connections for different SQL drivers + +## Version 1.3 + - added mysql backend for specific mysql and mariadb support + - added possibility for send_data to use infile for mysql databases + +## Version 1.3.8 + - dbrequests.mysql module: + - see #11 for motivation on moving dialect specific implementation into extras + - see #15 for mysql specific send_data via load data local infile + +## Version 1.3.9 + - dbrequests.mysql: + - see #20 for memory efficient send_data using datatable + +## Version 1.3.10 + - dbrequests.mysql: + - see #24 for bugfix while writing to csv + +## Version 1.3.11 + - dbrequests.mysql: + - see #28 for bugfix when sending escape sequences + +## Version 1.3.12 + - dbrequests.mysql: + - see #22 for performance improvements for send_query when reading large datasets + - support for pymysql and mysqldb + +## Version 1.3.13 + - dbrequests.mysql + - see #30: fixes handling of None/NULL values in columns + +## Version 1.3.14 + - dbrequests.mysql: + - see #32 for bugfix in send_query for empty result sets + +## Version 1.3.15 + - dbrequests: + - see #27 for bugfix in Database class when specifiying a port in a + credentials object. + - the argument 'creds' in the init method of a database class is now + deprecated + - the argument 'db_url' can now handle str and dict type; str is a + sqlalchemy url; a dict a credentials object + - credential objects can now have additional fields which will be used as + elements in connect_args for sqlalchemies create_engine: see #12 + - dbrequests.mysql + - see #36 for bugfix while sending an empty frame + +## Version 1.3.16 + - dbrequests.mysql + - see #35: New send_data modes: update_diffs, insert_diffs, replace_diffs + +## Version 1.3.17 + - dbrequests.mysql + - new function in database: send_delete for deleting rows in a database. + - send_data in mode 'update' now allows to send only partial subset of + columns. + +## Version 1.4.0 + - dbrequests.mysql + - bugfix in send_data with mode [update|insert|replace]_diffs: same as #36 + +## Version 1.4.1 + - dbrequests.mysql + - bugfix for upstream bug in mariadb: sending diffs needs persisten tables + instead of temporary. + +## Version 1.4.2 + - dbrequests.mysql + - creating temorary removing partitions and system versioned from temporary + tables. + - new mode for send data: sync_diffs: update differences and delete + deprecated rows. + - new mode for delete data: in_delete_col: mark rows to delete, then delete. + - bugfix for temporary tables: now properly removes tables. + +## Version 1.4.3-5 + - dbrequests.mysql + - More stable and reliable Version of sync_diffs mode for sending data. + Respects scarce resources on the mysql server. + +## Version 1.4.6 + - dbrequests.mysql + - HOTFIX #51: Bug with latest datatable version + +## Version 1.4.7 + - dbrequests.mysql + - bugfix of #52 + +## Version 1.4.8 + - dbrequests.mysql + - fix for creating temporary files on Windows. + + + + + +%package help +Summary: Development documents and examples for dbrequests +Provides: python3-dbrequests-doc +%description help +# dbrequests + +[](https://travis-ci.org/INWTlab/dbrequests) + +**dbrequests is a python package built for easy use of raw SQL within python and pandas projects.** + +It uses ideas from [records](https://github.com/kennethreitz/records/) and is built using [sqlalchemy-engines](https://www.sqlalchemy.org/), but is more heavily integrated with pandas. It aims to reproduce the pilosophy behind the R-package [dbtools](https://github.com/INWT/dbtools/). + + +_Database support includes RedShift, Postgres, MySQL, SQLite, Oracle, and MS-SQL (drivers not included)._ + +## Usage + +### Send queries and bulk queries + +Easy sending of raw sql and output as pandas DataFrames, with credentials given as dictionary (for an example see creds_example.json) or the url of the database: + +```python +from dbrequests import Database + +db = Database(creds=creds) +df = db.send_query(""" + SELECT * FROM test; + """) +df # table test as pandas DataFrame +``` + +You can put the sql query in a file and direct `send_query` to the file. The sql-file may be parametrized: + +```sql +SELECT {col1}, {col2} FROM test; + +``` + +```python +from dbrequests import Database + +db = Database(creds=creds, sql_dir = '/path/to/dir/') +db.send_query('select', col1='id', col2='name') +df # table test, including columns 'id', 'name' as pandas DataFrame +``` + +You can also pass arguments to pandas `read_sql`-Function: + +```python +from dbrequests import Database + +db = Database(creds=creds, sql_dir = '/path/to/dir/') +db.send_query('select', col1='id', col2='name', index_col='id') +df # table test, including column 'name' as pandas DataFrame with index 'id' +``` + +You may also send queries with no table as output to the database via `send_bulk_query`, which exhibits the same behavior as `send_query`: + +```python +db.send_bulk_query('drop test from test;') +``` + +### Send data + +Easy sending of pandas Dataframes in multiple modes: + +```python +db.send_data(df, 'table', mode='insert') +``` + +Supported modes are: + - 'insert': Appending new records. Duplicate primary keys will result in errors (sql insert into). + - 'truncate': Delete the table and completely rewrite it (sql truncate and insert into). + - 'replace': Replace records with duplicate primary keys (sql replace into). + - 'update': Update records with duplicate primary keys (sql insert into duplicate key update). + +### Utilities + +- Comments can be automatically removed from SQL code by adding `remove_comments=True` either to the Database defintion or send_query. This is especially useful if outcommenting code blocks including parametized variables and thus `{}`. The default of this behavior is `False`. +- Percentage signs can be transfered to a Python readable way by adding `escape_percentage=True` either to the Database definition or send_query. This means percentage signs dont have to be escaped manually when sending them via Python. The default is `False`. +- Database.get_table_names will give existing tables in the database +- Parameters such es `chunksize` for `pandas.to_sql` may be given to the wrapper function `send_data` and are handed over to pandas. The same is true for `send_query`. +- For transactions the context manager `transaction` may be of use. + +## Installation + +The package can be installed via pip: + +``` +pip install dbrequests +``` + +## Extensibility +dbrequests is designed to easily accommodate different needs in the form of drivers / dialects. For examples of how to extend the capabilities of the Connection class, see connection_subclass.py under examples. + +### Existing extensions +- MySQL / MariaDB: use +``` +from dbrequests.mysql import Database +``` +for using the MySQL specific extension as Database connector. The extension provides MySQL specific functionalities, like using `load data infile` for writing data to tables. + + +## Version 1.0 + - implemented send_data and send_query wrappers for pandas sql-functionality + +## Version 1.0.8 + - renamed to dbrequests + +## Version 1.1 + - added handling of percentage signs + - added possibility of automated comment removal + - ensured the compatibility of raw SQL code with code sent via dbrequests + +## Version 1.2 + - changed static loading of the connection class to a flexible model, allowing to override connections for different SQL drivers + +## Version 1.3 + - added mysql backend for specific mysql and mariadb support + - added possibility for send_data to use infile for mysql databases + +## Version 1.3.8 + - dbrequests.mysql module: + - see #11 for motivation on moving dialect specific implementation into extras + - see #15 for mysql specific send_data via load data local infile + +## Version 1.3.9 + - dbrequests.mysql: + - see #20 for memory efficient send_data using datatable + +## Version 1.3.10 + - dbrequests.mysql: + - see #24 for bugfix while writing to csv + +## Version 1.3.11 + - dbrequests.mysql: + - see #28 for bugfix when sending escape sequences + +## Version 1.3.12 + - dbrequests.mysql: + - see #22 for performance improvements for send_query when reading large datasets + - support for pymysql and mysqldb + +## Version 1.3.13 + - dbrequests.mysql + - see #30: fixes handling of None/NULL values in columns + +## Version 1.3.14 + - dbrequests.mysql: + - see #32 for bugfix in send_query for empty result sets + +## Version 1.3.15 + - dbrequests: + - see #27 for bugfix in Database class when specifiying a port in a + credentials object. + - the argument 'creds' in the init method of a database class is now + deprecated + - the argument 'db_url' can now handle str and dict type; str is a + sqlalchemy url; a dict a credentials object + - credential objects can now have additional fields which will be used as + elements in connect_args for sqlalchemies create_engine: see #12 + - dbrequests.mysql + - see #36 for bugfix while sending an empty frame + +## Version 1.3.16 + - dbrequests.mysql + - see #35: New send_data modes: update_diffs, insert_diffs, replace_diffs + +## Version 1.3.17 + - dbrequests.mysql + - new function in database: send_delete for deleting rows in a database. + - send_data in mode 'update' now allows to send only partial subset of + columns. + +## Version 1.4.0 + - dbrequests.mysql + - bugfix in send_data with mode [update|insert|replace]_diffs: same as #36 + +## Version 1.4.1 + - dbrequests.mysql + - bugfix for upstream bug in mariadb: sending diffs needs persisten tables + instead of temporary. + +## Version 1.4.2 + - dbrequests.mysql + - creating temorary removing partitions and system versioned from temporary + tables. + - new mode for send data: sync_diffs: update differences and delete + deprecated rows. + - new mode for delete data: in_delete_col: mark rows to delete, then delete. + - bugfix for temporary tables: now properly removes tables. + +## Version 1.4.3-5 + - dbrequests.mysql + - More stable and reliable Version of sync_diffs mode for sending data. + Respects scarce resources on the mysql server. + +## Version 1.4.6 + - dbrequests.mysql + - HOTFIX #51: Bug with latest datatable version + +## Version 1.4.7 + - dbrequests.mysql + - bugfix of #52 + +## Version 1.4.8 + - dbrequests.mysql + - fix for creating temporary files on Windows. + + + + + +%prep +%autosetup -n dbrequests-1.4.8 + +%build +%py3_build + +%install +%py3_install +install -d -m755 %{buildroot}/%{_pkgdocdir} +if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi +if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi +if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi +if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi +pushd %{buildroot} +if [ -d usr/lib ]; then + find usr/lib -type f -printf "\"/%h/%f\"\n" >> filelist.lst +fi +if [ -d usr/lib64 ]; then + find usr/lib64 -type f -printf "\"/%h/%f\"\n" >> filelist.lst +fi +if [ -d usr/bin ]; then + find usr/bin -type f -printf "\"/%h/%f\"\n" >> filelist.lst +fi +if [ -d usr/sbin ]; then + find usr/sbin -type f -printf "\"/%h/%f\"\n" >> filelist.lst +fi +touch doclist.lst +if [ -d usr/share/man ]; then + find usr/share/man -type f -printf "\"/%h/%f.gz\"\n" >> doclist.lst +fi +popd +mv %{buildroot}/filelist.lst . +mv %{buildroot}/doclist.lst . + +%files -n python3-dbrequests -f filelist.lst +%dir %{python3_sitelib}/* + +%files help -f doclist.lst +%{_docdir}/* + +%changelog +* Tue Jun 20 2023 Python_Bot <Python_Bot@openeuler.org> - 1.4.8-1 +- Package Spec generated @@ -0,0 +1 @@ +977009277e77f32d7553bed33f9a7824 dbrequests-1.4.8.tar.gz |