automatic import of python-bqtools-jsonopeneuler20.03

3 files changed, 275 insertions, 0 deletions

diff --git a/.gitignore b/.gitignore
index e69de29..d3d716d 100644
--- a/.gitignore
+++ b/.gitignore
@@ -0,0 +1 @@
+/bqtools-json-1.0.5.tar.gz
diff --git a/python-bqtools-json.spec b/python-bqtools-json.spec
new file mode 100644
index 0000000..09cc3c9
--- /dev/null
+++ b/python-bqtools-json.spec
@@ -0,0 +1,273 @@
+%global _empty_manifest_terminate_build 0
+Name:		python-bqtools-json
+Version:	1.0.5
+Release:	1
+Summary:	A Big Query json utility package
+License:	Apache Software License
+URL:		https://github.com/hsbc/bqtools
+Source0:	https://mirrors.nju.edu.cn/pypi/web/packages/6a/4d/2967729e8807462c6bd741315be39e48f56cb8876c30f5c414c158040528/bqtools-json-1.0.5.tar.gz
+BuildArch:	noarch
+
+Requires:	python3-jinja2
+Requires:	python3-google-cloud
+Requires:	python3-google-cloud-bigquery
+Requires:	python3-google-cloud-storage
+Requires:	python3-google-cloud-logging
+Requires:	python3-absl-py
+Requires:	python3-boto
+Requires:	python3-google-api-python-client
+Requires:	python3-grpcio
+
+%description
+This is bqtools a handy set of utils to help create big query tables from json objects and make them into tables (so get the schema) and to create json new line files.
+Plus a set of common functions to ease writing biq query python code
+
+Goals
+* Simplify handling and move between big query stryuctured data and json data
+* Allow you to create big query table schemas from json structures (as resource or schema objects) uses reflection of types
+* Provides easy generation of views for day partitioned tabled
+  * head - latest data
+  * diff views for time snap shots of data i.e. each day partition has current view of data
+* Calculate valid json structures from representative json data that can be used as basis of big query schemas
+* Clean json data such that it can be loaded into big query
+  * Replace bare lists with dictionaries
+  * Replace field  names with valid values that can be column names in big query (removes spaces, characters not allowed in field names using same algorithms big query uses when auto detecting schemas)
+  * Encodes json output of dates, datetimes, times, timedeltas encoded in format acceptable for big query corresponding field types
+* Generate code for bq command line tool from bq table structures
+* Simplify common tasks of handling big query data
+  * Basic tests on dataset or tables existing
+  * Schema patching compare an existing schema to a template json object calculate if changed and generate a merged schema that can be used in a patch
+  * Flattening views to avoid view depth limits
+  
+```
+import bqtools
+
+# if you load a json object say something like
+foo = {
+        "id":1,
+        "description":""
+        "aboolean":False
+      }
+      
+# generate a schema
+table = {
+   "type":"TABLE",
+   "location":os.environ["location"],
+   "tableReference":{
+       "projectId": os.environ["projectid"],
+       "datasetId": os.environ["dataset"],
+       "tableId": key
+   },
+   "timePartitioning":{
+       "type": "DAY",
+       "expirationMs": "94608000000"
+   },
+   "schema": {}
+}
+
+# use bqtools to create a schema structure
+table["schema"]["fields"] = bqtools.get_bq_schema_from_json_repr(foo)
+
+```
+
+Demonstrates some of power of tools via [bqsync](https://github.com/hsbc/bqtools/blob/master/BQSYNCUSAGE.md) that is installed if you install via pip.
+
+    pip install bqtools-json
+    
+Or you can find the source for this [here](https://github.com/hsbc/bqtools/blob/master/bqtools/bqsync)
+    
+Also provides means to handle views across environments supports SQL views based upon Jinja templates. So you can "configure" views for different environments.
+
+Provides support for calculating view dependencies such views can be applied in parallel tranches.
+
+Provides support for calculating authorised views based on set of views "compiled"
+
+Compilation flattens views to lower view depth but automatically does not flatten authorised views to keep access to data tight as feasible.
+
+
+%package -n python3-bqtools-json
+Summary:	A Big Query json utility package
+Provides:	python-bqtools-json
+BuildRequires:	python3-devel
+BuildRequires:	python3-setuptools
+BuildRequires:	python3-pip
+%description -n python3-bqtools-json
+This is bqtools a handy set of utils to help create big query tables from json objects and make them into tables (so get the schema) and to create json new line files.
+Plus a set of common functions to ease writing biq query python code
+
+Goals
+* Simplify handling and move between big query stryuctured data and json data
+* Allow you to create big query table schemas from json structures (as resource or schema objects) uses reflection of types
+* Provides easy generation of views for day partitioned tabled
+  * head - latest data
+  * diff views for time snap shots of data i.e. each day partition has current view of data
+* Calculate valid json structures from representative json data that can be used as basis of big query schemas
+* Clean json data such that it can be loaded into big query
+  * Replace bare lists with dictionaries
+  * Replace field  names with valid values that can be column names in big query (removes spaces, characters not allowed in field names using same algorithms big query uses when auto detecting schemas)
+  * Encodes json output of dates, datetimes, times, timedeltas encoded in format acceptable for big query corresponding field types
+* Generate code for bq command line tool from bq table structures
+* Simplify common tasks of handling big query data
+  * Basic tests on dataset or tables existing
+  * Schema patching compare an existing schema to a template json object calculate if changed and generate a merged schema that can be used in a patch
+  * Flattening views to avoid view depth limits
+  
+```
+import bqtools
+
+# if you load a json object say something like
+foo = {
+        "id":1,
+        "description":""
+        "aboolean":False
+      }
+      
+# generate a schema
+table = {
+   "type":"TABLE",
+   "location":os.environ["location"],
+   "tableReference":{
+       "projectId": os.environ["projectid"],
+       "datasetId": os.environ["dataset"],
+       "tableId": key
+   },
+   "timePartitioning":{
+       "type": "DAY",
+       "expirationMs": "94608000000"
+   },
+   "schema": {}
+}
+
+# use bqtools to create a schema structure
+table["schema"]["fields"] = bqtools.get_bq_schema_from_json_repr(foo)
+
+```
+
+Demonstrates some of power of tools via [bqsync](https://github.com/hsbc/bqtools/blob/master/BQSYNCUSAGE.md) that is installed if you install via pip.
+
+    pip install bqtools-json
+    
+Or you can find the source for this [here](https://github.com/hsbc/bqtools/blob/master/bqtools/bqsync)
+    
+Also provides means to handle views across environments supports SQL views based upon Jinja templates. So you can "configure" views for different environments.
+
+Provides support for calculating view dependencies such views can be applied in parallel tranches.
+
+Provides support for calculating authorised views based on set of views "compiled"
+
+Compilation flattens views to lower view depth but automatically does not flatten authorised views to keep access to data tight as feasible.
+
+
+%package help
+Summary:	Development documents and examples for bqtools-json
+Provides:	python3-bqtools-json-doc
+%description help
+This is bqtools a handy set of utils to help create big query tables from json objects and make them into tables (so get the schema) and to create json new line files.
+Plus a set of common functions to ease writing biq query python code
+
+Goals
+* Simplify handling and move between big query stryuctured data and json data
+* Allow you to create big query table schemas from json structures (as resource or schema objects) uses reflection of types
+* Provides easy generation of views for day partitioned tabled
+  * head - latest data
+  * diff views for time snap shots of data i.e. each day partition has current view of data
+* Calculate valid json structures from representative json data that can be used as basis of big query schemas
+* Clean json data such that it can be loaded into big query
+  * Replace bare lists with dictionaries
+  * Replace field  names with valid values that can be column names in big query (removes spaces, characters not allowed in field names using same algorithms big query uses when auto detecting schemas)
+  * Encodes json output of dates, datetimes, times, timedeltas encoded in format acceptable for big query corresponding field types
+* Generate code for bq command line tool from bq table structures
+* Simplify common tasks of handling big query data
+  * Basic tests on dataset or tables existing
+  * Schema patching compare an existing schema to a template json object calculate if changed and generate a merged schema that can be used in a patch
+  * Flattening views to avoid view depth limits
+  
+```
+import bqtools
+
+# if you load a json object say something like
+foo = {
+        "id":1,
+        "description":""
+        "aboolean":False
+      }
+      
+# generate a schema
+table = {
+   "type":"TABLE",
+   "location":os.environ["location"],
+   "tableReference":{
+       "projectId": os.environ["projectid"],
+       "datasetId": os.environ["dataset"],
+       "tableId": key
+   },
+   "timePartitioning":{
+       "type": "DAY",
+       "expirationMs": "94608000000"
+   },
+   "schema": {}
+}
+
+# use bqtools to create a schema structure
+table["schema"]["fields"] = bqtools.get_bq_schema_from_json_repr(foo)
+
+```
+
+Demonstrates some of power of tools via [bqsync](https://github.com/hsbc/bqtools/blob/master/BQSYNCUSAGE.md) that is installed if you install via pip.
+
+    pip install bqtools-json
+    
+Or you can find the source for this [here](https://github.com/hsbc/bqtools/blob/master/bqtools/bqsync)
+    
+Also provides means to handle views across environments supports SQL views based upon Jinja templates. So you can "configure" views for different environments.
+
+Provides support for calculating view dependencies such views can be applied in parallel tranches.
+
+Provides support for calculating authorised views based on set of views "compiled"
+
+Compilation flattens views to lower view depth but automatically does not flatten authorised views to keep access to data tight as feasible.
+
+
+%prep
+%autosetup -n bqtools-json-1.0.5
+
+%build
+%py3_build
+
+%install
+%py3_install
+install -d -m755 %{buildroot}/%{_pkgdocdir}
+if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi
+if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi
+if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi
+if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi
+pushd %{buildroot}
+if [ -d usr/lib ]; then
+	find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/lib64 ]; then
+	find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/bin ]; then
+	find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/sbin ]; then
+	find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+touch doclist.lst
+if [ -d usr/share/man ]; then
+	find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst
+fi
+popd
+mv %{buildroot}/filelist.lst .
+mv %{buildroot}/doclist.lst .
+
+%files -n python3-bqtools-json -f filelist.lst
+%dir %{python3_sitelib}/*
+
+%files help -f doclist.lst
+%{_docdir}/*
+
+%changelog
+* Fri May 05 2023 Python_Bot <Python_Bot@openeuler.org> - 1.0.5-1
+- Package Spec generated
diff --git a/sources b/sources
new file mode 100644
index 0000000..ea3ebf9
--- /dev/null
+++ b/sources
@@ -0,0 +1 @@
+82f99efe49f1c389e378621c69cfe4de  bqtools-json-1.0.5.tar.gz