diff options
author | CoprDistGit <infra@openeuler.org> | 2023-04-11 17:32:16 +0000 |
---|---|---|
committer | CoprDistGit <infra@openeuler.org> | 2023-04-11 17:32:16 +0000 |
commit | dcaccac5fbf0313ba3087a2433c202edbd2fedfe (patch) | |
tree | 9a3452662f037f7d92c4bd013909709d131ce08a | |
parent | de8e2f8310c141963957836e798d0875e66ecfa1 (diff) |
automatic import of python-read-protobuf
-rw-r--r-- | .gitignore | 1 | ||||
-rw-r--r-- | python-read-protobuf.spec | 482 | ||||
-rw-r--r-- | sources | 1 |
3 files changed, 484 insertions, 0 deletions
@@ -0,0 +1 @@ +/read-protobuf-0.1.1.tar.gz diff --git a/python-read-protobuf.spec b/python-read-protobuf.spec new file mode 100644 index 0000000..7dbf611 --- /dev/null +++ b/python-read-protobuf.spec @@ -0,0 +1,482 @@ +%global _empty_manifest_terminate_build 0 +Name: python-read-protobuf +Version: 0.1.1 +Release: 1 +Summary: Small library to read serialized protobuf(s) directly into Pandas Dataframe +License: Public Domain +URL: https://github.com/mlshapiro/read-protobuf.git +Source0: https://mirrors.nju.edu.cn/pypi/web/packages/df/4e/25d836d95376c33b311f633dffae5f78cb36c342195c85a469d3fe5b0104/read-protobuf-0.1.1.tar.gz +BuildArch: noarch + +Requires: python3-pandas +Requires: python3-protobuf + +%description +# read-protobuf + +Small library to read serialized protobuf(s) directly into Pandas Dataframe. + +This is meant to be a simple shortcut to getting from serialized protobuf bytes / files directly to a dataframe. + +>Note: This currently only supports basic proto3 features for Python 3. I have not yet tested it with proto2, though I believe that should work. I plan to expand the utility of this library with time and need. + +## Install + +Available via pip: + +```bash +$ pip install read-protobuf +``` + +## Usage + +Run the [demo-notebook](tests/demo.ipynb) for an interactive demo. + +```python +import demo_pb2 # compiled protobuf message module +from read_protobuf import read_protobuf + +MessageType = demo_pb2.MessageType() # instantiate a new message type +df = read_protobuf(b'\x00\x00', MessageType) # create a dataframe from serialized protobuf bytes +df = read_protobuf([b'\x00\x00', b'x00\x00'] MessageType) # read multiple protobuf bytes + +df = read_protobuf('demo.pb', MessageType) # use file instead of bytes +df = read_protobuf(['demo.pb', 'demo2.pb'], MessageType) # read multiple files + +# options +df = read_protobuf('demo.pb', MessageType, flatten=False) # don't flatten pb messages +df = read_protobuf('demo.pb', MessageType, prefix_nested=True) # prefix nested messages with parent keys (like pandas.io.json.json_normalize) +``` + + +To compile a protobuf Message class from python, use: + +```bash +$ protoc --python_out="." demo.proto +``` + +## Alternatives + +#### protobuf-to-dict + +https://github.com/benhodgson/protobuf-to-dict + +This library was developed earlier to convert protobufs to JSON via a dict. + +#### JSON + +The google protobuf library comes with a utility to convert messages to JSON. Then the JSON objects could be loaded into pands via `pd.read_json()`. + +```python +from google.protobuf.json_format import MessageToJson +``` + +In my brief tests, the `read_protobuf` package is about twice as fast as converting a protobuf to a dataframe using `MessageToJson`. + +## Develop + +Currently developed for Python 3 using the anaconda python distribution. To install a development version of the package, run from the root directory: + +```bash +$ pip install -e . +``` + +- To install development dependencies, use pip on the `dev-requirements.txt` file: + +```bash +$ pip install -r dev-requirements.txt +``` + +## Lint + +Uses `pylint` to lint application. + +``` +$ pylint read_protobuf +``` + +Configuration options are specified in `.pylintrc` + +## Test + +Uses `pytest` to run unit tests. From the root of the repository, run: + +``` +$ pytest +$ pytest -k "TestRead::test_read_bytes" # specify test +``` + +Configuration options are specified in `setup.cfg` + +## Code Coverage + +We use `coverage` to monitor code coverage during tests. To record coverage while running tests, run: + +```bash +$ coverage run -m pytest # watch files while testing +$ coverage report # will display coverage report +``` + + +## UnLicense + +This is free and unencumbered software released into the public domain. + +Anyone is free to copy, modify, publish, use, compile, sell, or +distribute this software, either in source code form or as a compiled +binary, for any purpose, commercial or non-commercial, and by any +means. + +In jurisdictions that recognize copyright laws, the author or authors +of this software dedicate any and all copyright interest in the +software to the public domain. We make this dedication for the benefit +of the public at large and to the detriment of our heirs and +successors. We intend this dedication to be an overt act of +relinquishment in perpetuity of all present and future rights to this +software under copyright law. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. +IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR +OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, +ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR +OTHER DEALINGS IN THE SOFTWARE. + +For more information, please refer to <http://unlicense.org/> + + + + + + +%package -n python3-read-protobuf +Summary: Small library to read serialized protobuf(s) directly into Pandas Dataframe +Provides: python-read-protobuf +BuildRequires: python3-devel +BuildRequires: python3-setuptools +BuildRequires: python3-pip +%description -n python3-read-protobuf +# read-protobuf + +Small library to read serialized protobuf(s) directly into Pandas Dataframe. + +This is meant to be a simple shortcut to getting from serialized protobuf bytes / files directly to a dataframe. + +>Note: This currently only supports basic proto3 features for Python 3. I have not yet tested it with proto2, though I believe that should work. I plan to expand the utility of this library with time and need. + +## Install + +Available via pip: + +```bash +$ pip install read-protobuf +``` + +## Usage + +Run the [demo-notebook](tests/demo.ipynb) for an interactive demo. + +```python +import demo_pb2 # compiled protobuf message module +from read_protobuf import read_protobuf + +MessageType = demo_pb2.MessageType() # instantiate a new message type +df = read_protobuf(b'\x00\x00', MessageType) # create a dataframe from serialized protobuf bytes +df = read_protobuf([b'\x00\x00', b'x00\x00'] MessageType) # read multiple protobuf bytes + +df = read_protobuf('demo.pb', MessageType) # use file instead of bytes +df = read_protobuf(['demo.pb', 'demo2.pb'], MessageType) # read multiple files + +# options +df = read_protobuf('demo.pb', MessageType, flatten=False) # don't flatten pb messages +df = read_protobuf('demo.pb', MessageType, prefix_nested=True) # prefix nested messages with parent keys (like pandas.io.json.json_normalize) +``` + + +To compile a protobuf Message class from python, use: + +```bash +$ protoc --python_out="." demo.proto +``` + +## Alternatives + +#### protobuf-to-dict + +https://github.com/benhodgson/protobuf-to-dict + +This library was developed earlier to convert protobufs to JSON via a dict. + +#### JSON + +The google protobuf library comes with a utility to convert messages to JSON. Then the JSON objects could be loaded into pands via `pd.read_json()`. + +```python +from google.protobuf.json_format import MessageToJson +``` + +In my brief tests, the `read_protobuf` package is about twice as fast as converting a protobuf to a dataframe using `MessageToJson`. + +## Develop + +Currently developed for Python 3 using the anaconda python distribution. To install a development version of the package, run from the root directory: + +```bash +$ pip install -e . +``` + +- To install development dependencies, use pip on the `dev-requirements.txt` file: + +```bash +$ pip install -r dev-requirements.txt +``` + +## Lint + +Uses `pylint` to lint application. + +``` +$ pylint read_protobuf +``` + +Configuration options are specified in `.pylintrc` + +## Test + +Uses `pytest` to run unit tests. From the root of the repository, run: + +``` +$ pytest +$ pytest -k "TestRead::test_read_bytes" # specify test +``` + +Configuration options are specified in `setup.cfg` + +## Code Coverage + +We use `coverage` to monitor code coverage during tests. To record coverage while running tests, run: + +```bash +$ coverage run -m pytest # watch files while testing +$ coverage report # will display coverage report +``` + + +## UnLicense + +This is free and unencumbered software released into the public domain. + +Anyone is free to copy, modify, publish, use, compile, sell, or +distribute this software, either in source code form or as a compiled +binary, for any purpose, commercial or non-commercial, and by any +means. + +In jurisdictions that recognize copyright laws, the author or authors +of this software dedicate any and all copyright interest in the +software to the public domain. We make this dedication for the benefit +of the public at large and to the detriment of our heirs and +successors. We intend this dedication to be an overt act of +relinquishment in perpetuity of all present and future rights to this +software under copyright law. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. +IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR +OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, +ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR +OTHER DEALINGS IN THE SOFTWARE. + +For more information, please refer to <http://unlicense.org/> + + + + + + +%package help +Summary: Development documents and examples for read-protobuf +Provides: python3-read-protobuf-doc +%description help +# read-protobuf + +Small library to read serialized protobuf(s) directly into Pandas Dataframe. + +This is meant to be a simple shortcut to getting from serialized protobuf bytes / files directly to a dataframe. + +>Note: This currently only supports basic proto3 features for Python 3. I have not yet tested it with proto2, though I believe that should work. I plan to expand the utility of this library with time and need. + +## Install + +Available via pip: + +```bash +$ pip install read-protobuf +``` + +## Usage + +Run the [demo-notebook](tests/demo.ipynb) for an interactive demo. + +```python +import demo_pb2 # compiled protobuf message module +from read_protobuf import read_protobuf + +MessageType = demo_pb2.MessageType() # instantiate a new message type +df = read_protobuf(b'\x00\x00', MessageType) # create a dataframe from serialized protobuf bytes +df = read_protobuf([b'\x00\x00', b'x00\x00'] MessageType) # read multiple protobuf bytes + +df = read_protobuf('demo.pb', MessageType) # use file instead of bytes +df = read_protobuf(['demo.pb', 'demo2.pb'], MessageType) # read multiple files + +# options +df = read_protobuf('demo.pb', MessageType, flatten=False) # don't flatten pb messages +df = read_protobuf('demo.pb', MessageType, prefix_nested=True) # prefix nested messages with parent keys (like pandas.io.json.json_normalize) +``` + + +To compile a protobuf Message class from python, use: + +```bash +$ protoc --python_out="." demo.proto +``` + +## Alternatives + +#### protobuf-to-dict + +https://github.com/benhodgson/protobuf-to-dict + +This library was developed earlier to convert protobufs to JSON via a dict. + +#### JSON + +The google protobuf library comes with a utility to convert messages to JSON. Then the JSON objects could be loaded into pands via `pd.read_json()`. + +```python +from google.protobuf.json_format import MessageToJson +``` + +In my brief tests, the `read_protobuf` package is about twice as fast as converting a protobuf to a dataframe using `MessageToJson`. + +## Develop + +Currently developed for Python 3 using the anaconda python distribution. To install a development version of the package, run from the root directory: + +```bash +$ pip install -e . +``` + +- To install development dependencies, use pip on the `dev-requirements.txt` file: + +```bash +$ pip install -r dev-requirements.txt +``` + +## Lint + +Uses `pylint` to lint application. + +``` +$ pylint read_protobuf +``` + +Configuration options are specified in `.pylintrc` + +## Test + +Uses `pytest` to run unit tests. From the root of the repository, run: + +``` +$ pytest +$ pytest -k "TestRead::test_read_bytes" # specify test +``` + +Configuration options are specified in `setup.cfg` + +## Code Coverage + +We use `coverage` to monitor code coverage during tests. To record coverage while running tests, run: + +```bash +$ coverage run -m pytest # watch files while testing +$ coverage report # will display coverage report +``` + + +## UnLicense + +This is free and unencumbered software released into the public domain. + +Anyone is free to copy, modify, publish, use, compile, sell, or +distribute this software, either in source code form or as a compiled +binary, for any purpose, commercial or non-commercial, and by any +means. + +In jurisdictions that recognize copyright laws, the author or authors +of this software dedicate any and all copyright interest in the +software to the public domain. We make this dedication for the benefit +of the public at large and to the detriment of our heirs and +successors. We intend this dedication to be an overt act of +relinquishment in perpetuity of all present and future rights to this +software under copyright law. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. +IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR +OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, +ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR +OTHER DEALINGS IN THE SOFTWARE. + +For more information, please refer to <http://unlicense.org/> + + + + + + +%prep +%autosetup -n read-protobuf-0.1.1 + +%build +%py3_build + +%install +%py3_install +install -d -m755 %{buildroot}/%{_pkgdocdir} +if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi +if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi +if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi +if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi +pushd %{buildroot} +if [ -d usr/lib ]; then + find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/lib64 ]; then + find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/bin ]; then + find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/sbin ]; then + find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst +fi +touch doclist.lst +if [ -d usr/share/man ]; then + find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst +fi +popd +mv %{buildroot}/filelist.lst . +mv %{buildroot}/doclist.lst . + +%files -n python3-read-protobuf -f filelist.lst +%dir %{python3_sitelib}/* + +%files help -f doclist.lst +%{_docdir}/* + +%changelog +* Tue Apr 11 2023 Python_Bot <Python_Bot@openeuler.org> - 0.1.1-1 +- Package Spec generated @@ -0,0 +1 @@ +305ec2d8334ec929e7af3e7ad39d2aec read-protobuf-0.1.1.tar.gz |