automatic import of python-blackboxprotobuf

3 files changed, 563 insertions, 0 deletions

diff --git a/.gitignore b/.gitignore
index e69de29..ffe27ff 100644
--- a/.gitignore
+++ b/.gitignore
@@ -0,0 +1 @@
+/blackboxprotobuf-1.0.1.tar.gz
diff --git a/python-blackboxprotobuf.spec b/python-blackboxprotobuf.spec
new file mode 100644
index 0000000..3a8bd7a
--- /dev/null
+++ b/python-blackboxprotobuf.spec
@@ -0,0 +1,561 @@
+%global _empty_manifest_terminate_build 0
+Name:		python-blackboxprotobuf
+Version:	1.0.1
+Release:	1
+Summary:	Library for reading protobuf buffers without .proto definitions
+License:	MIT License
+URL:		https://github.com/ydkhatri/blackboxprotobuf
+Source0:	https://mirrors.nju.edu.cn/pypi/web/packages/a5/39/d6a88c629f525113072625a6f8a15b4c5d17c17b47da3a208c6b1a1b23ce/blackboxprotobuf-1.0.1.tar.gz
+BuildArch:	noarch
+
+
+%description
+# BlackBox Protobuf Library
+
+### _Note: This is a fork of the library found [here](https://github.com/nccgroup/blackboxprotobuf). This original was written for adding protobuf reading to burp, this version strips out all burp related code and dependencies, and works with python3._
+
+## Description
+Blackbox protobuf library is a Python module for decoding and re-encoding protobuf
+messages without access to the source protobuf descriptor file. This library
+provides a simple Python interface to encode/decode messages that can be
+integrated into other tools.
+
+This library is targeted towards use in DFIR investigations where being able to
+read the content messages is critical and a protocol buffer definition may not be readily
+available.
+
+## Background
+Protocol Buffers (protobufs)  are a standard published by Google with
+accompanying libraries for binary serialization of data. Protocol buffers are
+defined by a `.proto` file known to both the sender and the receiver. The actual
+binary message does not contain information such as field names or most type
+information.
+
+For each field, the serialized protocol buffer includes two pieces of metadata,
+a field number and the wire type. The wire type tells a parser how to parse the
+length of the field, so that it can be skipped if it is not known (one protocol
+buffer design goal is being able to handle messages with unknown fields). A
+single wire-type generally encompasses multiple protocol buffer types, for
+example the length delimited wire-type can be used for string, bytestring,
+inner message or packed repeated fields. See
+<https://developers.google.com/protocol-buffers/docs/encoding#structure> for
+the breakdown of wire types.
+
+The protocol buffer compiler (`protoc`) does support a similar method of
+decoding protocol buffers without the definition with the `--decode_raw`
+option. However, it does not provide any functionality to re-encode the decoded
+message.
+
+## How it works
+The library makes a best effort guess of the type based on the provided wire type (and
+occasionally field content) and builds a type definition that can be used to
+re-encode the data. In general, most fields of interest are likely to be parsed
+into a usable form. Users can optionally pass in custom type definitions that
+override the guessed type. Custom type definitions also allow naming of fields to
+improve user friendliness.
+
+## Future Work
+- Allow import and export of type definitions to protobuf definition files.
+
+# Usage
+## Installation    
+
+```
+pip install blackboxprotobuf
+```
+
+## Interface
+The main `blackboxprotobuf` module defines five functions, the core
+encoding/decoding functions, two convenience functions that encode/decode JSON
+strings and a function to validate type definition changes.
+
+### Decode 
+Decoding functions takes a protobuf bytestring, and optionally
+either a type definition or a known message name mapped to a type definition
+(in `blackboxprotobuf.known_messages`). If a type definition isn't provided, an
+empty message type is assumed and all types are derived from the protobuf
+binary.
+
+The decoder returns a tuple containing a dictionary with the decoded data and a
+dictionary containing the generated type definition. If the input type
+definition does not include types for all fields in the message, the output
+type definitions will include type guesses for those fields.
+
+Example use:
+```python
+import blackboxprotobuf
+import base64
+
+data = base64.b64decode('KglNb2RpZnkgTWU=')
+message,typedef = blackboxprotobuf.protobuf_to_json(data)
+print(message)
+print(typedef)
+```
+
+### Encode
+The encoding functions takes a Python dictionary containing the data and a type
+definition. Unlike decoding, the type definition is required and will fail if
+any fields are not defined. Generally, the type definition should be the output
+from the decoding function or a modified version thereof.
+
+Example use:
+```python
+import blackboxprotobuf
+import base64
+
+data = base64.b64decode('KglNb2RpZnkgTWU=')
+message,typedef = blackboxprotobuf.decode_message(data)
+
+message['5'] = 'Modified Me'
+
+new_data = bytes(blackboxprotobuf.encode_message(message,typedef))
+print(data)
+print(new_data)
+```
+
+### Type definition structure
+The type definition object is a Python dictionary representing the type
+structure of a message, it includes a type for each field and optionally a
+name. Each entry in the dictionary represents a field in the message. The key
+should be the field number and the value is a dictionary containing attributes.
+
+At the minimum the dictionary should contain the 'type' entry which contains a
+string identifier for the type. Valid type identifiers can be found in
+`blackboxprotobuf/lib/types/type_maps.py`.
+
+Message fields will also contain one of two entries, 'message_typedef' or
+'message_type_name'. 'message_typedef' should contain a second type definition
+structure for the inner message. 'message_type_name' should contain the string
+identifier for a message type previously stored in
+`blackboxprotobuf.known_messages`. If both are specified, the 'message_type_name'
+will be ignored.
+
+## Type Breakdown
+The following is a quick breakdown of wire types and default values. See
+<https://developers.google.com/protocol-buffers/docs/encoding> for more detailed
+information from Google.
+
+### Variable Length Integers (varint)
+The `varint` wire type represents integers with multiple bytes where one bit of
+each is dedicated to indicating if it is the last byte. This can be used to
+represent integers (signed/unsigned), boolean values or enums. Integers can be
+encoded using three variations:
+
+- `uint`: Varint encoding with no representation of negative numbers.
+- `int`: Standard encoding but inefficient for negative numbers (always 10 bytes).
+- `sint`: Uses ZigZag encoding to efficiently represent negative numbers by
+  mapping negative numbers into the integer space. For example -1 is converted
+  to 1, 1 to 2, -2 to 3, and so on. This can result in drastically different
+  numbers if a type is misinterpreted and either the original or incorrect type
+  is `sint`.
+
+The default is currently `int` with no ZigZag encoding.
+
+### Fixed32/64
+The fixed length wire types have an implicit size based on the wire type. These
+support either fixed size integers (signed/unsigned) or fixed size floating
+point numbers (float/double). The default type for these is the floating point
+type as most integers are more likely to be represented by a varint.
+
+### Length Delimited
+Length delimited wire types are prefixed with a `varint` indicating the length.
+This is used for strings, bytestrings, inner messages and packed repeated
+fields. Messages can generally be identified by validating if it is a valid
+protobuf binary. If it is not a message, the default type is a string/byte
+which are relatively interchangeable in Python.
+
+Packed repeated fields are arrays of either `varints` or a fixed length wire
+type. Non-packed repeated fields use a separate tag (wire type + field number)
+for each element, allowing them to be easily identified and parsed. However,
+packed repeated fields only have the initial length delimited wire type tag.
+The parser is assumed to know the full type already for parsing out the
+individual elements. This makes this field type difficult to differentiate from
+an arbitrary byte string and will require user intervention to identify. In
+protobuf version 2, repeated fields had to be explicitly declared packed in the
+definition. In protobuf version 3, repeated fields are packed by default and
+are likely to become more common.
+
+%package -n python3-blackboxprotobuf
+Summary:	Library for reading protobuf buffers without .proto definitions
+Provides:	python-blackboxprotobuf
+BuildRequires:	python3-devel
+BuildRequires:	python3-setuptools
+BuildRequires:	python3-pip
+%description -n python3-blackboxprotobuf
+# BlackBox Protobuf Library
+
+### _Note: This is a fork of the library found [here](https://github.com/nccgroup/blackboxprotobuf). This original was written for adding protobuf reading to burp, this version strips out all burp related code and dependencies, and works with python3._
+
+## Description
+Blackbox protobuf library is a Python module for decoding and re-encoding protobuf
+messages without access to the source protobuf descriptor file. This library
+provides a simple Python interface to encode/decode messages that can be
+integrated into other tools.
+
+This library is targeted towards use in DFIR investigations where being able to
+read the content messages is critical and a protocol buffer definition may not be readily
+available.
+
+## Background
+Protocol Buffers (protobufs)  are a standard published by Google with
+accompanying libraries for binary serialization of data. Protocol buffers are
+defined by a `.proto` file known to both the sender and the receiver. The actual
+binary message does not contain information such as field names or most type
+information.
+
+For each field, the serialized protocol buffer includes two pieces of metadata,
+a field number and the wire type. The wire type tells a parser how to parse the
+length of the field, so that it can be skipped if it is not known (one protocol
+buffer design goal is being able to handle messages with unknown fields). A
+single wire-type generally encompasses multiple protocol buffer types, for
+example the length delimited wire-type can be used for string, bytestring,
+inner message or packed repeated fields. See
+<https://developers.google.com/protocol-buffers/docs/encoding#structure> for
+the breakdown of wire types.
+
+The protocol buffer compiler (`protoc`) does support a similar method of
+decoding protocol buffers without the definition with the `--decode_raw`
+option. However, it does not provide any functionality to re-encode the decoded
+message.
+
+## How it works
+The library makes a best effort guess of the type based on the provided wire type (and
+occasionally field content) and builds a type definition that can be used to
+re-encode the data. In general, most fields of interest are likely to be parsed
+into a usable form. Users can optionally pass in custom type definitions that
+override the guessed type. Custom type definitions also allow naming of fields to
+improve user friendliness.
+
+## Future Work
+- Allow import and export of type definitions to protobuf definition files.
+
+# Usage
+## Installation    
+
+```
+pip install blackboxprotobuf
+```
+
+## Interface
+The main `blackboxprotobuf` module defines five functions, the core
+encoding/decoding functions, two convenience functions that encode/decode JSON
+strings and a function to validate type definition changes.
+
+### Decode 
+Decoding functions takes a protobuf bytestring, and optionally
+either a type definition or a known message name mapped to a type definition
+(in `blackboxprotobuf.known_messages`). If a type definition isn't provided, an
+empty message type is assumed and all types are derived from the protobuf
+binary.
+
+The decoder returns a tuple containing a dictionary with the decoded data and a
+dictionary containing the generated type definition. If the input type
+definition does not include types for all fields in the message, the output
+type definitions will include type guesses for those fields.
+
+Example use:
+```python
+import blackboxprotobuf
+import base64
+
+data = base64.b64decode('KglNb2RpZnkgTWU=')
+message,typedef = blackboxprotobuf.protobuf_to_json(data)
+print(message)
+print(typedef)
+```
+
+### Encode
+The encoding functions takes a Python dictionary containing the data and a type
+definition. Unlike decoding, the type definition is required and will fail if
+any fields are not defined. Generally, the type definition should be the output
+from the decoding function or a modified version thereof.
+
+Example use:
+```python
+import blackboxprotobuf
+import base64
+
+data = base64.b64decode('KglNb2RpZnkgTWU=')
+message,typedef = blackboxprotobuf.decode_message(data)
+
+message['5'] = 'Modified Me'
+
+new_data = bytes(blackboxprotobuf.encode_message(message,typedef))
+print(data)
+print(new_data)
+```
+
+### Type definition structure
+The type definition object is a Python dictionary representing the type
+structure of a message, it includes a type for each field and optionally a
+name. Each entry in the dictionary represents a field in the message. The key
+should be the field number and the value is a dictionary containing attributes.
+
+At the minimum the dictionary should contain the 'type' entry which contains a
+string identifier for the type. Valid type identifiers can be found in
+`blackboxprotobuf/lib/types/type_maps.py`.
+
+Message fields will also contain one of two entries, 'message_typedef' or
+'message_type_name'. 'message_typedef' should contain a second type definition
+structure for the inner message. 'message_type_name' should contain the string
+identifier for a message type previously stored in
+`blackboxprotobuf.known_messages`. If both are specified, the 'message_type_name'
+will be ignored.
+
+## Type Breakdown
+The following is a quick breakdown of wire types and default values. See
+<https://developers.google.com/protocol-buffers/docs/encoding> for more detailed
+information from Google.
+
+### Variable Length Integers (varint)
+The `varint` wire type represents integers with multiple bytes where one bit of
+each is dedicated to indicating if it is the last byte. This can be used to
+represent integers (signed/unsigned), boolean values or enums. Integers can be
+encoded using three variations:
+
+- `uint`: Varint encoding with no representation of negative numbers.
+- `int`: Standard encoding but inefficient for negative numbers (always 10 bytes).
+- `sint`: Uses ZigZag encoding to efficiently represent negative numbers by
+  mapping negative numbers into the integer space. For example -1 is converted
+  to 1, 1 to 2, -2 to 3, and so on. This can result in drastically different
+  numbers if a type is misinterpreted and either the original or incorrect type
+  is `sint`.
+
+The default is currently `int` with no ZigZag encoding.
+
+### Fixed32/64
+The fixed length wire types have an implicit size based on the wire type. These
+support either fixed size integers (signed/unsigned) or fixed size floating
+point numbers (float/double). The default type for these is the floating point
+type as most integers are more likely to be represented by a varint.
+
+### Length Delimited
+Length delimited wire types are prefixed with a `varint` indicating the length.
+This is used for strings, bytestrings, inner messages and packed repeated
+fields. Messages can generally be identified by validating if it is a valid
+protobuf binary. If it is not a message, the default type is a string/byte
+which are relatively interchangeable in Python.
+
+Packed repeated fields are arrays of either `varints` or a fixed length wire
+type. Non-packed repeated fields use a separate tag (wire type + field number)
+for each element, allowing them to be easily identified and parsed. However,
+packed repeated fields only have the initial length delimited wire type tag.
+The parser is assumed to know the full type already for parsing out the
+individual elements. This makes this field type difficult to differentiate from
+an arbitrary byte string and will require user intervention to identify. In
+protobuf version 2, repeated fields had to be explicitly declared packed in the
+definition. In protobuf version 3, repeated fields are packed by default and
+are likely to become more common.
+
+%package help
+Summary:	Development documents and examples for blackboxprotobuf
+Provides:	python3-blackboxprotobuf-doc
+%description help
+# BlackBox Protobuf Library
+
+### _Note: This is a fork of the library found [here](https://github.com/nccgroup/blackboxprotobuf). This original was written for adding protobuf reading to burp, this version strips out all burp related code and dependencies, and works with python3._
+
+## Description
+Blackbox protobuf library is a Python module for decoding and re-encoding protobuf
+messages without access to the source protobuf descriptor file. This library
+provides a simple Python interface to encode/decode messages that can be
+integrated into other tools.
+
+This library is targeted towards use in DFIR investigations where being able to
+read the content messages is critical and a protocol buffer definition may not be readily
+available.
+
+## Background
+Protocol Buffers (protobufs)  are a standard published by Google with
+accompanying libraries for binary serialization of data. Protocol buffers are
+defined by a `.proto` file known to both the sender and the receiver. The actual
+binary message does not contain information such as field names or most type
+information.
+
+For each field, the serialized protocol buffer includes two pieces of metadata,
+a field number and the wire type. The wire type tells a parser how to parse the
+length of the field, so that it can be skipped if it is not known (one protocol
+buffer design goal is being able to handle messages with unknown fields). A
+single wire-type generally encompasses multiple protocol buffer types, for
+example the length delimited wire-type can be used for string, bytestring,
+inner message or packed repeated fields. See
+<https://developers.google.com/protocol-buffers/docs/encoding#structure> for
+the breakdown of wire types.
+
+The protocol buffer compiler (`protoc`) does support a similar method of
+decoding protocol buffers without the definition with the `--decode_raw`
+option. However, it does not provide any functionality to re-encode the decoded
+message.
+
+## How it works
+The library makes a best effort guess of the type based on the provided wire type (and
+occasionally field content) and builds a type definition that can be used to
+re-encode the data. In general, most fields of interest are likely to be parsed
+into a usable form. Users can optionally pass in custom type definitions that
+override the guessed type. Custom type definitions also allow naming of fields to
+improve user friendliness.
+
+## Future Work
+- Allow import and export of type definitions to protobuf definition files.
+
+# Usage
+## Installation    
+
+```
+pip install blackboxprotobuf
+```
+
+## Interface
+The main `blackboxprotobuf` module defines five functions, the core
+encoding/decoding functions, two convenience functions that encode/decode JSON
+strings and a function to validate type definition changes.
+
+### Decode 
+Decoding functions takes a protobuf bytestring, and optionally
+either a type definition or a known message name mapped to a type definition
+(in `blackboxprotobuf.known_messages`). If a type definition isn't provided, an
+empty message type is assumed and all types are derived from the protobuf
+binary.
+
+The decoder returns a tuple containing a dictionary with the decoded data and a
+dictionary containing the generated type definition. If the input type
+definition does not include types for all fields in the message, the output
+type definitions will include type guesses for those fields.
+
+Example use:
+```python
+import blackboxprotobuf
+import base64
+
+data = base64.b64decode('KglNb2RpZnkgTWU=')
+message,typedef = blackboxprotobuf.protobuf_to_json(data)
+print(message)
+print(typedef)
+```
+
+### Encode
+The encoding functions takes a Python dictionary containing the data and a type
+definition. Unlike decoding, the type definition is required and will fail if
+any fields are not defined. Generally, the type definition should be the output
+from the decoding function or a modified version thereof.
+
+Example use:
+```python
+import blackboxprotobuf
+import base64
+
+data = base64.b64decode('KglNb2RpZnkgTWU=')
+message,typedef = blackboxprotobuf.decode_message(data)
+
+message['5'] = 'Modified Me'
+
+new_data = bytes(blackboxprotobuf.encode_message(message,typedef))
+print(data)
+print(new_data)
+```
+
+### Type definition structure
+The type definition object is a Python dictionary representing the type
+structure of a message, it includes a type for each field and optionally a
+name. Each entry in the dictionary represents a field in the message. The key
+should be the field number and the value is a dictionary containing attributes.
+
+At the minimum the dictionary should contain the 'type' entry which contains a
+string identifier for the type. Valid type identifiers can be found in
+`blackboxprotobuf/lib/types/type_maps.py`.
+
+Message fields will also contain one of two entries, 'message_typedef' or
+'message_type_name'. 'message_typedef' should contain a second type definition
+structure for the inner message. 'message_type_name' should contain the string
+identifier for a message type previously stored in
+`blackboxprotobuf.known_messages`. If both are specified, the 'message_type_name'
+will be ignored.
+
+## Type Breakdown
+The following is a quick breakdown of wire types and default values. See
+<https://developers.google.com/protocol-buffers/docs/encoding> for more detailed
+information from Google.
+
+### Variable Length Integers (varint)
+The `varint` wire type represents integers with multiple bytes where one bit of
+each is dedicated to indicating if it is the last byte. This can be used to
+represent integers (signed/unsigned), boolean values or enums. Integers can be
+encoded using three variations:
+
+- `uint`: Varint encoding with no representation of negative numbers.
+- `int`: Standard encoding but inefficient for negative numbers (always 10 bytes).
+- `sint`: Uses ZigZag encoding to efficiently represent negative numbers by
+  mapping negative numbers into the integer space. For example -1 is converted
+  to 1, 1 to 2, -2 to 3, and so on. This can result in drastically different
+  numbers if a type is misinterpreted and either the original or incorrect type
+  is `sint`.
+
+The default is currently `int` with no ZigZag encoding.
+
+### Fixed32/64
+The fixed length wire types have an implicit size based on the wire type. These
+support either fixed size integers (signed/unsigned) or fixed size floating
+point numbers (float/double). The default type for these is the floating point
+type as most integers are more likely to be represented by a varint.
+
+### Length Delimited
+Length delimited wire types are prefixed with a `varint` indicating the length.
+This is used for strings, bytestrings, inner messages and packed repeated
+fields. Messages can generally be identified by validating if it is a valid
+protobuf binary. If it is not a message, the default type is a string/byte
+which are relatively interchangeable in Python.
+
+Packed repeated fields are arrays of either `varints` or a fixed length wire
+type. Non-packed repeated fields use a separate tag (wire type + field number)
+for each element, allowing them to be easily identified and parsed. However,
+packed repeated fields only have the initial length delimited wire type tag.
+The parser is assumed to know the full type already for parsing out the
+individual elements. This makes this field type difficult to differentiate from
+an arbitrary byte string and will require user intervention to identify. In
+protobuf version 2, repeated fields had to be explicitly declared packed in the
+definition. In protobuf version 3, repeated fields are packed by default and
+are likely to become more common.
+
+%prep
+%autosetup -n blackboxprotobuf-1.0.1
+
+%build
+%py3_build
+
+%install
+%py3_install
+install -d -m755 %{buildroot}/%{_pkgdocdir}
+if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi
+if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi
+if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi
+if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi
+pushd %{buildroot}
+if [ -d usr/lib ]; then
+	find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/lib64 ]; then
+	find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/bin ]; then
+	find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/sbin ]; then
+	find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+touch doclist.lst
+if [ -d usr/share/man ]; then
+	find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst
+fi
+popd
+mv %{buildroot}/filelist.lst .
+mv %{buildroot}/doclist.lst .
+
+%files -n python3-blackboxprotobuf -f filelist.lst
+%dir %{python3_sitelib}/*
+
+%files help -f doclist.lst
+%{_docdir}/*
+
+%changelog
+* Mon May 29 2023 Python_Bot <Python_Bot@openeuler.org> - 1.0.1-1
+- Package Spec generated
diff --git a/sources b/sources
new file mode 100644
index 0000000..2f8b7dc
--- /dev/null
+++ b/sources
@@ -0,0 +1 @@
+7bf754c4143ffbc8f69bca8924fe289f  blackboxprotobuf-1.0.1.tar.gz