path: root/python-blackboxprotobuf.spec

%global _empty_manifest_terminate_build 0
Name:		python-blackboxprotobuf
Version:	1.0.1
Release:	1
Summary:	Library for reading protobuf buffers without .proto definitions
License:	MIT License
URL:		https://github.com/ydkhatri/blackboxprotobuf
Source0:	https://mirrors.nju.edu.cn/pypi/web/packages/a5/39/d6a88c629f525113072625a6f8a15b4c5d17c17b47da3a208c6b1a1b23ce/blackboxprotobuf-1.0.1.tar.gz
BuildArch:	noarch


%description
# BlackBox Protobuf Library

### _Note: This is a fork of the library found [here](https://github.com/nccgroup/blackboxprotobuf). This original was written for adding protobuf reading to burp, this version strips out all burp related code and dependencies, and works with python3._

## Description
Blackbox protobuf library is a Python module for decoding and re-encoding protobuf
messages without access to the source protobuf descriptor file. This library
provides a simple Python interface to encode/decode messages that can be
integrated into other tools.

This library is targeted towards use in DFIR investigations where being able to
read the content messages is critical and a protocol buffer definition may not be readily
available.

## Background
Protocol Buffers (protobufs)  are a standard published by Google with
accompanying libraries for binary serialization of data. Protocol buffers are
defined by a `.proto` file known to both the sender and the receiver. The actual
binary message does not contain information such as field names or most type
information.

For each field, the serialized protocol buffer includes two pieces of metadata,
a field number and the wire type. The wire type tells a parser how to parse the
length of the field, so that it can be skipped if it is not known (one protocol
buffer design goal is being able to handle messages with unknown fields). A
single wire-type generally encompasses multiple protocol buffer types, for
example the length delimited wire-type can be used for string, bytestring,
inner message or packed repeated fields. See
<https://developers.google.com/protocol-buffers/docs/encoding#structure> for
the breakdown of wire types.

The protocol buffer compiler (`protoc`) does support a similar method of
decoding protocol buffers without the definition with the `--decode_raw`
option. However, it does not provide any functionality to re-encode the decoded
message.

## How it works
The library makes a best effort guess of the type based on the provided wire type (and
occasionally field content) and builds a type definition that can be used to
re-encode the data. In general, most fields of interest are likely to be parsed
into a usable form. Users can optionally pass in custom type definitions that
override the guessed type. Custom type definitions also allow naming of fields to
improve user friendliness.

## Future Work
- Allow import and export of type definitions to protobuf definition files.

# Usage
## Installation    

```
pip install blackboxprotobuf
```

## Interface
The main `blackboxprotobuf` module defines five functions, the core
encoding/decoding functions, two convenience functions that encode/decode JSON
strings and a function to validate type definition changes.

### Decode 
Decoding functions takes a protobuf bytestring, and optionally
either a type definition or a known message name mapped to a type definition
(in `blackboxprotobuf.known_messages`). If a type definition isn't provided, an
empty message type is assumed and all types are derived from the protobuf
binary.

The decoder returns a tuple containing a dictionary with the decoded data and a
dictionary containing the generated type definition. If the input type
definition does not include types for all fields in the message, the output
type definitions will include type guesses for those fields.

Example use:
```python
import blackboxprotobuf
import base64

data = base64.b64decode('KglNb2RpZnkgTWU=')
message,typedef = blackboxprotobuf.protobuf_to_json(data)
print(message)
print(typedef)
```

### Encode
The encoding functions takes a Python dictionary containing the data and a type
definition. Unlike decoding, the type definition is required and will fail if
any fields are not defined. Generally, the type definition should be the output
from the decoding function or a modified version thereof.

Example use:
```python
import blackboxprotobuf
import base64

data = base64.b64decode('KglNb2RpZnkgTWU=')
message,typedef = blackboxprotobuf.decode_message(data)

message['5'] = 'Modified Me'

new_data = bytes(blackboxprotobuf.encode_message(message,typedef))
print(data)
print(new_data)
```

### Type definition structure
The type definition object is a Python dictionary representing the type
structure of a message, it includes a type for each field and optionally a
name. Each entry in the dictionary represents a field in the message. The key
should be the field number and the value is a dictionary containing attributes.

At the minimum the dictionary should contain the 'type' entry which contains a
string identifier for the type. Valid type identifiers can be found in
`blackboxprotobuf/lib/types/type_maps.py`.

Message fields will also contain one of two entries, 'message_typedef' or
'message_type_name'. 'message_typedef' should contain a second type definition
structure for the inner message. 'message_type_name' should contain the string
identifier for a message type previously stored in
`blackboxprotobuf.known_messages`. If both are specified, the 'message_type_name'
will be ignored.

## Type Breakdown
The following is a quick breakdown of wire types and default values. See
<https://developers.google.com/protocol-buffers/docs/encoding> for more detailed
information from Google.

### Variable Length Integers (varint)
The `varint` wire type represents integers with multiple bytes where one bit of
each is dedicated to indicating if it is the last byte. This can be used to
represent integers (signed/unsigned), boolean values or enums. Integers can be
encoded using three variations:

- `uint`: Varint encoding with no representation of negative numbers.
- `int`: Standard encoding but inefficient for negative numbers (always 10 bytes).
- `sint`: Uses ZigZag encoding to efficiently represent negative numbers by
  mapping negative numbers into the integer space. For example -1 is converted
  to 1, 1 to 2, -2 to 3, and so on. This can result in drastically different
  numbers if a type is misinterpreted and either the original or incorrect type
  is `sint`.

The default is currently `int` with no ZigZag encoding.

### Fixed32/64
The fixed length wire types have an implicit size based on the wire type. These
support either fixed size integers (signed/unsigned) or fixed size floating
point numbers (float/double). The default type for these is the floating point
type as most integers are more likely to be represented by a varint.

### Length Delimited
Length delimited wire types are prefixed with a `varint` indicating the length.
This is used for strings, bytestrings, inner messages and packed repeated
fields. Messages can generally be identified by validating if it is a valid
protobuf binary. If it is not a message, the default type is a string/byte
which are relatively interchangeable in Python.

Packed repeated fields are arrays of either `varints` or a fixed length wire
type. Non-packed repeated fields use a separate tag (wire type + field number)
for each element, allowing them to be easily identified and parsed. However,
packed repeated fields only have the initial length delimited wire type tag.
The parser is assumed to know the full type already for parsing out the
individual elements. This makes this field type difficult to differentiate from
an arbitrary byte string and will require user intervention to identify. In
protobuf version 2, repeated fields had to be explicitly declared packed in the
definition. In protobuf version 3, repeated fields are packed by default and
are likely to become more common.

%package -n python3-blackboxprotobuf
Summary:	Library for reading protobuf buffers without .proto definitions
Provides:	python-blackboxprotobuf
BuildRequires:	python3-devel
BuildRequires:	python3-setuptools
BuildRequires:	python3-pip
%description -n python3-blackboxprotobuf
# BlackBox Protobuf Library

### _Note: This is a fork of the library found [here](https://github.com/nccgroup/blackboxprotobuf). This original was written for adding protobuf reading to burp, this version strips out all burp related code and dependencies, and works with python3._

## Description
Blackbox protobuf library is a Python module for decoding and re-encoding protobuf
messages without access to the source protobuf descriptor file. This library
provides a simple Python interface to encode/decode messages that can be
integrated into other tools.

This library is targeted towards use in DFIR investigations where being able to
read the content messages is critical and a protocol buffer definition may not be readily
available.

## Background
Protocol Buffers (protobufs)  are a standard published by Google with
accompanying libraries for binary serialization of data. Protocol buffers are
defined by a `.proto` file known to both the sender and the receiver. The actual
binary message does not contain information such as field names or most type
information.

For each field, the serialized protocol buffer includes two pieces of metadata,
a field number and the wire type. The wire type tells a parser how to parse the
length of the field, so that it can be skipped if it is not known (one protocol
buffer design goal is being able to handle messages with unknown fields). A
single wire-type generally encompasses multiple protocol buffer types, for
example the length delimited wire-type can be used for string, bytestring,
inner message or packed repeated fields. See
<https://developers.google.com/protocol-buffers/docs/encoding#structure> for
the breakdown of wire types.

The protocol buffer compiler (`protoc`) does support a similar method of
decoding protocol buffers without the definition with the `--decode_raw`
option. However, it does not provide any functionality to re-encode the decoded
message.

## How it works
The library makes a best effort guess of the type based on the provided wire type (and
occasionally field content) and builds a type definition that can be used to
re-encode the data. In general, most fields of interest are likely to be parsed
into a usable form. Users can optionally pass in custom type definitions that
override the guessed type. Custom type definitions also allow naming of fields to
improve user friendliness.

## Future Work
- Allow import and export of type definitions to protobuf definition files.

# Usage
## Installation    

```
pip install blackboxprotobuf
```

## Interface
The main `blackboxprotobuf` module defines five functions, the core
encoding/decoding functions, two convenience functions that encode/decode JSON
strings and a function to validate type definition changes.

### Decode 
Decoding functions takes a protobuf bytestring, and optionally
either a type definition or a known message name mapped to a type definition
(in `blackboxprotobuf.known_messages`). If a type definition isn't provided, an
empty message type is assumed and all types are derived from the protobuf
binary.

The decoder returns a tuple containing a dictionary with the decoded data and a
dictionary containing the generated type definition. If the input type
definition does not include types for all fields in the message, the output
type definitions will include type guesses for those fields.

Example use:
```python
import blackboxprotobuf
import base64

data = base64.b64decode('KglNb2RpZnkgTWU=')
message,typedef = blackboxprotobuf.protobuf_to_json(data)
print(message)
print(typedef)
```

### Encode
The encoding functions takes a Python dictionary containing the data and a type
definition. Unlike decoding, the type definition is required and will fail if
any fields are not defined. Generally, the type definition should be the output
from the decoding function or a modified version thereof.

Example use:
```python
import blackboxprotobuf
import base64

data = base64.b64decode('KglNb2RpZnkgTWU=')
message,typedef = blackboxprotobuf.decode_message(data)

message['5'] = 'Modified Me'

new_data = bytes(blackboxprotobuf.encode_message(message,typedef))
print(data)
print(new_data)
```

### Type definition structure
The type definition object is a Python dictionary representing the type
structure of a message, it includes a type for each field and optionally a
name. Each entry in the dictionary represents a field in the message. The key
should be the field number and the value is a dictionary containing attributes.

At the minimum the dictionary should contain the 'type' entry which contains a
string identifier for the type. Valid type identifiers can be found in
`blackboxprotobuf/lib/types/type_maps.py`.

Message fields will also contain one of two entries, 'message_typedef' or
'message_type_name'. 'message_typedef' should contain a second type definition
structure for the inner message. 'message_type_name' should contain the string
identifier for a message type previously stored in
`blackboxprotobuf.known_messages`. If both are specified, the 'message_type_name'
will be ignored.

## Type Breakdown
The following is a quick breakdown of wire types and default values. See
<https://developers.google.com/protocol-buffers/docs/encoding> for more detailed
information from Google.

### Variable Length Integers (varint)
The `varint` wire type represents integers with multiple bytes where one bit of
each is dedicated to indicating if it is the last byte. This can be used to
represent integers (signed/unsigned), boolean values or enums. Integers can be
encoded using three variations:

- `uint`: Varint encoding with no representation of negative numbers.
- `int`: Standard encoding but inefficient for negative numbers (always 10 bytes).
- `sint`: Uses ZigZag encoding to efficiently represent negative numbers by
  mapping negative numbers into the integer space. For example -1 is converted
  to 1, 1 to 2, -2 to 3, and so on. This can result in drastically different
  numbers if a type is misinterpreted and either the original or incorrect type
  is `sint`.

The default is currently `int` with no ZigZag encoding.

### Fixed32/64
The fixed length wire types have an implicit size based on the wire type. These
support either fixed size integers (signed/unsigned) or fixed size floating
point numbers (float/double). The default type for these is the floating point
type as most integers are more likely to be represented by a varint.

### Length Delimited
Length delimited wire types are prefixed with a `varint` indicating the length.
This is used for strings, bytestrings, inner messages and packed repeated
fields. Messages can generally be identified by validating if it is a valid
protobuf binary. If it is not a message, the default type is a string/byte
which are relatively interchangeable in Python.

Packed repeated fields are arrays of either `varints` or a fixed length wire
type. Non-packed repeated fields use a separate tag (wire type + field number)
for each element, allowing them to be easily identified and parsed. However,
packed repeated fields only have the initial length delimited wire type tag.
The parser is assumed to know the full type already for parsing out the
individual elements. This makes this field type difficult to differentiate from
an arbitrary byte string and will require user intervention to identify. In
protobuf version 2, repeated fields had to be explicitly declared packed in the
definition. In protobuf version 3, repeated fields are packed by default and
are likely to become more common.

%package help
Summary:	Development documents and examples for blackboxprotobuf
Provides:	python3-blackboxprotobuf-doc
%description help
# BlackBox Protobuf Library

### _Note: This is a fork of the library found [here](https://github.com/nccgroup/blackboxprotobuf). This original was written for adding protobuf reading to burp, this version strips out all burp related code and dependencies, and works with python3._

## Description
Blackbox protobuf library is a Python module for decoding and re-encoding protobuf
messages without access to the source protobuf descriptor file. This library
provides a simple Python interface to encode/decode messages that can be
integrated into other tools.

This library is targeted towards use in DFIR investigations where being able to
read the content messages is critical and a protocol buffer definition may not be readily
available.

## Background
Protocol Buffers (protobufs)  are a standard published by Google with
accompanying libraries for binary serialization of data. Protocol buffers are
defined by a `.proto` file known to both the sender and the receiver. The actual
binary message does not contain information such as field names or most type
information.

For each field, the serialized protocol buffer includes two pieces of metadata,
a field number and the wire type. The wire type tells a parser how to parse the
length of the field, so that it can be skipped if it is not known (one protocol
buffer design goal is being able to handle messages with unknown fields). A
single wire-type generally encompasses multiple protocol buffer types, for
example the length delimited wire-type can be used for string, bytestring,
inner message or packed repeated fields. See
<https://developers.google.com/protocol-buffers/docs/encoding#structure> for
the breakdown of wire types.

The protocol buffer compiler (`protoc`) does support a similar method of
decoding protocol buffers without the definition with the `--decode_raw`
option. However, it does not provide any functionality to re-encode the decoded
message.

## How it works
The library makes a best effort guess of the type based on the provided wire type (and
occasionally field content) and builds a type definition that can be used to
re-encode the data. In general, most fields of interest are likely to be parsed
into a usable form. Users can optionally pass in custom type definitions that
override the guessed type. Custom type definitions also allow naming of fields to
improve user friendliness.

## Future Work
- Allow import and export of type definitions to protobuf definition files.

# Usage
## Installation    

```
pip install blackboxprotobuf
```

## Interface
The main `blackboxprotobuf` module defines five functions, the core
encoding/decoding functions, two convenience functions that encode/decode JSON
strings and a function to validate type definition changes.

### Decode 
Decoding functions takes a protobuf bytestring, and optionally
either a type definition or a known message name mapped to a type definition
(in `blackboxprotobuf.known_messages`). If a type definition isn't provided, an
empty message type is assumed and all types are derived from the protobuf
binary.

The decoder returns a tuple containing a dictionary with the decoded data and a
dictionary containing the generated type definition. If the input type
definition does not include types for all fields in the message, the output
type definitions will include type guesses for those fields.

Example use:
```python
import blackboxprotobuf
import base64

data = base64.b64decode('KglNb2RpZnkgTWU=')
message,typedef = blackboxprotobuf.protobuf_to_json(data)
print(message)
print(typedef)
```

### Encode
The encoding functions takes a Python dictionary containing the data and a type
definition. Unlike decoding, the type definition is required and will fail if
any fields are not defined. Generally, the type definition should be the output
from the decoding function or a modified version thereof.

Example use:
```python
import blackboxprotobuf
import base64

data = base64.b64decode('KglNb2RpZnkgTWU=')
message,typedef = blackboxprotobuf.decode_message(data)

message['5'] = 'Modified Me'

new_data = bytes(blackboxprotobuf.encode_message(message,typedef))
print(data)
print(new_data)
```

### Type definition structure
The type definition object is a Python dictionary representing the type
structure of a message, it includes a type for each field and optionally a
name. Each entry in the dictionary represents a field in the message. The key
should be the field number and the value is a dictionary containing attributes.

At the minimum the dictionary should contain the 'type' entry which contains a
string identifier for the type. Valid type identifiers can be found in
`blackboxprotobuf/lib/types/type_maps.py`.

Message fields will also contain one of two entries, 'message_typedef' or
'message_type_name'. 'message_typedef' should contain a second type definition
structure for the inner message. 'message_type_name' should contain the string
identifier for a message type previously stored in
`blackboxprotobuf.known_messages`. If both are specified, the 'message_type_name'
will be ignored.

## Type Breakdown
The following is a quick breakdown of wire types and default values. See
<https://developers.google.com/protocol-buffers/docs/encoding> for more detailed
information from Google.

### Variable Length Integers (varint)
The `varint` wire type represents integers with multiple bytes where one bit of
each is dedicated to indicating if it is the last byte. This can be used to
represent integers (signed/unsigned), boolean values or enums. Integers can be
encoded using three variations:

- `uint`: Varint encoding with no representation of negative numbers.
- `int`: Standard encoding but inefficient for negative numbers (always 10 bytes).
- `sint`: Uses ZigZag encoding to efficiently represent negative numbers by
  mapping negative numbers into the integer space. For example -1 is converted
  to 1, 1 to 2, -2 to 3, and so on. This can result in drastically different
  numbers if a type is misinterpreted and either the original or incorrect type
  is `sint`.

The default is currently `int` with no ZigZag encoding.

### Fixed32/64
The fixed length wire types have an implicit size based on the wire type. These
support either fixed size integers (signed/unsigned) or fixed size floating
point numbers (float/double). The default type for these is the floating point
type as most integers are more likely to be represented by a varint.

### Length Delimited
Length delimited wire types are prefixed with a `varint` indicating the length.
This is used for strings, bytestrings, inner messages and packed repeated
fields. Messages can generally be identified by validating if it is a valid
protobuf binary. If it is not a message, the default type is a string/byte
which are relatively interchangeable in Python.

Packed repeated fields are arrays of either `varints` or a fixed length wire
type. Non-packed repeated fields use a separate tag (wire type + field number)
for each element, allowing them to be easily identified and parsed. However,
packed repeated fields only have the initial length delimited wire type tag.
The parser is assumed to know the full type already for parsing out the
individual elements. This makes this field type difficult to differentiate from
an arbitrary byte string and will require user intervention to identify. In
protobuf version 2, repeated fields had to be explicitly declared packed in the
definition. In protobuf version 3, repeated fields are packed by default and
are likely to become more common.

%prep
%autosetup -n blackboxprotobuf-1.0.1

%build
%py3_build

%install
%py3_install
install -d -m755 %{buildroot}/%{_pkgdocdir}
if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi
if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi
if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi
if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi
pushd %{buildroot}
if [ -d usr/lib ]; then
	find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst
fi
if [ -d usr/lib64 ]; then
	find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst
fi
if [ -d usr/bin ]; then
	find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst
fi
if [ -d usr/sbin ]; then
	find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst
fi
touch doclist.lst
if [ -d usr/share/man ]; then
	find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst
fi
popd
mv %{buildroot}/filelist.lst .
mv %{buildroot}/doclist.lst .

%files -n python3-blackboxprotobuf -f filelist.lst
%dir %{python3_sitelib}/*

%files help -f doclist.lst
%{_docdir}/*

%changelog
* Wed May 31 2023 Python_Bot <Python_Bot@openeuler.org> - 1.0.1-1
- Package Spec generated