diff options
Diffstat (limited to 'python-syaz0.spec')
| -rw-r--r-- | python-syaz0.spec | 317 |
1 files changed, 317 insertions, 0 deletions
diff --git a/python-syaz0.spec b/python-syaz0.spec new file mode 100644 index 0000000..9a6f313 --- /dev/null +++ b/python-syaz0.spec @@ -0,0 +1,317 @@ +%global _empty_manifest_terminate_build 0 +Name: python-syaz0 +Version: 1.0.1 +Release: 1 +Summary: Library for data compression using Nintendo's Yaz0 algorithm +License: GNU General Public License v2 or later (GPLv2+) +URL: https://github.com/zeldamods/syaz0 +Source0: https://mirrors.nju.edu.cn/pypi/web/packages/e4/9f/6fcaf32537b4eb277dc24ccccabd2aaac2962772c845cfac64c953b6eaf7/syaz0-1.0.1.tar.gz + + +%description +## syaz0 +### Library for data compression using the Yaz0 algorithm + +*syaz0* is a native module for Python 3.6+ that provides fast data compression and decompression using Nintendo's [Yaz0](https://zeldamods.org/wiki/Yaz0) algorithm. + +## Performance + +Decompression performance is on par with existing Yaz0 decoders. + +As of late December 2019, syaz0 is able to compress files much faster than existing Yaz0 encoders. Files that are representative of *Breath of the Wild* assets were compressed 20x to 30x faster than with existing public tools for an equivalent or better compression ratio, and 70-80x faster (with a slightly worse ratio) in extreme cases. + +At the default compression level, file sizes are typically within 1% of Nintendo's. + +For detailed benchmarks, see the results files in the test directory. + +## Usage + +*syaz0* can be installed with `pip3 install syaz0`. Binary builds are provided for Windows 64 bits (only). On all other platforms, building from source is required. Skip to the end of the README for more information. + +### `syaz0.get_header(data)` + +Returns a syaz0.Header corresponding to the Yaz0 file header with the fields *magic*, *uncompressed_size*, *data_alignment*, *reserved*. + +### `syaz0.decompress(data)` + +Decompresses Yaz0-compressed data from a bytes-like object `data`. Returns a bytes object containing the uncompressed data. + +### `syaz0.decompress_unsafe(data)` + +Decompresses Yaz0-compressed data from a bytes (*not* bytes-like) object `data`. Returns a bytes object containing the uncompressed data. + +Unlike syaz0.decompress, this function assumes that the input data is well-formed. In exchange for slightly improved performance, no sanity checks are performed. **Warning**: Do not use on untrusted data. + +### `syaz0.compress(data, data_alignment=0, level=7)` + +Compresses a bytes-like object `data`. Returns a bytes-like object containing the Yaz0-compressed data. + +`data_alignment` is a hint for decoders to allocate buffers with the required data alignment. Defaults to 0, which indicates that no particular alignment is required. + +`level` is the compression level (6-9). 6 is fastest and 9 is slowest. Higher compression levels result in better compression. 7 is a good compromise between compression ratio and performance. + +## Project information + +*syaz0* was written with two goals in mind: to improve performance for Yaz0 compression — which is excruciatingly slow if one also desires decent compression ratios — and to let me learn a bit more about compression. + +After doing more research on compression algorithms and finding out +just how similar Yaz0 and LZ77-style compression algorithms are, +I tried to implement some common tricks to help improve the extremely +poor compression performance due to the sliding window search. + +But the implementation was still extremely slow compared to gzip. And probably badly implemented. + +It turns out that many more tricks were needed for fast compression. + +After stumbling upon [zlib-ng](https://github.com/zlib-ng/zlib-ng) and seeing how well-optimised it is +and all the intrinsics I decided it was best not to reinvent the wheel. +Thus *syaz0* uses a copy of zlib-ng for all the heavy lifting (match searching). The following modifications were made: + +* The window size was reduced to 4K (2^12) to match Yaz0. +* The compress function and the stream structures were changed to take + a callback that is invoked every time a distance/length pair or a + literal is emitted. (I'm not proud, but it works.) +* MAX_MATCH was *not* increased. zlib assumes it is equal to 258 in + too many places and increasing it actually gives worse compression ratios. + +### Building from source + +Building *syaz0* from source requires: + +* CMake 3.10+ +* A compiler that supports C++17 +* Everything needed to build [zlib-ng](https://github.com/zlib-ng/zlib-ng) +* pybind11 2.4+ (including CMake config files) +* setuptools + +When no binary build is available, pip will automatically build from source during the install process. + +To build from source manually, run `python3 setup.py bdist_wheel`. + +## License + +This software is licensed under the terms of the GNU General Public License, version 2 or later. + +%package -n python3-syaz0 +Summary: Library for data compression using Nintendo's Yaz0 algorithm +Provides: python-syaz0 +BuildRequires: python3-devel +BuildRequires: python3-setuptools +BuildRequires: python3-pip +BuildRequires: python3-cffi +BuildRequires: gcc +BuildRequires: gdb +%description -n python3-syaz0 +## syaz0 +### Library for data compression using the Yaz0 algorithm + +*syaz0* is a native module for Python 3.6+ that provides fast data compression and decompression using Nintendo's [Yaz0](https://zeldamods.org/wiki/Yaz0) algorithm. + +## Performance + +Decompression performance is on par with existing Yaz0 decoders. + +As of late December 2019, syaz0 is able to compress files much faster than existing Yaz0 encoders. Files that are representative of *Breath of the Wild* assets were compressed 20x to 30x faster than with existing public tools for an equivalent or better compression ratio, and 70-80x faster (with a slightly worse ratio) in extreme cases. + +At the default compression level, file sizes are typically within 1% of Nintendo's. + +For detailed benchmarks, see the results files in the test directory. + +## Usage + +*syaz0* can be installed with `pip3 install syaz0`. Binary builds are provided for Windows 64 bits (only). On all other platforms, building from source is required. Skip to the end of the README for more information. + +### `syaz0.get_header(data)` + +Returns a syaz0.Header corresponding to the Yaz0 file header with the fields *magic*, *uncompressed_size*, *data_alignment*, *reserved*. + +### `syaz0.decompress(data)` + +Decompresses Yaz0-compressed data from a bytes-like object `data`. Returns a bytes object containing the uncompressed data. + +### `syaz0.decompress_unsafe(data)` + +Decompresses Yaz0-compressed data from a bytes (*not* bytes-like) object `data`. Returns a bytes object containing the uncompressed data. + +Unlike syaz0.decompress, this function assumes that the input data is well-formed. In exchange for slightly improved performance, no sanity checks are performed. **Warning**: Do not use on untrusted data. + +### `syaz0.compress(data, data_alignment=0, level=7)` + +Compresses a bytes-like object `data`. Returns a bytes-like object containing the Yaz0-compressed data. + +`data_alignment` is a hint for decoders to allocate buffers with the required data alignment. Defaults to 0, which indicates that no particular alignment is required. + +`level` is the compression level (6-9). 6 is fastest and 9 is slowest. Higher compression levels result in better compression. 7 is a good compromise between compression ratio and performance. + +## Project information + +*syaz0* was written with two goals in mind: to improve performance for Yaz0 compression — which is excruciatingly slow if one also desires decent compression ratios — and to let me learn a bit more about compression. + +After doing more research on compression algorithms and finding out +just how similar Yaz0 and LZ77-style compression algorithms are, +I tried to implement some common tricks to help improve the extremely +poor compression performance due to the sliding window search. + +But the implementation was still extremely slow compared to gzip. And probably badly implemented. + +It turns out that many more tricks were needed for fast compression. + +After stumbling upon [zlib-ng](https://github.com/zlib-ng/zlib-ng) and seeing how well-optimised it is +and all the intrinsics I decided it was best not to reinvent the wheel. +Thus *syaz0* uses a copy of zlib-ng for all the heavy lifting (match searching). The following modifications were made: + +* The window size was reduced to 4K (2^12) to match Yaz0. +* The compress function and the stream structures were changed to take + a callback that is invoked every time a distance/length pair or a + literal is emitted. (I'm not proud, but it works.) +* MAX_MATCH was *not* increased. zlib assumes it is equal to 258 in + too many places and increasing it actually gives worse compression ratios. + +### Building from source + +Building *syaz0* from source requires: + +* CMake 3.10+ +* A compiler that supports C++17 +* Everything needed to build [zlib-ng](https://github.com/zlib-ng/zlib-ng) +* pybind11 2.4+ (including CMake config files) +* setuptools + +When no binary build is available, pip will automatically build from source during the install process. + +To build from source manually, run `python3 setup.py bdist_wheel`. + +## License + +This software is licensed under the terms of the GNU General Public License, version 2 or later. + +%package help +Summary: Development documents and examples for syaz0 +Provides: python3-syaz0-doc +%description help +## syaz0 +### Library for data compression using the Yaz0 algorithm + +*syaz0* is a native module for Python 3.6+ that provides fast data compression and decompression using Nintendo's [Yaz0](https://zeldamods.org/wiki/Yaz0) algorithm. + +## Performance + +Decompression performance is on par with existing Yaz0 decoders. + +As of late December 2019, syaz0 is able to compress files much faster than existing Yaz0 encoders. Files that are representative of *Breath of the Wild* assets were compressed 20x to 30x faster than with existing public tools for an equivalent or better compression ratio, and 70-80x faster (with a slightly worse ratio) in extreme cases. + +At the default compression level, file sizes are typically within 1% of Nintendo's. + +For detailed benchmarks, see the results files in the test directory. + +## Usage + +*syaz0* can be installed with `pip3 install syaz0`. Binary builds are provided for Windows 64 bits (only). On all other platforms, building from source is required. Skip to the end of the README for more information. + +### `syaz0.get_header(data)` + +Returns a syaz0.Header corresponding to the Yaz0 file header with the fields *magic*, *uncompressed_size*, *data_alignment*, *reserved*. + +### `syaz0.decompress(data)` + +Decompresses Yaz0-compressed data from a bytes-like object `data`. Returns a bytes object containing the uncompressed data. + +### `syaz0.decompress_unsafe(data)` + +Decompresses Yaz0-compressed data from a bytes (*not* bytes-like) object `data`. Returns a bytes object containing the uncompressed data. + +Unlike syaz0.decompress, this function assumes that the input data is well-formed. In exchange for slightly improved performance, no sanity checks are performed. **Warning**: Do not use on untrusted data. + +### `syaz0.compress(data, data_alignment=0, level=7)` + +Compresses a bytes-like object `data`. Returns a bytes-like object containing the Yaz0-compressed data. + +`data_alignment` is a hint for decoders to allocate buffers with the required data alignment. Defaults to 0, which indicates that no particular alignment is required. + +`level` is the compression level (6-9). 6 is fastest and 9 is slowest. Higher compression levels result in better compression. 7 is a good compromise between compression ratio and performance. + +## Project information + +*syaz0* was written with two goals in mind: to improve performance for Yaz0 compression — which is excruciatingly slow if one also desires decent compression ratios — and to let me learn a bit more about compression. + +After doing more research on compression algorithms and finding out +just how similar Yaz0 and LZ77-style compression algorithms are, +I tried to implement some common tricks to help improve the extremely +poor compression performance due to the sliding window search. + +But the implementation was still extremely slow compared to gzip. And probably badly implemented. + +It turns out that many more tricks were needed for fast compression. + +After stumbling upon [zlib-ng](https://github.com/zlib-ng/zlib-ng) and seeing how well-optimised it is +and all the intrinsics I decided it was best not to reinvent the wheel. +Thus *syaz0* uses a copy of zlib-ng for all the heavy lifting (match searching). The following modifications were made: + +* The window size was reduced to 4K (2^12) to match Yaz0. +* The compress function and the stream structures were changed to take + a callback that is invoked every time a distance/length pair or a + literal is emitted. (I'm not proud, but it works.) +* MAX_MATCH was *not* increased. zlib assumes it is equal to 258 in + too many places and increasing it actually gives worse compression ratios. + +### Building from source + +Building *syaz0* from source requires: + +* CMake 3.10+ +* A compiler that supports C++17 +* Everything needed to build [zlib-ng](https://github.com/zlib-ng/zlib-ng) +* pybind11 2.4+ (including CMake config files) +* setuptools + +When no binary build is available, pip will automatically build from source during the install process. + +To build from source manually, run `python3 setup.py bdist_wheel`. + +## License + +This software is licensed under the terms of the GNU General Public License, version 2 or later. + +%prep +%autosetup -n syaz0-1.0.1 + +%build +%py3_build + +%install +%py3_install +install -d -m755 %{buildroot}/%{_pkgdocdir} +if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi +if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi +if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi +if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi +pushd %{buildroot} +if [ -d usr/lib ]; then + find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/lib64 ]; then + find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/bin ]; then + find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/sbin ]; then + find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst +fi +touch doclist.lst +if [ -d usr/share/man ]; then + find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst +fi +popd +mv %{buildroot}/filelist.lst . +mv %{buildroot}/doclist.lst . + +%files -n python3-syaz0 -f filelist.lst +%dir %{python3_sitearch}/* + +%files help -f doclist.lst +%{_docdir}/* + +%changelog +* Wed May 31 2023 Python_Bot <Python_Bot@openeuler.org> - 1.0.1-1 +- Package Spec generated |