path: root/python-lightseq.spec

%global _empty_manifest_terminate_build 0
Name:		python-lightseq
Version:	3.0.1
Release:	1
Summary:	LightSeq is a high performance library for sequence processing and generation
License:	Apache Software License
URL:		https://github.com/bytedance/lightseq
Source0:	https://mirrors.nju.edu.cn/pypi/web/packages/92/c3/ca4ed0027fb97a4fb6f0cf30010f7e0111bf688975c97daee297d1de0e51/lightseq-3.0.1.tar.gz
BuildArch:	noarch

Requires:	python3-ninja
Requires:	python3-numpy
Requires:	python3-scipy

%description
LightSeq is a high performance training and inference library for sequence processing and generation implemented
in CUDA.
It enables highly efficient computation of modern NLP models such as **BERT**, **GPT**,
**Transformer**, etc.
It is therefore best useful for *Machine Translation*, *Text Generation*, *Dialog*, *Language
Modelling*, *Sentiment Analysis*, and other related tasks with sequence data.
The library is built on top of CUDA official
library([cuBLAS](https://docs.nvidia.com/cuda/cublas/index.html),
[Thrust](https://docs.nvidia.com/cuda/thrust/index.html), [CUB](http://nvlabs.github.io/cub/)) and
custom kernel functions which are specially fused and optimized for Transformer model family. In
addition to model components, the inference library also provide easy-to deploy model management and serving backend based on
[TensorRT Inference
Server](https://docs.nvidia.com/deeplearning/sdk/inference-server-archived/tensorrt_inference_server_120/tensorrt-inference-server-guide/docs/quickstart.html).
With LightSeq, one can easily develop modified Transformer architecture with little additional code.
## Features
### [>>> Training](./lightseq/training)
The following is a support matrix of LightSeq **training** library compared with
[DeepSpeed](https://github.com/microsoft/DeepSpeed).
![features](./docs/training/images/features.png)
### [>>> Inference](./lightseq/inference)
The following is a support matrix of LightSeq **inference** library compared with
[TurboTransformers](https://github.com/Tencent/TurboTransformers) and
[FasterTransformer](https://github.com/NVIDIA/DeepLearningExamples/tree/master/FasterTransformer).
![support](./docs/inference/images/support.png)
## Performance
### [>>> Training](./lightseq/training)
Here we present the experimental results on WMT14 English to German translation task based on Transformer-big models. We train Transformer models of different sizes on eight NVIDIA Tesla V100/NVIDIA Tesla A100 GPUs with data parallel and fp16 mixed precision.
[Fairseq](https://github.com/pytorch/fairseq) with [Apex](https://github.com/NVIDIA/apex) is choosed as our baseline.
<img src="./docs/training/images/single_step.png"  width="80%" aligned="middle">
We compute speedup on different batch size using the WPS (real words per second) metric.
More results is available [here](./docs/training/performance.md)
### [>>> Inference](./lightseq/inference)
Here we present the experimental results on neural machine translation based on Transformer-base models using beam search methods.
We choose Tensorflow and
[FasterTransformer](https://github.com/NVIDIA/DeepLearningExamples/tree/master/FasterTransformer) as a comparison.
The implementation from
[tensor2tensor](https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/models/transformer.py)
was used as the benchmark of Tensorflow.
<img src="./docs/inference/images/nmt.png"  width="80%" aligned="middle">
More results is available [here](./docs/inference/performance.md).
## Quick Start
Complete user guide is available [here](docs/guide.md).
### Installation
You can install LightSeq from PyPI:
```shell
$ pip install lightseq
```
LightSeq installation from PyPI only supports Python 3.6 to 3.8 on Linux for now. Consider compiling from source if you have other environments:
```shell
$ PATH=/usr/local/hdf5/:$PATH ENABLE_FP32=0 ENABLE_DEBUG=0 pip install -e $PROJECT_DIR
```
Detailed building introduction is available [here](docs/inference/build.md).
### Fast training from Fairseq
You can experience lightning fast training by running following commands,
Firstly install these requirements.
```shell
$ pip install lightseq fairseq sacremoses
```
Then you can train a translation task on wmt14 en2de dataset by running the following script
```shell
$ sh examples/training/fairseq/ls_fairseq_wmt14en2de.sh
```
To compare lightseq with fairseq, delete the arguments with `ls_` prefix to using the original fairseq implementation
More usage is available [here](./lightseq/training/README.md).
### Fast inference from HuggingFace bart
We provide an end2end bart-base example to see how fast Lightseq is compared to HuggingFace. First you should install these requirements.
```shell
$ pip install torch tensorflow transformers lightseq
$ cd examples/inference/python
```
then you can check the performance by simply running following commands. `hf_bart_export.py` is used to transform pytorch weights to LightSeq protobuffer.
```shell
$ python export/huggingface/hf_bart_export.py
$ python test/ls_bart.py
```
More usage is available [here](./lightseq/inference/README.md).
### Fast deploy inference server
We provide a docker image which contains tritonserver and lightseq's dynamic link library, and you can deploy a inference server by simply replace the model file with your own model file.
```shell
$ sudo docker pull hexisyztem/tritonserver_lightseq:22.01-1
```
More usage is available [here](https://github.com/bytedance/lightseq/tree/master/examples/triton_backend).
## Cite Us
If you use LightSeq in your research, please cite the following paper.
```
@InProceedings{wang2021lightseq,
    title = "{L}ight{S}eq: A High Performance Inference Library for Transformers",
    author = "Wang, Xiaohui and Xiong, Ying and Wei, Yang and Wang, Mingxuan and Li, Lei",
    booktitle = "Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers (NAACL-HLT)",
    month = jun,
    year = "2021",
    publisher = "Association for Computational Linguistics",
    pages = "113--120",
}
@article{wang2021lightseq2,
  title={LightSeq2: Accelerated Training for Transformer-based Models on GPUs},
  author={Wang, Xiaohui and Xiong, Ying and Qian, Xian and Wei, Yang and Li, Lei and Wang, Mingxuan},
  journal={arXiv preprint arXiv:2110.05722},
  year={2021}
}
```
## Contact
Any questions or suggestions, please feel free to contact us at
wangxiaohui.neo@bytedance.com, xiongying.taka@bytedance.com, qian.xian@bytedance.com, weiyang.god@bytedance.com, wangmingxuan.89@bytedance.com, lilei@cs.ucsb.edu
## Hiring
The LightSeq team is hiring Interns/FTEs with backgrounds in deep learning system/natural language processing/computer vision/speech.
We are based in Beijing and Shanghai. If you are interested, please send your resume to wangxiaohui.neo@bytedance.com.

%package -n python3-lightseq
Summary:	LightSeq is a high performance library for sequence processing and generation
Provides:	python-lightseq
BuildRequires:	python3-devel
BuildRequires:	python3-setuptools
BuildRequires:	python3-pip
%description -n python3-lightseq
LightSeq is a high performance training and inference library for sequence processing and generation implemented
in CUDA.
It enables highly efficient computation of modern NLP models such as **BERT**, **GPT**,
**Transformer**, etc.
It is therefore best useful for *Machine Translation*, *Text Generation*, *Dialog*, *Language
Modelling*, *Sentiment Analysis*, and other related tasks with sequence data.
The library is built on top of CUDA official
library([cuBLAS](https://docs.nvidia.com/cuda/cublas/index.html),
[Thrust](https://docs.nvidia.com/cuda/thrust/index.html), [CUB](http://nvlabs.github.io/cub/)) and
custom kernel functions which are specially fused and optimized for Transformer model family. In
addition to model components, the inference library also provide easy-to deploy model management and serving backend based on
[TensorRT Inference
Server](https://docs.nvidia.com/deeplearning/sdk/inference-server-archived/tensorrt_inference_server_120/tensorrt-inference-server-guide/docs/quickstart.html).
With LightSeq, one can easily develop modified Transformer architecture with little additional code.
## Features
### [>>> Training](./lightseq/training)
The following is a support matrix of LightSeq **training** library compared with
[DeepSpeed](https://github.com/microsoft/DeepSpeed).
![features](./docs/training/images/features.png)
### [>>> Inference](./lightseq/inference)
The following is a support matrix of LightSeq **inference** library compared with
[TurboTransformers](https://github.com/Tencent/TurboTransformers) and
[FasterTransformer](https://github.com/NVIDIA/DeepLearningExamples/tree/master/FasterTransformer).
![support](./docs/inference/images/support.png)
## Performance
### [>>> Training](./lightseq/training)
Here we present the experimental results on WMT14 English to German translation task based on Transformer-big models. We train Transformer models of different sizes on eight NVIDIA Tesla V100/NVIDIA Tesla A100 GPUs with data parallel and fp16 mixed precision.
[Fairseq](https://github.com/pytorch/fairseq) with [Apex](https://github.com/NVIDIA/apex) is choosed as our baseline.
<img src="./docs/training/images/single_step.png"  width="80%" aligned="middle">
We compute speedup on different batch size using the WPS (real words per second) metric.
More results is available [here](./docs/training/performance.md)
### [>>> Inference](./lightseq/inference)
Here we present the experimental results on neural machine translation based on Transformer-base models using beam search methods.
We choose Tensorflow and
[FasterTransformer](https://github.com/NVIDIA/DeepLearningExamples/tree/master/FasterTransformer) as a comparison.
The implementation from
[tensor2tensor](https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/models/transformer.py)
was used as the benchmark of Tensorflow.
<img src="./docs/inference/images/nmt.png"  width="80%" aligned="middle">
More results is available [here](./docs/inference/performance.md).
## Quick Start
Complete user guide is available [here](docs/guide.md).
### Installation
You can install LightSeq from PyPI:
```shell
$ pip install lightseq
```
LightSeq installation from PyPI only supports Python 3.6 to 3.8 on Linux for now. Consider compiling from source if you have other environments:
```shell
$ PATH=/usr/local/hdf5/:$PATH ENABLE_FP32=0 ENABLE_DEBUG=0 pip install -e $PROJECT_DIR
```
Detailed building introduction is available [here](docs/inference/build.md).
### Fast training from Fairseq
You can experience lightning fast training by running following commands,
Firstly install these requirements.
```shell
$ pip install lightseq fairseq sacremoses
```
Then you can train a translation task on wmt14 en2de dataset by running the following script
```shell
$ sh examples/training/fairseq/ls_fairseq_wmt14en2de.sh
```
To compare lightseq with fairseq, delete the arguments with `ls_` prefix to using the original fairseq implementation
More usage is available [here](./lightseq/training/README.md).
### Fast inference from HuggingFace bart
We provide an end2end bart-base example to see how fast Lightseq is compared to HuggingFace. First you should install these requirements.
```shell
$ pip install torch tensorflow transformers lightseq
$ cd examples/inference/python
```
then you can check the performance by simply running following commands. `hf_bart_export.py` is used to transform pytorch weights to LightSeq protobuffer.
```shell
$ python export/huggingface/hf_bart_export.py
$ python test/ls_bart.py
```
More usage is available [here](./lightseq/inference/README.md).
### Fast deploy inference server
We provide a docker image which contains tritonserver and lightseq's dynamic link library, and you can deploy a inference server by simply replace the model file with your own model file.
```shell
$ sudo docker pull hexisyztem/tritonserver_lightseq:22.01-1
```
More usage is available [here](https://github.com/bytedance/lightseq/tree/master/examples/triton_backend).
## Cite Us
If you use LightSeq in your research, please cite the following paper.
```
@InProceedings{wang2021lightseq,
    title = "{L}ight{S}eq: A High Performance Inference Library for Transformers",
    author = "Wang, Xiaohui and Xiong, Ying and Wei, Yang and Wang, Mingxuan and Li, Lei",
    booktitle = "Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers (NAACL-HLT)",
    month = jun,
    year = "2021",
    publisher = "Association for Computational Linguistics",
    pages = "113--120",
}
@article{wang2021lightseq2,
  title={LightSeq2: Accelerated Training for Transformer-based Models on GPUs},
  author={Wang, Xiaohui and Xiong, Ying and Qian, Xian and Wei, Yang and Li, Lei and Wang, Mingxuan},
  journal={arXiv preprint arXiv:2110.05722},
  year={2021}
}
```
## Contact
Any questions or suggestions, please feel free to contact us at
wangxiaohui.neo@bytedance.com, xiongying.taka@bytedance.com, qian.xian@bytedance.com, weiyang.god@bytedance.com, wangmingxuan.89@bytedance.com, lilei@cs.ucsb.edu
## Hiring
The LightSeq team is hiring Interns/FTEs with backgrounds in deep learning system/natural language processing/computer vision/speech.
We are based in Beijing and Shanghai. If you are interested, please send your resume to wangxiaohui.neo@bytedance.com.

%package help
Summary:	Development documents and examples for lightseq
Provides:	python3-lightseq-doc
%description help
LightSeq is a high performance training and inference library for sequence processing and generation implemented
in CUDA.
It enables highly efficient computation of modern NLP models such as **BERT**, **GPT**,
**Transformer**, etc.
It is therefore best useful for *Machine Translation*, *Text Generation*, *Dialog*, *Language
Modelling*, *Sentiment Analysis*, and other related tasks with sequence data.
The library is built on top of CUDA official
library([cuBLAS](https://docs.nvidia.com/cuda/cublas/index.html),
[Thrust](https://docs.nvidia.com/cuda/thrust/index.html), [CUB](http://nvlabs.github.io/cub/)) and
custom kernel functions which are specially fused and optimized for Transformer model family. In
addition to model components, the inference library also provide easy-to deploy model management and serving backend based on
[TensorRT Inference
Server](https://docs.nvidia.com/deeplearning/sdk/inference-server-archived/tensorrt_inference_server_120/tensorrt-inference-server-guide/docs/quickstart.html).
With LightSeq, one can easily develop modified Transformer architecture with little additional code.
## Features
### [>>> Training](./lightseq/training)
The following is a support matrix of LightSeq **training** library compared with
[DeepSpeed](https://github.com/microsoft/DeepSpeed).
![features](./docs/training/images/features.png)
### [>>> Inference](./lightseq/inference)
The following is a support matrix of LightSeq **inference** library compared with
[TurboTransformers](https://github.com/Tencent/TurboTransformers) and
[FasterTransformer](https://github.com/NVIDIA/DeepLearningExamples/tree/master/FasterTransformer).
![support](./docs/inference/images/support.png)
## Performance
### [>>> Training](./lightseq/training)
Here we present the experimental results on WMT14 English to German translation task based on Transformer-big models. We train Transformer models of different sizes on eight NVIDIA Tesla V100/NVIDIA Tesla A100 GPUs with data parallel and fp16 mixed precision.
[Fairseq](https://github.com/pytorch/fairseq) with [Apex](https://github.com/NVIDIA/apex) is choosed as our baseline.
<img src="./docs/training/images/single_step.png"  width="80%" aligned="middle">
We compute speedup on different batch size using the WPS (real words per second) metric.
More results is available [here](./docs/training/performance.md)
### [>>> Inference](./lightseq/inference)
Here we present the experimental results on neural machine translation based on Transformer-base models using beam search methods.
We choose Tensorflow and
[FasterTransformer](https://github.com/NVIDIA/DeepLearningExamples/tree/master/FasterTransformer) as a comparison.
The implementation from
[tensor2tensor](https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/models/transformer.py)
was used as the benchmark of Tensorflow.
<img src="./docs/inference/images/nmt.png"  width="80%" aligned="middle">
More results is available [here](./docs/inference/performance.md).
## Quick Start
Complete user guide is available [here](docs/guide.md).
### Installation
You can install LightSeq from PyPI:
```shell
$ pip install lightseq
```
LightSeq installation from PyPI only supports Python 3.6 to 3.8 on Linux for now. Consider compiling from source if you have other environments:
```shell
$ PATH=/usr/local/hdf5/:$PATH ENABLE_FP32=0 ENABLE_DEBUG=0 pip install -e $PROJECT_DIR
```
Detailed building introduction is available [here](docs/inference/build.md).
### Fast training from Fairseq
You can experience lightning fast training by running following commands,
Firstly install these requirements.
```shell
$ pip install lightseq fairseq sacremoses
```
Then you can train a translation task on wmt14 en2de dataset by running the following script
```shell
$ sh examples/training/fairseq/ls_fairseq_wmt14en2de.sh
```
To compare lightseq with fairseq, delete the arguments with `ls_` prefix to using the original fairseq implementation
More usage is available [here](./lightseq/training/README.md).
### Fast inference from HuggingFace bart
We provide an end2end bart-base example to see how fast Lightseq is compared to HuggingFace. First you should install these requirements.
```shell
$ pip install torch tensorflow transformers lightseq
$ cd examples/inference/python
```
then you can check the performance by simply running following commands. `hf_bart_export.py` is used to transform pytorch weights to LightSeq protobuffer.
```shell
$ python export/huggingface/hf_bart_export.py
$ python test/ls_bart.py
```
More usage is available [here](./lightseq/inference/README.md).
### Fast deploy inference server
We provide a docker image which contains tritonserver and lightseq's dynamic link library, and you can deploy a inference server by simply replace the model file with your own model file.
```shell
$ sudo docker pull hexisyztem/tritonserver_lightseq:22.01-1
```
More usage is available [here](https://github.com/bytedance/lightseq/tree/master/examples/triton_backend).
## Cite Us
If you use LightSeq in your research, please cite the following paper.
```
@InProceedings{wang2021lightseq,
    title = "{L}ight{S}eq: A High Performance Inference Library for Transformers",
    author = "Wang, Xiaohui and Xiong, Ying and Wei, Yang and Wang, Mingxuan and Li, Lei",
    booktitle = "Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers (NAACL-HLT)",
    month = jun,
    year = "2021",
    publisher = "Association for Computational Linguistics",
    pages = "113--120",
}
@article{wang2021lightseq2,
  title={LightSeq2: Accelerated Training for Transformer-based Models on GPUs},
  author={Wang, Xiaohui and Xiong, Ying and Qian, Xian and Wei, Yang and Li, Lei and Wang, Mingxuan},
  journal={arXiv preprint arXiv:2110.05722},
  year={2021}
}
```
## Contact
Any questions or suggestions, please feel free to contact us at
wangxiaohui.neo@bytedance.com, xiongying.taka@bytedance.com, qian.xian@bytedance.com, weiyang.god@bytedance.com, wangmingxuan.89@bytedance.com, lilei@cs.ucsb.edu
## Hiring
The LightSeq team is hiring Interns/FTEs with backgrounds in deep learning system/natural language processing/computer vision/speech.
We are based in Beijing and Shanghai. If you are interested, please send your resume to wangxiaohui.neo@bytedance.com.

%prep
%autosetup -n lightseq-3.0.1

%build
%py3_build

%install
%py3_install
install -d -m755 %{buildroot}/%{_pkgdocdir}
if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi
if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi
if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi
if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi
pushd %{buildroot}
if [ -d usr/lib ]; then
	find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst
fi
if [ -d usr/lib64 ]; then
	find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst
fi
if [ -d usr/bin ]; then
	find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst
fi
if [ -d usr/sbin ]; then
	find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst
fi
touch doclist.lst
if [ -d usr/share/man ]; then
	find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst
fi
popd
mv %{buildroot}/filelist.lst .
mv %{buildroot}/doclist.lst .

%files -n python3-lightseq -f filelist.lst
%dir %{python3_sitelib}/*

%files help -f doclist.lst
%{_docdir}/*

%changelog
* Fri May 05 2023 Python_Bot <Python_Bot@openeuler.org> - 3.0.1-1
- Package Spec generated