diff options
| -rw-r--r-- | .gitignore | 1 | ||||
| -rw-r--r-- | python-lightseq.spec | 393 | ||||
| -rw-r--r-- | sources | 1 |
3 files changed, 395 insertions, 0 deletions
@@ -0,0 +1 @@ +/lightseq-3.0.1.tar.gz diff --git a/python-lightseq.spec b/python-lightseq.spec new file mode 100644 index 0000000..e81248a --- /dev/null +++ b/python-lightseq.spec @@ -0,0 +1,393 @@ +%global _empty_manifest_terminate_build 0 +Name: python-lightseq +Version: 3.0.1 +Release: 1 +Summary: LightSeq is a high performance library for sequence processing and generation +License: Apache Software License +URL: https://github.com/bytedance/lightseq +Source0: https://mirrors.nju.edu.cn/pypi/web/packages/92/c3/ca4ed0027fb97a4fb6f0cf30010f7e0111bf688975c97daee297d1de0e51/lightseq-3.0.1.tar.gz +BuildArch: noarch + +Requires: python3-ninja +Requires: python3-numpy +Requires: python3-scipy + +%description +LightSeq is a high performance training and inference library for sequence processing and generation implemented +in CUDA. +It enables highly efficient computation of modern NLP models such as **BERT**, **GPT**, +**Transformer**, etc. +It is therefore best useful for *Machine Translation*, *Text Generation*, *Dialog*, *Language +Modelling*, *Sentiment Analysis*, and other related tasks with sequence data. +The library is built on top of CUDA official +library([cuBLAS](https://docs.nvidia.com/cuda/cublas/index.html), +[Thrust](https://docs.nvidia.com/cuda/thrust/index.html), [CUB](http://nvlabs.github.io/cub/)) and +custom kernel functions which are specially fused and optimized for Transformer model family. In +addition to model components, the inference library also provide easy-to deploy model management and serving backend based on +[TensorRT Inference +Server](https://docs.nvidia.com/deeplearning/sdk/inference-server-archived/tensorrt_inference_server_120/tensorrt-inference-server-guide/docs/quickstart.html). +With LightSeq, one can easily develop modified Transformer architecture with little additional code. +## Features +### [>>> Training](./lightseq/training) +The following is a support matrix of LightSeq **training** library compared with +[DeepSpeed](https://github.com/microsoft/DeepSpeed). + +### [>>> Inference](./lightseq/inference) +The following is a support matrix of LightSeq **inference** library compared with +[TurboTransformers](https://github.com/Tencent/TurboTransformers) and +[FasterTransformer](https://github.com/NVIDIA/DeepLearningExamples/tree/master/FasterTransformer). + +## Performance +### [>>> Training](./lightseq/training) +Here we present the experimental results on WMT14 English to German translation task based on Transformer-big models. We train Transformer models of different sizes on eight NVIDIA Tesla V100/NVIDIA Tesla A100 GPUs with data parallel and fp16 mixed precision. +[Fairseq](https://github.com/pytorch/fairseq) with [Apex](https://github.com/NVIDIA/apex) is choosed as our baseline. +<img src="./docs/training/images/single_step.png" width="80%" aligned="middle"> +We compute speedup on different batch size using the WPS (real words per second) metric. +More results is available [here](./docs/training/performance.md) +### [>>> Inference](./lightseq/inference) +Here we present the experimental results on neural machine translation based on Transformer-base models using beam search methods. +We choose Tensorflow and +[FasterTransformer](https://github.com/NVIDIA/DeepLearningExamples/tree/master/FasterTransformer) as a comparison. +The implementation from +[tensor2tensor](https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/models/transformer.py) +was used as the benchmark of Tensorflow. +<img src="./docs/inference/images/nmt.png" width="80%" aligned="middle"> +More results is available [here](./docs/inference/performance.md). +## Quick Start +Complete user guide is available [here](docs/guide.md). +### Installation +You can install LightSeq from PyPI: +```shell +$ pip install lightseq +``` +LightSeq installation from PyPI only supports Python 3.6 to 3.8 on Linux for now. Consider compiling from source if you have other environments: +```shell +$ PATH=/usr/local/hdf5/:$PATH ENABLE_FP32=0 ENABLE_DEBUG=0 pip install -e $PROJECT_DIR +``` +Detailed building introduction is available [here](docs/inference/build.md). +### Fast training from Fairseq +You can experience lightning fast training by running following commands, +Firstly install these requirements. +```shell +$ pip install lightseq fairseq sacremoses +``` +Then you can train a translation task on wmt14 en2de dataset by running the following script +```shell +$ sh examples/training/fairseq/ls_fairseq_wmt14en2de.sh +``` +To compare lightseq with fairseq, delete the arguments with `ls_` prefix to using the original fairseq implementation +More usage is available [here](./lightseq/training/README.md). +### Fast inference from HuggingFace bart +We provide an end2end bart-base example to see how fast Lightseq is compared to HuggingFace. First you should install these requirements. +```shell +$ pip install torch tensorflow transformers lightseq +$ cd examples/inference/python +``` +then you can check the performance by simply running following commands. `hf_bart_export.py` is used to transform pytorch weights to LightSeq protobuffer. +```shell +$ python export/huggingface/hf_bart_export.py +$ python test/ls_bart.py +``` +More usage is available [here](./lightseq/inference/README.md). +### Fast deploy inference server +We provide a docker image which contains tritonserver and lightseq's dynamic link library, and you can deploy a inference server by simply replace the model file with your own model file. +```shell +$ sudo docker pull hexisyztem/tritonserver_lightseq:22.01-1 +``` +More usage is available [here](https://github.com/bytedance/lightseq/tree/master/examples/triton_backend). +## Cite Us +If you use LightSeq in your research, please cite the following paper. +``` +@InProceedings{wang2021lightseq, + title = "{L}ight{S}eq: A High Performance Inference Library for Transformers", + author = "Wang, Xiaohui and Xiong, Ying and Wei, Yang and Wang, Mingxuan and Li, Lei", + booktitle = "Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers (NAACL-HLT)", + month = jun, + year = "2021", + publisher = "Association for Computational Linguistics", + pages = "113--120", +} +@article{wang2021lightseq2, + title={LightSeq2: Accelerated Training for Transformer-based Models on GPUs}, + author={Wang, Xiaohui and Xiong, Ying and Qian, Xian and Wei, Yang and Li, Lei and Wang, Mingxuan}, + journal={arXiv preprint arXiv:2110.05722}, + year={2021} +} +``` +## Contact +Any questions or suggestions, please feel free to contact us at +wangxiaohui.neo@bytedance.com, xiongying.taka@bytedance.com, qian.xian@bytedance.com, weiyang.god@bytedance.com, wangmingxuan.89@bytedance.com, lilei@cs.ucsb.edu +## Hiring +The LightSeq team is hiring Interns/FTEs with backgrounds in deep learning system/natural language processing/computer vision/speech. +We are based in Beijing and Shanghai. If you are interested, please send your resume to wangxiaohui.neo@bytedance.com. + +%package -n python3-lightseq +Summary: LightSeq is a high performance library for sequence processing and generation +Provides: python-lightseq +BuildRequires: python3-devel +BuildRequires: python3-setuptools +BuildRequires: python3-pip +%description -n python3-lightseq +LightSeq is a high performance training and inference library for sequence processing and generation implemented +in CUDA. +It enables highly efficient computation of modern NLP models such as **BERT**, **GPT**, +**Transformer**, etc. +It is therefore best useful for *Machine Translation*, *Text Generation*, *Dialog*, *Language +Modelling*, *Sentiment Analysis*, and other related tasks with sequence data. +The library is built on top of CUDA official +library([cuBLAS](https://docs.nvidia.com/cuda/cublas/index.html), +[Thrust](https://docs.nvidia.com/cuda/thrust/index.html), [CUB](http://nvlabs.github.io/cub/)) and +custom kernel functions which are specially fused and optimized for Transformer model family. In +addition to model components, the inference library also provide easy-to deploy model management and serving backend based on +[TensorRT Inference +Server](https://docs.nvidia.com/deeplearning/sdk/inference-server-archived/tensorrt_inference_server_120/tensorrt-inference-server-guide/docs/quickstart.html). +With LightSeq, one can easily develop modified Transformer architecture with little additional code. +## Features +### [>>> Training](./lightseq/training) +The following is a support matrix of LightSeq **training** library compared with +[DeepSpeed](https://github.com/microsoft/DeepSpeed). + +### [>>> Inference](./lightseq/inference) +The following is a support matrix of LightSeq **inference** library compared with +[TurboTransformers](https://github.com/Tencent/TurboTransformers) and +[FasterTransformer](https://github.com/NVIDIA/DeepLearningExamples/tree/master/FasterTransformer). + +## Performance +### [>>> Training](./lightseq/training) +Here we present the experimental results on WMT14 English to German translation task based on Transformer-big models. We train Transformer models of different sizes on eight NVIDIA Tesla V100/NVIDIA Tesla A100 GPUs with data parallel and fp16 mixed precision. +[Fairseq](https://github.com/pytorch/fairseq) with [Apex](https://github.com/NVIDIA/apex) is choosed as our baseline. +<img src="./docs/training/images/single_step.png" width="80%" aligned="middle"> +We compute speedup on different batch size using the WPS (real words per second) metric. +More results is available [here](./docs/training/performance.md) +### [>>> Inference](./lightseq/inference) +Here we present the experimental results on neural machine translation based on Transformer-base models using beam search methods. +We choose Tensorflow and +[FasterTransformer](https://github.com/NVIDIA/DeepLearningExamples/tree/master/FasterTransformer) as a comparison. +The implementation from +[tensor2tensor](https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/models/transformer.py) +was used as the benchmark of Tensorflow. +<img src="./docs/inference/images/nmt.png" width="80%" aligned="middle"> +More results is available [here](./docs/inference/performance.md). +## Quick Start +Complete user guide is available [here](docs/guide.md). +### Installation +You can install LightSeq from PyPI: +```shell +$ pip install lightseq +``` +LightSeq installation from PyPI only supports Python 3.6 to 3.8 on Linux for now. Consider compiling from source if you have other environments: +```shell +$ PATH=/usr/local/hdf5/:$PATH ENABLE_FP32=0 ENABLE_DEBUG=0 pip install -e $PROJECT_DIR +``` +Detailed building introduction is available [here](docs/inference/build.md). +### Fast training from Fairseq +You can experience lightning fast training by running following commands, +Firstly install these requirements. +```shell +$ pip install lightseq fairseq sacremoses +``` +Then you can train a translation task on wmt14 en2de dataset by running the following script +```shell +$ sh examples/training/fairseq/ls_fairseq_wmt14en2de.sh +``` +To compare lightseq with fairseq, delete the arguments with `ls_` prefix to using the original fairseq implementation +More usage is available [here](./lightseq/training/README.md). +### Fast inference from HuggingFace bart +We provide an end2end bart-base example to see how fast Lightseq is compared to HuggingFace. First you should install these requirements. +```shell +$ pip install torch tensorflow transformers lightseq +$ cd examples/inference/python +``` +then you can check the performance by simply running following commands. `hf_bart_export.py` is used to transform pytorch weights to LightSeq protobuffer. +```shell +$ python export/huggingface/hf_bart_export.py +$ python test/ls_bart.py +``` +More usage is available [here](./lightseq/inference/README.md). +### Fast deploy inference server +We provide a docker image which contains tritonserver and lightseq's dynamic link library, and you can deploy a inference server by simply replace the model file with your own model file. +```shell +$ sudo docker pull hexisyztem/tritonserver_lightseq:22.01-1 +``` +More usage is available [here](https://github.com/bytedance/lightseq/tree/master/examples/triton_backend). +## Cite Us +If you use LightSeq in your research, please cite the following paper. +``` +@InProceedings{wang2021lightseq, + title = "{L}ight{S}eq: A High Performance Inference Library for Transformers", + author = "Wang, Xiaohui and Xiong, Ying and Wei, Yang and Wang, Mingxuan and Li, Lei", + booktitle = "Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers (NAACL-HLT)", + month = jun, + year = "2021", + publisher = "Association for Computational Linguistics", + pages = "113--120", +} +@article{wang2021lightseq2, + title={LightSeq2: Accelerated Training for Transformer-based Models on GPUs}, + author={Wang, Xiaohui and Xiong, Ying and Qian, Xian and Wei, Yang and Li, Lei and Wang, Mingxuan}, + journal={arXiv preprint arXiv:2110.05722}, + year={2021} +} +``` +## Contact +Any questions or suggestions, please feel free to contact us at +wangxiaohui.neo@bytedance.com, xiongying.taka@bytedance.com, qian.xian@bytedance.com, weiyang.god@bytedance.com, wangmingxuan.89@bytedance.com, lilei@cs.ucsb.edu +## Hiring +The LightSeq team is hiring Interns/FTEs with backgrounds in deep learning system/natural language processing/computer vision/speech. +We are based in Beijing and Shanghai. If you are interested, please send your resume to wangxiaohui.neo@bytedance.com. + +%package help +Summary: Development documents and examples for lightseq +Provides: python3-lightseq-doc +%description help +LightSeq is a high performance training and inference library for sequence processing and generation implemented +in CUDA. +It enables highly efficient computation of modern NLP models such as **BERT**, **GPT**, +**Transformer**, etc. +It is therefore best useful for *Machine Translation*, *Text Generation*, *Dialog*, *Language +Modelling*, *Sentiment Analysis*, and other related tasks with sequence data. +The library is built on top of CUDA official +library([cuBLAS](https://docs.nvidia.com/cuda/cublas/index.html), +[Thrust](https://docs.nvidia.com/cuda/thrust/index.html), [CUB](http://nvlabs.github.io/cub/)) and +custom kernel functions which are specially fused and optimized for Transformer model family. In +addition to model components, the inference library also provide easy-to deploy model management and serving backend based on +[TensorRT Inference +Server](https://docs.nvidia.com/deeplearning/sdk/inference-server-archived/tensorrt_inference_server_120/tensorrt-inference-server-guide/docs/quickstart.html). +With LightSeq, one can easily develop modified Transformer architecture with little additional code. +## Features +### [>>> Training](./lightseq/training) +The following is a support matrix of LightSeq **training** library compared with +[DeepSpeed](https://github.com/microsoft/DeepSpeed). + +### [>>> Inference](./lightseq/inference) +The following is a support matrix of LightSeq **inference** library compared with +[TurboTransformers](https://github.com/Tencent/TurboTransformers) and +[FasterTransformer](https://github.com/NVIDIA/DeepLearningExamples/tree/master/FasterTransformer). + +## Performance +### [>>> Training](./lightseq/training) +Here we present the experimental results on WMT14 English to German translation task based on Transformer-big models. We train Transformer models of different sizes on eight NVIDIA Tesla V100/NVIDIA Tesla A100 GPUs with data parallel and fp16 mixed precision. +[Fairseq](https://github.com/pytorch/fairseq) with [Apex](https://github.com/NVIDIA/apex) is choosed as our baseline. +<img src="./docs/training/images/single_step.png" width="80%" aligned="middle"> +We compute speedup on different batch size using the WPS (real words per second) metric. +More results is available [here](./docs/training/performance.md) +### [>>> Inference](./lightseq/inference) +Here we present the experimental results on neural machine translation based on Transformer-base models using beam search methods. +We choose Tensorflow and +[FasterTransformer](https://github.com/NVIDIA/DeepLearningExamples/tree/master/FasterTransformer) as a comparison. +The implementation from +[tensor2tensor](https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/models/transformer.py) +was used as the benchmark of Tensorflow. +<img src="./docs/inference/images/nmt.png" width="80%" aligned="middle"> +More results is available [here](./docs/inference/performance.md). +## Quick Start +Complete user guide is available [here](docs/guide.md). +### Installation +You can install LightSeq from PyPI: +```shell +$ pip install lightseq +``` +LightSeq installation from PyPI only supports Python 3.6 to 3.8 on Linux for now. Consider compiling from source if you have other environments: +```shell +$ PATH=/usr/local/hdf5/:$PATH ENABLE_FP32=0 ENABLE_DEBUG=0 pip install -e $PROJECT_DIR +``` +Detailed building introduction is available [here](docs/inference/build.md). +### Fast training from Fairseq +You can experience lightning fast training by running following commands, +Firstly install these requirements. +```shell +$ pip install lightseq fairseq sacremoses +``` +Then you can train a translation task on wmt14 en2de dataset by running the following script +```shell +$ sh examples/training/fairseq/ls_fairseq_wmt14en2de.sh +``` +To compare lightseq with fairseq, delete the arguments with `ls_` prefix to using the original fairseq implementation +More usage is available [here](./lightseq/training/README.md). +### Fast inference from HuggingFace bart +We provide an end2end bart-base example to see how fast Lightseq is compared to HuggingFace. First you should install these requirements. +```shell +$ pip install torch tensorflow transformers lightseq +$ cd examples/inference/python +``` +then you can check the performance by simply running following commands. `hf_bart_export.py` is used to transform pytorch weights to LightSeq protobuffer. +```shell +$ python export/huggingface/hf_bart_export.py +$ python test/ls_bart.py +``` +More usage is available [here](./lightseq/inference/README.md). +### Fast deploy inference server +We provide a docker image which contains tritonserver and lightseq's dynamic link library, and you can deploy a inference server by simply replace the model file with your own model file. +```shell +$ sudo docker pull hexisyztem/tritonserver_lightseq:22.01-1 +``` +More usage is available [here](https://github.com/bytedance/lightseq/tree/master/examples/triton_backend). +## Cite Us +If you use LightSeq in your research, please cite the following paper. +``` +@InProceedings{wang2021lightseq, + title = "{L}ight{S}eq: A High Performance Inference Library for Transformers", + author = "Wang, Xiaohui and Xiong, Ying and Wei, Yang and Wang, Mingxuan and Li, Lei", + booktitle = "Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers (NAACL-HLT)", + month = jun, + year = "2021", + publisher = "Association for Computational Linguistics", + pages = "113--120", +} +@article{wang2021lightseq2, + title={LightSeq2: Accelerated Training for Transformer-based Models on GPUs}, + author={Wang, Xiaohui and Xiong, Ying and Qian, Xian and Wei, Yang and Li, Lei and Wang, Mingxuan}, + journal={arXiv preprint arXiv:2110.05722}, + year={2021} +} +``` +## Contact +Any questions or suggestions, please feel free to contact us at +wangxiaohui.neo@bytedance.com, xiongying.taka@bytedance.com, qian.xian@bytedance.com, weiyang.god@bytedance.com, wangmingxuan.89@bytedance.com, lilei@cs.ucsb.edu +## Hiring +The LightSeq team is hiring Interns/FTEs with backgrounds in deep learning system/natural language processing/computer vision/speech. +We are based in Beijing and Shanghai. If you are interested, please send your resume to wangxiaohui.neo@bytedance.com. + +%prep +%autosetup -n lightseq-3.0.1 + +%build +%py3_build + +%install +%py3_install +install -d -m755 %{buildroot}/%{_pkgdocdir} +if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi +if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi +if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi +if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi +pushd %{buildroot} +if [ -d usr/lib ]; then + find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/lib64 ]; then + find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/bin ]; then + find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst +fi +if [ -d usr/sbin ]; then + find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst +fi +touch doclist.lst +if [ -d usr/share/man ]; then + find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst +fi +popd +mv %{buildroot}/filelist.lst . +mv %{buildroot}/doclist.lst . + +%files -n python3-lightseq -f filelist.lst +%dir %{python3_sitelib}/* + +%files help -f doclist.lst +%{_docdir}/* + +%changelog +* Fri May 05 2023 Python_Bot <Python_Bot@openeuler.org> - 3.0.1-1 +- Package Spec generated @@ -0,0 +1 @@ +33730b8782d7956ea9de94e0ea95b321 lightseq-3.0.1.tar.gz |