automatic import of python-lingtrain-aligner

3 files changed, 236 insertions, 0 deletions

diff --git a/.gitignore b/.gitignore
index e69de29..aa68ba7 100644
--- a/.gitignore
+++ b/.gitignore
@@ -0,0 +1 @@
+/lingtrain-aligner-0.8.7.tar.gz
diff --git a/python-lingtrain-aligner.spec b/python-lingtrain-aligner.spec
new file mode 100644
index 0000000..774a5d5
--- /dev/null
+++ b/python-lingtrain-aligner.spec
@@ -0,0 +1,234 @@
+%global _empty_manifest_terminate_build 0
+Name:		python-lingtrain-aligner
+Version:	0.8.7
+Release:	1
+Summary:	Alignment tool for texts in different languages
+License:	GNU General Public License v3 (GPLv3)
+URL:		https://github.com/averkij/lingtrain-aligner
+Source0:	https://mirrors.nju.edu.cn/pypi/web/packages/3b/8b/e70ab4a0a3ae773e16b91c3935297fdde6b74f00fe1e6aafa617d8f02386/lingtrain-aligner-0.8.7.tar.gz
+BuildArch:	noarch
+
+
+%description
+[![PyPI - PyPi](https://img.shields.io/pypi/v/lingtrain-aligner)](https://pypi.org/project/lingtrain-aligner) [![Downloads](https://static.pepy.tech/personalized-badge/lingtrain-aligner?period=total&units=abbreviation&left_color=grey&right_color=green&left_text=Downloads)](https://pepy.tech/project/lingtrain-aligner)
+
+# Lingtrain Aligner
+
+ML powered library for the accurate texts alignment in different languages.
+
+- 🔥 [How to create bilingual books. Part 2. Lingtrain Alignment Studio](https://habr.com/ru/post/590549/)
+- 💡 [Lingtrain Aligner. How to make parallel books for language learning. Part 1. Python and Colab version](https://habr.com/ru/post/586574/)
+
+![Cover](https://i.imgur.com/WQWB4v0.png)
+
+## Purpose
+
+Main purpose of this alignment tool is to build parallel corpora using two or more raw texts in different languages. Texts should contain the same information (i.e., one text should be a translated analog oh the other text). E.g., it can be the _Drei Kameraden_ by Remarque in German and the _Three Comrades_ — it's translation into English.
+
+## Process
+
+There are plenty of obstacles during the alignment process:
+
+- The translator could translate several sentences as one.
+- The translator could translate one sentence as many.
+- There are some service marks in the text
+    - Page numbers
+    - Chapters and other section headings
+    - Author and title information
+    - Notes
+
+While service marks can be handled manually (the tool helps to detect them), the translation conflicts should be handled more carefully.
+
+Lingtrain Aligner tool will do almost all alignment work for you. It matches the sentence pairs automatically using the multilingual machine learning models. Then it searches for the alignment conflicts and resolves them. As output you will have the parallel corpora either as two distinct plain text files or as the merged corpora in widely used TMX format.
+
+### Supported languages and models
+
+Automated alignment process relies on the sentence embeddings models. Embeddings are multidimensional vectors of a special kind which are used to calculate a distance between the sentences. Supported languages list depend on the selected backend model.
+
+- **distiluse-base-multilingual-cased-v2**
+  - more reliable and fast
+  - moderate weights size — 500MB
+  - supports 50+ languages
+  - full list of supported languages can be found in [this paper](https://arxiv.org/abs/2004.09813)
+- **LaBSE (Language-agnostic BERT Sentence Embedding)**
+  - can be used for rare languages
+  - pretty heavy weights — 1.8GB
+  - supports 100+ languages
+  - full list of supported languages can be found [here](https://arxiv.org/abs/2007.01852)
+
+## Profit
+
+- Parallel corpora by itself can used as the resource for machine translation models or for linguistic researches.
+- My personal goal of this project is to help people building parallel translated books for the foreign language learning. 
+
+
+
+
+
+
+%package -n python3-lingtrain-aligner
+Summary:	Alignment tool for texts in different languages
+Provides:	python-lingtrain-aligner
+BuildRequires:	python3-devel
+BuildRequires:	python3-setuptools
+BuildRequires:	python3-pip
+%description -n python3-lingtrain-aligner
+[![PyPI - PyPi](https://img.shields.io/pypi/v/lingtrain-aligner)](https://pypi.org/project/lingtrain-aligner) [![Downloads](https://static.pepy.tech/personalized-badge/lingtrain-aligner?period=total&units=abbreviation&left_color=grey&right_color=green&left_text=Downloads)](https://pepy.tech/project/lingtrain-aligner)
+
+# Lingtrain Aligner
+
+ML powered library for the accurate texts alignment in different languages.
+
+- 🔥 [How to create bilingual books. Part 2. Lingtrain Alignment Studio](https://habr.com/ru/post/590549/)
+- 💡 [Lingtrain Aligner. How to make parallel books for language learning. Part 1. Python and Colab version](https://habr.com/ru/post/586574/)
+
+![Cover](https://i.imgur.com/WQWB4v0.png)
+
+## Purpose
+
+Main purpose of this alignment tool is to build parallel corpora using two or more raw texts in different languages. Texts should contain the same information (i.e., one text should be a translated analog oh the other text). E.g., it can be the _Drei Kameraden_ by Remarque in German and the _Three Comrades_ — it's translation into English.
+
+## Process
+
+There are plenty of obstacles during the alignment process:
+
+- The translator could translate several sentences as one.
+- The translator could translate one sentence as many.
+- There are some service marks in the text
+    - Page numbers
+    - Chapters and other section headings
+    - Author and title information
+    - Notes
+
+While service marks can be handled manually (the tool helps to detect them), the translation conflicts should be handled more carefully.
+
+Lingtrain Aligner tool will do almost all alignment work for you. It matches the sentence pairs automatically using the multilingual machine learning models. Then it searches for the alignment conflicts and resolves them. As output you will have the parallel corpora either as two distinct plain text files or as the merged corpora in widely used TMX format.
+
+### Supported languages and models
+
+Automated alignment process relies on the sentence embeddings models. Embeddings are multidimensional vectors of a special kind which are used to calculate a distance between the sentences. Supported languages list depend on the selected backend model.
+
+- **distiluse-base-multilingual-cased-v2**
+  - more reliable and fast
+  - moderate weights size — 500MB
+  - supports 50+ languages
+  - full list of supported languages can be found in [this paper](https://arxiv.org/abs/2004.09813)
+- **LaBSE (Language-agnostic BERT Sentence Embedding)**
+  - can be used for rare languages
+  - pretty heavy weights — 1.8GB
+  - supports 100+ languages
+  - full list of supported languages can be found [here](https://arxiv.org/abs/2007.01852)
+
+## Profit
+
+- Parallel corpora by itself can used as the resource for machine translation models or for linguistic researches.
+- My personal goal of this project is to help people building parallel translated books for the foreign language learning. 
+
+
+
+
+
+
+%package help
+Summary:	Development documents and examples for lingtrain-aligner
+Provides:	python3-lingtrain-aligner-doc
+%description help
+[![PyPI - PyPi](https://img.shields.io/pypi/v/lingtrain-aligner)](https://pypi.org/project/lingtrain-aligner) [![Downloads](https://static.pepy.tech/personalized-badge/lingtrain-aligner?period=total&units=abbreviation&left_color=grey&right_color=green&left_text=Downloads)](https://pepy.tech/project/lingtrain-aligner)
+
+# Lingtrain Aligner
+
+ML powered library for the accurate texts alignment in different languages.
+
+- 🔥 [How to create bilingual books. Part 2. Lingtrain Alignment Studio](https://habr.com/ru/post/590549/)
+- 💡 [Lingtrain Aligner. How to make parallel books for language learning. Part 1. Python and Colab version](https://habr.com/ru/post/586574/)
+
+![Cover](https://i.imgur.com/WQWB4v0.png)
+
+## Purpose
+
+Main purpose of this alignment tool is to build parallel corpora using two or more raw texts in different languages. Texts should contain the same information (i.e., one text should be a translated analog oh the other text). E.g., it can be the _Drei Kameraden_ by Remarque in German and the _Three Comrades_ — it's translation into English.
+
+## Process
+
+There are plenty of obstacles during the alignment process:
+
+- The translator could translate several sentences as one.
+- The translator could translate one sentence as many.
+- There are some service marks in the text
+    - Page numbers
+    - Chapters and other section headings
+    - Author and title information
+    - Notes
+
+While service marks can be handled manually (the tool helps to detect them), the translation conflicts should be handled more carefully.
+
+Lingtrain Aligner tool will do almost all alignment work for you. It matches the sentence pairs automatically using the multilingual machine learning models. Then it searches for the alignment conflicts and resolves them. As output you will have the parallel corpora either as two distinct plain text files or as the merged corpora in widely used TMX format.
+
+### Supported languages and models
+
+Automated alignment process relies on the sentence embeddings models. Embeddings are multidimensional vectors of a special kind which are used to calculate a distance between the sentences. Supported languages list depend on the selected backend model.
+
+- **distiluse-base-multilingual-cased-v2**
+  - more reliable and fast
+  - moderate weights size — 500MB
+  - supports 50+ languages
+  - full list of supported languages can be found in [this paper](https://arxiv.org/abs/2004.09813)
+- **LaBSE (Language-agnostic BERT Sentence Embedding)**
+  - can be used for rare languages
+  - pretty heavy weights — 1.8GB
+  - supports 100+ languages
+  - full list of supported languages can be found [here](https://arxiv.org/abs/2007.01852)
+
+## Profit
+
+- Parallel corpora by itself can used as the resource for machine translation models or for linguistic researches.
+- My personal goal of this project is to help people building parallel translated books for the foreign language learning. 
+
+
+
+
+
+
+%prep
+%autosetup -n lingtrain-aligner-0.8.7
+
+%build
+%py3_build
+
+%install
+%py3_install
+install -d -m755 %{buildroot}/%{_pkgdocdir}
+if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi
+if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi
+if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi
+if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi
+pushd %{buildroot}
+if [ -d usr/lib ]; then
+	find usr/lib -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/lib64 ]; then
+	find usr/lib64 -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/bin ]; then
+	find usr/bin -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+if [ -d usr/sbin ]; then
+	find usr/sbin -type f -printf "/%h/%f\n" >> filelist.lst
+fi
+touch doclist.lst
+if [ -d usr/share/man ]; then
+	find usr/share/man -type f -printf "/%h/%f.gz\n" >> doclist.lst
+fi
+popd
+mv %{buildroot}/filelist.lst .
+mv %{buildroot}/doclist.lst .
+
+%files -n python3-lingtrain-aligner -f filelist.lst
+%dir %{python3_sitelib}/*
+
+%files help -f doclist.lst
+%{_docdir}/*
+
+%changelog
+* Mon May 15 2023 Python_Bot <Python_Bot@openeuler.org> - 0.8.7-1
+- Package Spec generated
diff --git a/sources b/sources
new file mode 100644
index 0000000..6344106
--- /dev/null
+++ b/sources
@@ -0,0 +1 @@
+23a95781bf8e94507164eed0f3cc2fac  lingtrain-aligner-0.8.7.tar.gz