path: root/python-finbert-embedding.spec

%global _empty_manifest_terminate_build 0
Name:		python-finbert-embedding
Version:	0.1.5
Release:	1
Summary:	Embeddings from Financial BERT
License:	MIT License
URL:		https://github.com/abhijeet3922/finbert_embedding
Source0:	https://mirrors.aliyun.com/pypi/web/packages/a1/54/beeb2b0f98bcb1a2e560ae29a728fd518526300bf2abc423cd8dc74d36d8/finbert-embedding-0.1.5.tar.gz
BuildArch:	noarch

Requires:	python3-torch
Requires:	python3-pytorch-pretrained-bert
Requires:	python3-tensorflow

%description
# finbert_embedding
Token and sentence level embeddings from FinBERT model (Financial Domain).

[BERT](https://arxiv.org/abs/1810.04805), published by Google, is conceptually simple and empirically powerful as it obtained state-of-the-art results on eleven natural language processing tasks.  

The objective of this project is to obtain the word or sentence embeddings from [FinBERT](https://github.com/ProsusAI/finBERT), pre-trained model by Dogu Tan Araci (University of Amsterdam). FinBERT, which is a BERT language model further trained on Financial news articles for adapting financial domain. It achieved the state-of-the-art on FiQA sentiment scoring and Financial PhraseBank dataset. Paper [here](https://arxiv.org/abs/1908.10063).

Instead of building and do fine-tuning for an end-to-end NLP model, You can directly utilize word embeddings from Financial BERT to build NLP models for various downstream tasks eg. Financial text classification, Text clustering, Extractive summarization or Entity extraction etc.



## Features
* Creates an abstraction to remove dealing with inferencing pre-trained FinBERT model.
* Require only two lines of code to get sentence/token-level encoding for a text sentence.
* The package takes care of OOVs (out of vocabulary) inherently.
* Downloads and installs FinBERT pre-trained model (first initialization, usage in next section).

## Install
(Recommended to create a conda env to have isolation and avoid dependency clashes)

```
pip install finbert-embedding==0.1.4
```

Note: If you get error in installing this package (common error with Tf): <br>

Installing collected packages: wrapt, tensorflow <br>
  Found existing installation: wrapt 1.10.11 <br>
ERROR: Cannot uninstall 'wrapt'. It is a distutils installed project....

then, just do this:
```
pip install wrapt --upgrade --ignore-installed
pip install finbert-embedding==0.1.4
```

## Usage 1

word embeddings generated are list of 768 dimensional embeddings for each word. <br>
sentence embedding generated is 768 dimensional embedding which is average of each token.

```python
from finbert_embedding.embedding import FinbertEmbedding

text = "Another PSU bank, Punjab National Bank which also reported numbers managed to see a slight improvement in asset quality."

# Class Initialization (You can set default 'model_path=None' as your finetuned BERT model path while Initialization)
finbert = FinbertEmbedding()

word_embeddings = finbert.word_vector(text)
sentence_embedding = finbert.sentence_vector(text)

print("Text Tokens: ", finbert.tokens)
# Text Tokens:  ['another', 'psu', 'bank', ',', 'punjab', 'national', 'bank', 'which', 'also', 'reported', 'numbers', 'managed', 'to', 'see', 'a', 'slight', 'improvement', 'in', 'asset', 'quality', '.']

print ('Shape of Word Embeddings: %d x %d' % (len(word_embeddings), len(word_embeddings[0])))
# Shape of Word Embeddings: 21 x 768

print("Shape of Sentence Embedding = ",len(sentence_embedding))
# Shape of Sentence Embedding =  768
```

## Usage 2

A decent representation for a downstream task doesn't mean that it will be meaningful in terms of cosine distance. Since cosine distance is a linear space where all dimensions are weighted equally. if you want to use cosine distance anyway, then please focus on the rank not the absolute value.

Namely, do not use: <br>
  if cosine(A, B) > 0.9, then A and B are similar

Please consider the following instead: <br>
  if cosine(A, B) > cosine(A, C), then A is more similar to B than C.

```python
from finbert_embedding.embedding import FinbertEmbedding

text = "After stealing money from the bank vault, the bank robber was seen fishing on the Mississippi river bank."
finbert = FinbertEmbedding()
word_embeddings = finbert.word_vector(text)

from scipy.spatial.distance import cosine
diff_bank = 1 - cosine(word_embeddings[9], word_embeddings[18])
same_bank = 1 - cosine(word_embeddings[9], word_embeddings[5])

print('Vector similarity for similar bank meanings (bank vault & bank robber):  %.2f' % same_bank)
print('Vector similarity for different bank meanings (bank robber & river bank):  %.2f' % diff_bank)

# Vector similarity for similar bank meanings (bank vault & bank robber):  0.92
# Vector similarity for different bank meanings (bank robber & river bank):  0.64
```

### Warning

According to BERT author Jacob Devlin:
```I'm not sure what these vectors are, since BERT does not generate meaningful sentence vectors. It seems that this is doing average pooling over the word tokens to get a sentence vector, but we never suggested that this will generate meaningful sentence representations. And even if they are decent representations when fed into a DNN trained for a downstream task, it doesn't mean that they will be meaningful in terms of cosine distance. (Since cosine distance is a linear space where all dimensions are weighted equally).```

However, with the [CLS] token, it does become meaningful if the model has been fine-tuned, where the last hidden layer of this token is used as the “sentence vector” for downstream sequence classification task. This package encode sentence in similar manner.   

### To Do (Next Version)

* Extend it to give word embeddings for a paragram/Document (Currently, it takes one sentence as input). Chunkize your paragraph or text document into sentences using Spacy or NLTK before using finbert_embedding.
* Adding batch processing feature.
* More ways of handing OOVs (Currently, uses average of all tokens of a OOV word)
* Ingesting and extending it to more pre-trained financial models.

### Future Goal

* Create generic downstream framework using various FinBERT language model for any financial labelled text classifcation task like sentiment classification, Financial news classification, Financial Document classification.




%package -n python3-finbert-embedding
Summary:	Embeddings from Financial BERT
Provides:	python-finbert-embedding
BuildRequires:	python3-devel
BuildRequires:	python3-setuptools
BuildRequires:	python3-pip
%description -n python3-finbert-embedding
# finbert_embedding
Token and sentence level embeddings from FinBERT model (Financial Domain).

[BERT](https://arxiv.org/abs/1810.04805), published by Google, is conceptually simple and empirically powerful as it obtained state-of-the-art results on eleven natural language processing tasks.  

The objective of this project is to obtain the word or sentence embeddings from [FinBERT](https://github.com/ProsusAI/finBERT), pre-trained model by Dogu Tan Araci (University of Amsterdam). FinBERT, which is a BERT language model further trained on Financial news articles for adapting financial domain. It achieved the state-of-the-art on FiQA sentiment scoring and Financial PhraseBank dataset. Paper [here](https://arxiv.org/abs/1908.10063).

Instead of building and do fine-tuning for an end-to-end NLP model, You can directly utilize word embeddings from Financial BERT to build NLP models for various downstream tasks eg. Financial text classification, Text clustering, Extractive summarization or Entity extraction etc.



## Features
* Creates an abstraction to remove dealing with inferencing pre-trained FinBERT model.
* Require only two lines of code to get sentence/token-level encoding for a text sentence.
* The package takes care of OOVs (out of vocabulary) inherently.
* Downloads and installs FinBERT pre-trained model (first initialization, usage in next section).

## Install
(Recommended to create a conda env to have isolation and avoid dependency clashes)

```
pip install finbert-embedding==0.1.4
```

Note: If you get error in installing this package (common error with Tf): <br>

Installing collected packages: wrapt, tensorflow <br>
  Found existing installation: wrapt 1.10.11 <br>
ERROR: Cannot uninstall 'wrapt'. It is a distutils installed project....

then, just do this:
```
pip install wrapt --upgrade --ignore-installed
pip install finbert-embedding==0.1.4
```

## Usage 1

word embeddings generated are list of 768 dimensional embeddings for each word. <br>
sentence embedding generated is 768 dimensional embedding which is average of each token.

```python
from finbert_embedding.embedding import FinbertEmbedding

text = "Another PSU bank, Punjab National Bank which also reported numbers managed to see a slight improvement in asset quality."

# Class Initialization (You can set default 'model_path=None' as your finetuned BERT model path while Initialization)
finbert = FinbertEmbedding()

word_embeddings = finbert.word_vector(text)
sentence_embedding = finbert.sentence_vector(text)

print("Text Tokens: ", finbert.tokens)
# Text Tokens:  ['another', 'psu', 'bank', ',', 'punjab', 'national', 'bank', 'which', 'also', 'reported', 'numbers', 'managed', 'to', 'see', 'a', 'slight', 'improvement', 'in', 'asset', 'quality', '.']

print ('Shape of Word Embeddings: %d x %d' % (len(word_embeddings), len(word_embeddings[0])))
# Shape of Word Embeddings: 21 x 768

print("Shape of Sentence Embedding = ",len(sentence_embedding))
# Shape of Sentence Embedding =  768
```

## Usage 2

A decent representation for a downstream task doesn't mean that it will be meaningful in terms of cosine distance. Since cosine distance is a linear space where all dimensions are weighted equally. if you want to use cosine distance anyway, then please focus on the rank not the absolute value.

Namely, do not use: <br>
  if cosine(A, B) > 0.9, then A and B are similar

Please consider the following instead: <br>
  if cosine(A, B) > cosine(A, C), then A is more similar to B than C.

```python
from finbert_embedding.embedding import FinbertEmbedding

text = "After stealing money from the bank vault, the bank robber was seen fishing on the Mississippi river bank."
finbert = FinbertEmbedding()
word_embeddings = finbert.word_vector(text)

from scipy.spatial.distance import cosine
diff_bank = 1 - cosine(word_embeddings[9], word_embeddings[18])
same_bank = 1 - cosine(word_embeddings[9], word_embeddings[5])

print('Vector similarity for similar bank meanings (bank vault & bank robber):  %.2f' % same_bank)
print('Vector similarity for different bank meanings (bank robber & river bank):  %.2f' % diff_bank)

# Vector similarity for similar bank meanings (bank vault & bank robber):  0.92
# Vector similarity for different bank meanings (bank robber & river bank):  0.64
```

### Warning

According to BERT author Jacob Devlin:
```I'm not sure what these vectors are, since BERT does not generate meaningful sentence vectors. It seems that this is doing average pooling over the word tokens to get a sentence vector, but we never suggested that this will generate meaningful sentence representations. And even if they are decent representations when fed into a DNN trained for a downstream task, it doesn't mean that they will be meaningful in terms of cosine distance. (Since cosine distance is a linear space where all dimensions are weighted equally).```

However, with the [CLS] token, it does become meaningful if the model has been fine-tuned, where the last hidden layer of this token is used as the “sentence vector” for downstream sequence classification task. This package encode sentence in similar manner.   

### To Do (Next Version)

* Extend it to give word embeddings for a paragram/Document (Currently, it takes one sentence as input). Chunkize your paragraph or text document into sentences using Spacy or NLTK before using finbert_embedding.
* Adding batch processing feature.
* More ways of handing OOVs (Currently, uses average of all tokens of a OOV word)
* Ingesting and extending it to more pre-trained financial models.

### Future Goal

* Create generic downstream framework using various FinBERT language model for any financial labelled text classifcation task like sentiment classification, Financial news classification, Financial Document classification.




%package help
Summary:	Development documents and examples for finbert-embedding
Provides:	python3-finbert-embedding-doc
%description help
# finbert_embedding
Token and sentence level embeddings from FinBERT model (Financial Domain).

[BERT](https://arxiv.org/abs/1810.04805), published by Google, is conceptually simple and empirically powerful as it obtained state-of-the-art results on eleven natural language processing tasks.  

The objective of this project is to obtain the word or sentence embeddings from [FinBERT](https://github.com/ProsusAI/finBERT), pre-trained model by Dogu Tan Araci (University of Amsterdam). FinBERT, which is a BERT language model further trained on Financial news articles for adapting financial domain. It achieved the state-of-the-art on FiQA sentiment scoring and Financial PhraseBank dataset. Paper [here](https://arxiv.org/abs/1908.10063).

Instead of building and do fine-tuning for an end-to-end NLP model, You can directly utilize word embeddings from Financial BERT to build NLP models for various downstream tasks eg. Financial text classification, Text clustering, Extractive summarization or Entity extraction etc.



## Features
* Creates an abstraction to remove dealing with inferencing pre-trained FinBERT model.
* Require only two lines of code to get sentence/token-level encoding for a text sentence.
* The package takes care of OOVs (out of vocabulary) inherently.
* Downloads and installs FinBERT pre-trained model (first initialization, usage in next section).

## Install
(Recommended to create a conda env to have isolation and avoid dependency clashes)

```
pip install finbert-embedding==0.1.4
```

Note: If you get error in installing this package (common error with Tf): <br>

Installing collected packages: wrapt, tensorflow <br>
  Found existing installation: wrapt 1.10.11 <br>
ERROR: Cannot uninstall 'wrapt'. It is a distutils installed project....

then, just do this:
```
pip install wrapt --upgrade --ignore-installed
pip install finbert-embedding==0.1.4
```

## Usage 1

word embeddings generated are list of 768 dimensional embeddings for each word. <br>
sentence embedding generated is 768 dimensional embedding which is average of each token.

```python
from finbert_embedding.embedding import FinbertEmbedding

text = "Another PSU bank, Punjab National Bank which also reported numbers managed to see a slight improvement in asset quality."

# Class Initialization (You can set default 'model_path=None' as your finetuned BERT model path while Initialization)
finbert = FinbertEmbedding()

word_embeddings = finbert.word_vector(text)
sentence_embedding = finbert.sentence_vector(text)

print("Text Tokens: ", finbert.tokens)
# Text Tokens:  ['another', 'psu', 'bank', ',', 'punjab', 'national', 'bank', 'which', 'also', 'reported', 'numbers', 'managed', 'to', 'see', 'a', 'slight', 'improvement', 'in', 'asset', 'quality', '.']

print ('Shape of Word Embeddings: %d x %d' % (len(word_embeddings), len(word_embeddings[0])))
# Shape of Word Embeddings: 21 x 768

print("Shape of Sentence Embedding = ",len(sentence_embedding))
# Shape of Sentence Embedding =  768
```

## Usage 2

A decent representation for a downstream task doesn't mean that it will be meaningful in terms of cosine distance. Since cosine distance is a linear space where all dimensions are weighted equally. if you want to use cosine distance anyway, then please focus on the rank not the absolute value.

Namely, do not use: <br>
  if cosine(A, B) > 0.9, then A and B are similar

Please consider the following instead: <br>
  if cosine(A, B) > cosine(A, C), then A is more similar to B than C.

```python
from finbert_embedding.embedding import FinbertEmbedding

text = "After stealing money from the bank vault, the bank robber was seen fishing on the Mississippi river bank."
finbert = FinbertEmbedding()
word_embeddings = finbert.word_vector(text)

from scipy.spatial.distance import cosine
diff_bank = 1 - cosine(word_embeddings[9], word_embeddings[18])
same_bank = 1 - cosine(word_embeddings[9], word_embeddings[5])

print('Vector similarity for similar bank meanings (bank vault & bank robber):  %.2f' % same_bank)
print('Vector similarity for different bank meanings (bank robber & river bank):  %.2f' % diff_bank)

# Vector similarity for similar bank meanings (bank vault & bank robber):  0.92
# Vector similarity for different bank meanings (bank robber & river bank):  0.64
```

### Warning

According to BERT author Jacob Devlin:
```I'm not sure what these vectors are, since BERT does not generate meaningful sentence vectors. It seems that this is doing average pooling over the word tokens to get a sentence vector, but we never suggested that this will generate meaningful sentence representations. And even if they are decent representations when fed into a DNN trained for a downstream task, it doesn't mean that they will be meaningful in terms of cosine distance. (Since cosine distance is a linear space where all dimensions are weighted equally).```

However, with the [CLS] token, it does become meaningful if the model has been fine-tuned, where the last hidden layer of this token is used as the “sentence vector” for downstream sequence classification task. This package encode sentence in similar manner.   

### To Do (Next Version)

* Extend it to give word embeddings for a paragram/Document (Currently, it takes one sentence as input). Chunkize your paragraph or text document into sentences using Spacy or NLTK before using finbert_embedding.
* Adding batch processing feature.
* More ways of handing OOVs (Currently, uses average of all tokens of a OOV word)
* Ingesting and extending it to more pre-trained financial models.

### Future Goal

* Create generic downstream framework using various FinBERT language model for any financial labelled text classifcation task like sentiment classification, Financial news classification, Financial Document classification.




%prep
%autosetup -n finbert-embedding-0.1.5

%build
%py3_build

%install
%py3_install
install -d -m755 %{buildroot}/%{_pkgdocdir}
if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi
if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi
if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi
if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi
pushd %{buildroot}
if [ -d usr/lib ]; then
	find usr/lib -type f -printf "\"/%h/%f\"\n" >> filelist.lst
fi
if [ -d usr/lib64 ]; then
	find usr/lib64 -type f -printf "\"/%h/%f\"\n" >> filelist.lst
fi
if [ -d usr/bin ]; then
	find usr/bin -type f -printf "\"/%h/%f\"\n" >> filelist.lst
fi
if [ -d usr/sbin ]; then
	find usr/sbin -type f -printf "\"/%h/%f\"\n" >> filelist.lst
fi
touch doclist.lst
if [ -d usr/share/man ]; then
	find usr/share/man -type f -printf "\"/%h/%f.gz\"\n" >> doclist.lst
fi
popd
mv %{buildroot}/filelist.lst .
mv %{buildroot}/doclist.lst .

%files -n python3-finbert-embedding -f filelist.lst
%dir %{python3_sitelib}/*

%files help -f doclist.lst
%{_docdir}/*

%changelog
* Thu Jun 08 2023 Python_Bot <Python_Bot@openeuler.org> - 0.1.5-1
- Package Spec generated