%global _empty_manifest_terminate_build 0 Name: python-finbert-embedding Version: 0.1.5 Release: 1 Summary: Embeddings from Financial BERT License: MIT License URL: https://github.com/abhijeet3922/finbert_embedding Source0: https://mirrors.aliyun.com/pypi/web/packages/a1/54/beeb2b0f98bcb1a2e560ae29a728fd518526300bf2abc423cd8dc74d36d8/finbert-embedding-0.1.5.tar.gz BuildArch: noarch Requires: python3-torch Requires: python3-pytorch-pretrained-bert Requires: python3-tensorflow %description # finbert_embedding Token and sentence level embeddings from FinBERT model (Financial Domain). [BERT](https://arxiv.org/abs/1810.04805), published by Google, is conceptually simple and empirically powerful as it obtained state-of-the-art results on eleven natural language processing tasks. The objective of this project is to obtain the word or sentence embeddings from [FinBERT](https://github.com/ProsusAI/finBERT), pre-trained model by Dogu Tan Araci (University of Amsterdam). FinBERT, which is a BERT language model further trained on Financial news articles for adapting financial domain. It achieved the state-of-the-art on FiQA sentiment scoring and Financial PhraseBank dataset. Paper [here](https://arxiv.org/abs/1908.10063). Instead of building and do fine-tuning for an end-to-end NLP model, You can directly utilize word embeddings from Financial BERT to build NLP models for various downstream tasks eg. Financial text classification, Text clustering, Extractive summarization or Entity extraction etc. ## Features * Creates an abstraction to remove dealing with inferencing pre-trained FinBERT model. * Require only two lines of code to get sentence/token-level encoding for a text sentence. * The package takes care of OOVs (out of vocabulary) inherently. * Downloads and installs FinBERT pre-trained model (first initialization, usage in next section). ## Install (Recommended to create a conda env to have isolation and avoid dependency clashes) ``` pip install finbert-embedding==0.1.4 ``` Note: If you get error in installing this package (common error with Tf):
Installing collected packages: wrapt, tensorflow
Found existing installation: wrapt 1.10.11
ERROR: Cannot uninstall 'wrapt'. It is a distutils installed project.... then, just do this: ``` pip install wrapt --upgrade --ignore-installed pip install finbert-embedding==0.1.4 ``` ## Usage 1 word embeddings generated are list of 768 dimensional embeddings for each word.
sentence embedding generated is 768 dimensional embedding which is average of each token. ```python from finbert_embedding.embedding import FinbertEmbedding text = "Another PSU bank, Punjab National Bank which also reported numbers managed to see a slight improvement in asset quality." # Class Initialization (You can set default 'model_path=None' as your finetuned BERT model path while Initialization) finbert = FinbertEmbedding() word_embeddings = finbert.word_vector(text) sentence_embedding = finbert.sentence_vector(text) print("Text Tokens: ", finbert.tokens) # Text Tokens: ['another', 'psu', 'bank', ',', 'punjab', 'national', 'bank', 'which', 'also', 'reported', 'numbers', 'managed', 'to', 'see', 'a', 'slight', 'improvement', 'in', 'asset', 'quality', '.'] print ('Shape of Word Embeddings: %d x %d' % (len(word_embeddings), len(word_embeddings[0]))) # Shape of Word Embeddings: 21 x 768 print("Shape of Sentence Embedding = ",len(sentence_embedding)) # Shape of Sentence Embedding = 768 ``` ## Usage 2 A decent representation for a downstream task doesn't mean that it will be meaningful in terms of cosine distance. Since cosine distance is a linear space where all dimensions are weighted equally. if you want to use cosine distance anyway, then please focus on the rank not the absolute value. Namely, do not use:
if cosine(A, B) > 0.9, then A and B are similar Please consider the following instead:
if cosine(A, B) > cosine(A, C), then A is more similar to B than C. ```python from finbert_embedding.embedding import FinbertEmbedding text = "After stealing money from the bank vault, the bank robber was seen fishing on the Mississippi river bank." finbert = FinbertEmbedding() word_embeddings = finbert.word_vector(text) from scipy.spatial.distance import cosine diff_bank = 1 - cosine(word_embeddings[9], word_embeddings[18]) same_bank = 1 - cosine(word_embeddings[9], word_embeddings[5]) print('Vector similarity for similar bank meanings (bank vault & bank robber): %.2f' % same_bank) print('Vector similarity for different bank meanings (bank robber & river bank): %.2f' % diff_bank) # Vector similarity for similar bank meanings (bank vault & bank robber): 0.92 # Vector similarity for different bank meanings (bank robber & river bank): 0.64 ``` ### Warning According to BERT author Jacob Devlin: ```I'm not sure what these vectors are, since BERT does not generate meaningful sentence vectors. It seems that this is doing average pooling over the word tokens to get a sentence vector, but we never suggested that this will generate meaningful sentence representations. And even if they are decent representations when fed into a DNN trained for a downstream task, it doesn't mean that they will be meaningful in terms of cosine distance. (Since cosine distance is a linear space where all dimensions are weighted equally).``` However, with the [CLS] token, it does become meaningful if the model has been fine-tuned, where the last hidden layer of this token is used as the “sentence vector” for downstream sequence classification task. This package encode sentence in similar manner. ### To Do (Next Version) * Extend it to give word embeddings for a paragram/Document (Currently, it takes one sentence as input). Chunkize your paragraph or text document into sentences using Spacy or NLTK before using finbert_embedding. * Adding batch processing feature. * More ways of handing OOVs (Currently, uses average of all tokens of a OOV word) * Ingesting and extending it to more pre-trained financial models. ### Future Goal * Create generic downstream framework using various FinBERT language model for any financial labelled text classifcation task like sentiment classification, Financial news classification, Financial Document classification. %package -n python3-finbert-embedding Summary: Embeddings from Financial BERT Provides: python-finbert-embedding BuildRequires: python3-devel BuildRequires: python3-setuptools BuildRequires: python3-pip %description -n python3-finbert-embedding # finbert_embedding Token and sentence level embeddings from FinBERT model (Financial Domain). [BERT](https://arxiv.org/abs/1810.04805), published by Google, is conceptually simple and empirically powerful as it obtained state-of-the-art results on eleven natural language processing tasks. The objective of this project is to obtain the word or sentence embeddings from [FinBERT](https://github.com/ProsusAI/finBERT), pre-trained model by Dogu Tan Araci (University of Amsterdam). FinBERT, which is a BERT language model further trained on Financial news articles for adapting financial domain. It achieved the state-of-the-art on FiQA sentiment scoring and Financial PhraseBank dataset. Paper [here](https://arxiv.org/abs/1908.10063). Instead of building and do fine-tuning for an end-to-end NLP model, You can directly utilize word embeddings from Financial BERT to build NLP models for various downstream tasks eg. Financial text classification, Text clustering, Extractive summarization or Entity extraction etc. ## Features * Creates an abstraction to remove dealing with inferencing pre-trained FinBERT model. * Require only two lines of code to get sentence/token-level encoding for a text sentence. * The package takes care of OOVs (out of vocabulary) inherently. * Downloads and installs FinBERT pre-trained model (first initialization, usage in next section). ## Install (Recommended to create a conda env to have isolation and avoid dependency clashes) ``` pip install finbert-embedding==0.1.4 ``` Note: If you get error in installing this package (common error with Tf):
Installing collected packages: wrapt, tensorflow
Found existing installation: wrapt 1.10.11
ERROR: Cannot uninstall 'wrapt'. It is a distutils installed project.... then, just do this: ``` pip install wrapt --upgrade --ignore-installed pip install finbert-embedding==0.1.4 ``` ## Usage 1 word embeddings generated are list of 768 dimensional embeddings for each word.
sentence embedding generated is 768 dimensional embedding which is average of each token. ```python from finbert_embedding.embedding import FinbertEmbedding text = "Another PSU bank, Punjab National Bank which also reported numbers managed to see a slight improvement in asset quality." # Class Initialization (You can set default 'model_path=None' as your finetuned BERT model path while Initialization) finbert = FinbertEmbedding() word_embeddings = finbert.word_vector(text) sentence_embedding = finbert.sentence_vector(text) print("Text Tokens: ", finbert.tokens) # Text Tokens: ['another', 'psu', 'bank', ',', 'punjab', 'national', 'bank', 'which', 'also', 'reported', 'numbers', 'managed', 'to', 'see', 'a', 'slight', 'improvement', 'in', 'asset', 'quality', '.'] print ('Shape of Word Embeddings: %d x %d' % (len(word_embeddings), len(word_embeddings[0]))) # Shape of Word Embeddings: 21 x 768 print("Shape of Sentence Embedding = ",len(sentence_embedding)) # Shape of Sentence Embedding = 768 ``` ## Usage 2 A decent representation for a downstream task doesn't mean that it will be meaningful in terms of cosine distance. Since cosine distance is a linear space where all dimensions are weighted equally. if you want to use cosine distance anyway, then please focus on the rank not the absolute value. Namely, do not use:
if cosine(A, B) > 0.9, then A and B are similar Please consider the following instead:
if cosine(A, B) > cosine(A, C), then A is more similar to B than C. ```python from finbert_embedding.embedding import FinbertEmbedding text = "After stealing money from the bank vault, the bank robber was seen fishing on the Mississippi river bank." finbert = FinbertEmbedding() word_embeddings = finbert.word_vector(text) from scipy.spatial.distance import cosine diff_bank = 1 - cosine(word_embeddings[9], word_embeddings[18]) same_bank = 1 - cosine(word_embeddings[9], word_embeddings[5]) print('Vector similarity for similar bank meanings (bank vault & bank robber): %.2f' % same_bank) print('Vector similarity for different bank meanings (bank robber & river bank): %.2f' % diff_bank) # Vector similarity for similar bank meanings (bank vault & bank robber): 0.92 # Vector similarity for different bank meanings (bank robber & river bank): 0.64 ``` ### Warning According to BERT author Jacob Devlin: ```I'm not sure what these vectors are, since BERT does not generate meaningful sentence vectors. It seems that this is doing average pooling over the word tokens to get a sentence vector, but we never suggested that this will generate meaningful sentence representations. And even if they are decent representations when fed into a DNN trained for a downstream task, it doesn't mean that they will be meaningful in terms of cosine distance. (Since cosine distance is a linear space where all dimensions are weighted equally).``` However, with the [CLS] token, it does become meaningful if the model has been fine-tuned, where the last hidden layer of this token is used as the “sentence vector” for downstream sequence classification task. This package encode sentence in similar manner. ### To Do (Next Version) * Extend it to give word embeddings for a paragram/Document (Currently, it takes one sentence as input). Chunkize your paragraph or text document into sentences using Spacy or NLTK before using finbert_embedding. * Adding batch processing feature. * More ways of handing OOVs (Currently, uses average of all tokens of a OOV word) * Ingesting and extending it to more pre-trained financial models. ### Future Goal * Create generic downstream framework using various FinBERT language model for any financial labelled text classifcation task like sentiment classification, Financial news classification, Financial Document classification. %package help Summary: Development documents and examples for finbert-embedding Provides: python3-finbert-embedding-doc %description help # finbert_embedding Token and sentence level embeddings from FinBERT model (Financial Domain). [BERT](https://arxiv.org/abs/1810.04805), published by Google, is conceptually simple and empirically powerful as it obtained state-of-the-art results on eleven natural language processing tasks. The objective of this project is to obtain the word or sentence embeddings from [FinBERT](https://github.com/ProsusAI/finBERT), pre-trained model by Dogu Tan Araci (University of Amsterdam). FinBERT, which is a BERT language model further trained on Financial news articles for adapting financial domain. It achieved the state-of-the-art on FiQA sentiment scoring and Financial PhraseBank dataset. Paper [here](https://arxiv.org/abs/1908.10063). Instead of building and do fine-tuning for an end-to-end NLP model, You can directly utilize word embeddings from Financial BERT to build NLP models for various downstream tasks eg. Financial text classification, Text clustering, Extractive summarization or Entity extraction etc. ## Features * Creates an abstraction to remove dealing with inferencing pre-trained FinBERT model. * Require only two lines of code to get sentence/token-level encoding for a text sentence. * The package takes care of OOVs (out of vocabulary) inherently. * Downloads and installs FinBERT pre-trained model (first initialization, usage in next section). ## Install (Recommended to create a conda env to have isolation and avoid dependency clashes) ``` pip install finbert-embedding==0.1.4 ``` Note: If you get error in installing this package (common error with Tf):
Installing collected packages: wrapt, tensorflow
Found existing installation: wrapt 1.10.11
ERROR: Cannot uninstall 'wrapt'. It is a distutils installed project.... then, just do this: ``` pip install wrapt --upgrade --ignore-installed pip install finbert-embedding==0.1.4 ``` ## Usage 1 word embeddings generated are list of 768 dimensional embeddings for each word.
sentence embedding generated is 768 dimensional embedding which is average of each token. ```python from finbert_embedding.embedding import FinbertEmbedding text = "Another PSU bank, Punjab National Bank which also reported numbers managed to see a slight improvement in asset quality." # Class Initialization (You can set default 'model_path=None' as your finetuned BERT model path while Initialization) finbert = FinbertEmbedding() word_embeddings = finbert.word_vector(text) sentence_embedding = finbert.sentence_vector(text) print("Text Tokens: ", finbert.tokens) # Text Tokens: ['another', 'psu', 'bank', ',', 'punjab', 'national', 'bank', 'which', 'also', 'reported', 'numbers', 'managed', 'to', 'see', 'a', 'slight', 'improvement', 'in', 'asset', 'quality', '.'] print ('Shape of Word Embeddings: %d x %d' % (len(word_embeddings), len(word_embeddings[0]))) # Shape of Word Embeddings: 21 x 768 print("Shape of Sentence Embedding = ",len(sentence_embedding)) # Shape of Sentence Embedding = 768 ``` ## Usage 2 A decent representation for a downstream task doesn't mean that it will be meaningful in terms of cosine distance. Since cosine distance is a linear space where all dimensions are weighted equally. if you want to use cosine distance anyway, then please focus on the rank not the absolute value. Namely, do not use:
if cosine(A, B) > 0.9, then A and B are similar Please consider the following instead:
if cosine(A, B) > cosine(A, C), then A is more similar to B than C. ```python from finbert_embedding.embedding import FinbertEmbedding text = "After stealing money from the bank vault, the bank robber was seen fishing on the Mississippi river bank." finbert = FinbertEmbedding() word_embeddings = finbert.word_vector(text) from scipy.spatial.distance import cosine diff_bank = 1 - cosine(word_embeddings[9], word_embeddings[18]) same_bank = 1 - cosine(word_embeddings[9], word_embeddings[5]) print('Vector similarity for similar bank meanings (bank vault & bank robber): %.2f' % same_bank) print('Vector similarity for different bank meanings (bank robber & river bank): %.2f' % diff_bank) # Vector similarity for similar bank meanings (bank vault & bank robber): 0.92 # Vector similarity for different bank meanings (bank robber & river bank): 0.64 ``` ### Warning According to BERT author Jacob Devlin: ```I'm not sure what these vectors are, since BERT does not generate meaningful sentence vectors. It seems that this is doing average pooling over the word tokens to get a sentence vector, but we never suggested that this will generate meaningful sentence representations. And even if they are decent representations when fed into a DNN trained for a downstream task, it doesn't mean that they will be meaningful in terms of cosine distance. (Since cosine distance is a linear space where all dimensions are weighted equally).``` However, with the [CLS] token, it does become meaningful if the model has been fine-tuned, where the last hidden layer of this token is used as the “sentence vector” for downstream sequence classification task. This package encode sentence in similar manner. ### To Do (Next Version) * Extend it to give word embeddings for a paragram/Document (Currently, it takes one sentence as input). Chunkize your paragraph or text document into sentences using Spacy or NLTK before using finbert_embedding. * Adding batch processing feature. * More ways of handing OOVs (Currently, uses average of all tokens of a OOV word) * Ingesting and extending it to more pre-trained financial models. ### Future Goal * Create generic downstream framework using various FinBERT language model for any financial labelled text classifcation task like sentiment classification, Financial news classification, Financial Document classification. %prep %autosetup -n finbert-embedding-0.1.5 %build %py3_build %install %py3_install install -d -m755 %{buildroot}/%{_pkgdocdir} if [ -d doc ]; then cp -arf doc %{buildroot}/%{_pkgdocdir}; fi if [ -d docs ]; then cp -arf docs %{buildroot}/%{_pkgdocdir}; fi if [ -d example ]; then cp -arf example %{buildroot}/%{_pkgdocdir}; fi if [ -d examples ]; then cp -arf examples %{buildroot}/%{_pkgdocdir}; fi pushd %{buildroot} if [ -d usr/lib ]; then find usr/lib -type f -printf "\"/%h/%f\"\n" >> filelist.lst fi if [ -d usr/lib64 ]; then find usr/lib64 -type f -printf "\"/%h/%f\"\n" >> filelist.lst fi if [ -d usr/bin ]; then find usr/bin -type f -printf "\"/%h/%f\"\n" >> filelist.lst fi if [ -d usr/sbin ]; then find usr/sbin -type f -printf "\"/%h/%f\"\n" >> filelist.lst fi touch doclist.lst if [ -d usr/share/man ]; then find usr/share/man -type f -printf "\"/%h/%f.gz\"\n" >> doclist.lst fi popd mv %{buildroot}/filelist.lst . mv %{buildroot}/doclist.lst . %files -n python3-finbert-embedding -f filelist.lst %dir %{python3_sitelib}/* %files help -f doclist.lst %{_docdir}/* %changelog * Thu Jun 08 2023 Python_Bot - 0.1.5-1 - Package Spec generated