Latent Semantic Analysis (LSA)

Latent Semantic Analysis
Latent Semantic Analysis (LSA) [44] presents a technique for representing a text T in a semantic space based on corpus statistics. Latent Semantic Analysis (LSA) [44] is a purely statistical technique, which leverages word co-occurrence information from a large unlabeled large corpus of text. In LSA, a set of representative words needs to be identified from a large number of contexts. A word by context matrix is formed based on the presence of words in contexts.
LSA does not rely on any human-organized knowledge; rather, it “learns” its representation by applying Singular Value Decomposition (SVD) to the words-by-documents co-occurrence matrix. LSA is essentially a dimensionality reduction technique that identifies

The diagonal singular matrix is truncated by deleting small singular values. In this way, the dimensionality is reduced. The original word by context matrix is then reconstructed from the reduced dimensional space. Through the process of decomposition and reconstruction, LSA acquires word knowledge that spreads in contexts. When LSA is used to compute sentence similarity, a vector for each sentence is formed in the reduced dimension space; similarity is then measured by computing the similarity of these two vectors [92].
I. Because of the computational limit of SVD, the dimension size of the word by context matrix is limited to the several hundred. As the input sentences may be from an unconstrained domain (and thus not represented in the contexts) some important words from the input sentences may not be included in the LSA dimension space.
II. Secondly, the dimension is fixed and so the vector is fixed and is thus likely to be a very sparse representation of a short text such as a

The idea underlying ESA is to represent and compare texts (from single terms to entire documents) as vectors in a high dimensional concept space. ESA was introduced as an approach to compute the semantic relatedness of terms or short phrases. ESA is a vectorial reorientation of entire document that uses a document as knowledge base. The ESA representation of a real-world document d is a vector dESA whose elements are the cosine similarities between d and all documents in a collection DI, called index collection here. The supposed rationale of the ESA retrieval model is that each document in DI functions as a semantic concept to which the original document d is compared: dESA is understood as a projection of d into the concept space spanned by DI. The semantic relatedness between two documents d1 and d2 is computed by the cosine similarity between the ESA vectors of d1 and