Semantic Textual Similarity

Abstract

We present a new approach to Machine Translation evaluation based on the recently defined task Semantic Textual Similarity. Our approach explores lexical, syntactic and semantic machine translation evaluation metrics combined with distributional and knowledge-based word similarity metrics.

Semantic sentence similarity evaluates give a progressively important role in text-related research and applications in areas such as text mining, Web page retrieval, and dialogue systems. This paper focuses mainly on computing the textual similarity between short texts of sentence length. It presents an algorithm that takes account of semantic textual information and word order information implied in the sentences. The semantic textual similarity

However, recently few attempts have been made for the use of semantic information for MT evaluation. Moreover, only one paper work has been published about semantic equivalence (known as Textual Entailment) of texts for MT evaluation.

Recent applications of natural language processing(NLP) present a need for an effective method to evaluate the similarity between short texts or sentences [6]. In text mining, sentence similarity is used as a criterion to find unseen knowledge from textual databases [2]. In addition, the NLP for the incorporation of short-text similarity is beneficial to applications such as text summarization [3], text categorization [4], and machine translation [5].

For the huge studies in the field of Machine Translation Evaluation, we shortly mention a few worked attempts to evaluate MT based on semantic features, which we count most recent and which are important. In this piece of work, we propose an improved metric, based on TE features, that shows to what extent a candidate sentence is equivalent to a reference and some related work in order to explore the particular evaluation in computing sentence

There are many approaches to the evaluation of MT systems, ranging from manually measuring MT system output to automatically analyzing quality against texts that were translated by humans previously.

3.1 MT evaluation

MT evaluation with Sagan is based on a core development to approach the Semantic Textual Similarity task (STS). The pilot task STS was recently produced in Semeval 2012 (Aguirre et al., 2012) and has an as main goal is to measure the degree of semantic equivalence between two text fragments.

The goal of the RTE task (Bentivogli et al., 2009) is determining whether the meaning of a hypothesis H can be concluded from a text T. Thus, TE(Textual Entailment) is a directional task and we say that T entails H, if a person reading T would infer that H is most likely true. The main difference with STS is that STS consists of explaining how two fragment texts are similar, in a range from 5 (total semantic equivalence) to 0 (no relation). Thus, STS mainly differs from TE in that the classification is graded instead of a binary number. In this manner, STS is filling the gap in