QA System Evaluation

Evaluation

Evaluation of a QA system is done using three parameters: precision, recall, and F-measure. Precision specifies how many of the answers given by the are correct. It is computed as
Precision(q) = number of correct answers by the system /number of answers generated by the system . The next parameter is recall. It is computed as: Recall(q) = number of correct answers by the system / number of expected correct answers from the system
The F-measure is the weighted average between the precision and recall and it is computed as follows:
F-measure =( 2 *Precision * Recall)/(Precision + Recall )

LITERATURE REVIEW
In this section we describe some of the QA system

AQUA: Ontology Based Question Answering System (Maria Vargas-Vera and

On failure automatically derive an answer, the system generates clarification dialogs for the user. The choice of user is saved for training purpose to improve the performance of the system over time. For performing QA the system initially identifies and verifies the concepts of the ontology. Then it generates the SPARQL query. Finally the system identifies the answer type result is formed. Stanford Parser is used for generating syntactic parse tree .Mapping of user query and ontology is performed in two ways-automatically and engaging the user .

QASYO A Question Answering System for YAGO Ontology (Abdullah M. Moussa et al,2011); QASYO joins various techniques in a modest way to make sense of NL and map them to semantic markup .It uses WordNet and ontology-based services.It uses YAGO ontology. The oracle database and visual C# was used for developing YAGO ontology. The system consists of four phases : Question Classifier, Linguistic component(maps the NL query to linguistic-triple I.e subject, relation and object triple), Query generator(query is generated using the linguistic triple),Query