Answer extraction for simple and complex questions
Date
2008
Authors
Joty, Shafiq Rayhan
University of Lethbridge. Faculty of Arts and Science
Journal Title
Journal ISSN
Volume Title
Publisher
Lethbridge, Alta. : University of Lethbridge, Faculty of Arts and Science, 2008
Abstract
When a user is served with a ranked list of relevant documents by the standard document
search engines, his search task is usually not over. He has to go through the entire
document contents to find the precise piece of information he was looking for. Question
answering, which is the retrieving of answers to natural language questions from a document
collection, tries to remove the onus on the end-user by providing direct access to
relevant information. This thesis is concerned with open-domain question answering. We
have considered both simple and complex questions. Simple questions (i.e. factoid and
list) are easier to answer than questions that have complex information needs and require
inferencing and synthesizing information from multiple documents.
Our question answering system for simple questions is based on question classification
and document tagging. Question classification extracts useful information (i.e. answer
type) about how to answer the question and document tagging extracts useful information
from the documents, which is used in finding the answer to the question.
For complex questions, we experimented with both empirical and machine learning approaches.
We extracted several features of different types (i.e. lexical, lexical semantic,
syntactic and semantic) for each of the sentences in the document collection in order to
measure its relevancy to the user query. One hill climbing local search strategy is used
to fine-tune the feature-weights. We also experimented with two unsupervised machine
learning techniques: k-means and Expectation Maximization (EM) algorithms and evaluated
their performance. For all these methods, we have shown the effects of different kinds
of features.
Description
xi, 214 leaves : ill. (some col.) ; 29 cm. --
Keywords
Dissertations, Academic , Natural language processing (Computer science) , Keyword searching