Robot Socrates: Contradiction Identification with Minimum Questioning (thesis)

Abstract

I develop a concept of computer-assisted Socratic dialogue, aimed to enable productive and efficient argumentation. The goal is a framework for argumentation that is as rigorous as a formal debate, yet as convenient as an online quiz. A human specifies questions and answer choices, and designates certain sets of answer choices as contradictory. The computer's task is to end the dialogue asking as few questions as possible, by eliciting a set of answers from a questionee that either includes a contradiction, or eliminates the possibility of any contradiction. I formalize this minimum questioning problem, and prove it is NP-hard. I then analyze the problem in terms of a trade-off between asking questions that are most likely to achieve immediate progress towards termination, and asking questions which are most likely to yield knowledge which will lead towards greater progress towards termination in the future. I develop a greedy algorithm that maximizes a multi-objective utility function embodying this trade-off, and evaluate its performance on a set of randomly generated dialogues and questionees. Results suggest that favoring future progress over immediate progress is a better strategy only in contrived cases. The algorithm is able to adapt well between between contradictory and non-contradictory cases, requiring on average only a fifth as many questions as random question selection in the contradictory case, and less than half as many questions in the non-contradictory case.