A statistical simulation technique to develop and evaluate conversational agents Articles uri icon

publication date

  • September 2013

start page

  • 355

end page

  • 371

issue

  • 4

volume

  • 26

International Standard Serial Number (ISSN)

  • 0921-7126

Electronic International Standard Serial Number (EISSN)

  • 1875-8452

abstract

  • In this paper, we present a technique for developing user simulators which are able to interact and evaluate conversational agents. Our technique is based on a statistical model that is automatically learned from a dialog corpus. This model is used by the user simulator to provide the next answer taking into account the complete history of the interaction. The main objective of our proposal is not only to evaluate the conversational agent, but also to improve this agent by employing the simulated dialogs to learn a better dialog model. We have applied this technique to design and evaluate a conversational agent which provides academic information in a multi-agent system. The results of the evaluation show that the proposed user simulation methodology can be used not only to evaluate conversational agents but also to explore new enhanced dialog strategies, thereby allowing the conversational agent to reduce the time needed to complete the dialogs and automatically detect new valid paths to achieve each of the required objectives defined for the task.

subjects

  • Computer Science

keywords

  • conversational agents; user simulation; statistical methodologies; multiagent systems