About

I am a Research Associate Professor in the Department of Computer Science at the University of Southern California and at the USC Institute for Creative Technologies. I am a member of the Natural Language Dialogue Group at the USC Institute for Creative Technologies. My research focuses on natural language processing, dialogue systems, and computational linguistics.

Research topics

• Computer Science
• Natural Language Processing
• Artificial Intelligence
• Social psychology
• Engineering
• Management science
• Mechanical engineering
• World Wide Web
• Data science
• Engineering ethics
• Linguistics
• Programming language
• Psychology
• Speech recognition

Selected publications

• Disentangling Approaches to Conversation Disentanglement: Fine-Tune or Learn from Scratch?

  2026-04-30

  articleSenior author
  PublisherDOI

• Comparing Pre-Trained Embeddings and Domain-Independent Features for Regression-Based Evaluation of Task-Oriented Dialogue Systems

  2024-01-01 · 2 citations

  articleOpen access1st authorCorresponding

  We use Gaussian Process Regression to predict different types of ratings provided by users after interacting with various task-oriented dialogue systems.We compare the performance of domain-independent dialogue features (e.g., duration, number of filled slots, number of confirmed slots, word error rate) with pre-trained dialogue embeddings.These pre-trained dialogue embeddings are computed by averaging over sentence embeddings in a dialogue.Sentence embeddings are created using various models based on sentence transformers (appearing on the Hugging Face Massive Text Embedding Benchmark leaderboard) or by averaging over BERT word embeddings (varying the BERT layers used).We also compare pre-trained embeddings extracted from human transcriptions with pre-trained embeddings extracted from speech recognition outputs, to determine the robustness of these models to errors.Our results show that overall, for most types of user satisfaction ratings and advanced/recent (or sometimes less advanced/recent) pre-trained embedding models, using only pre-trained embeddings outperforms using only domain-independent features.However, this pattern varies depending on the type of rating and the embedding model used.Also, pre-trained embeddings are found to be robust to speech recognition errors, more advanced/recent embedding models do not always perform better than less advanced/recent ones, and larger models do not necessarily outperform smaller ones.The best prediction performance is achieved by combining pre-trained embeddings with domain-independent features.

  PublisherOA PDFDOI

• Report from the NSF Future Directions Workshop on Automatic Evaluation of Dialog: Research Directions and Challenges

  arXiv (Cornell University) · 2022 · 24 citations

  • Computer Science
  • Computer Science
  • Data science

  This is a report on the NSF Future Directions Workshop on Automatic Evaluation of Dialog. The workshop explored the current state of the art along with its limitations and suggested promising directions for future work in this important and very rapidly changing area of research.

  PublisherOA PDFDOI

• Evaluation of Off-the-shelf Speech Recognizers on Different Accents in a Dialogue Domain

  2022-06-01

  articleOpen access
  PublisherDOI

• Strategy-level Entrainment of Dialogue System Users in a Creative Visual Reference Resolution Task

  2022-06-01

  articleOpen accessSenior author
  PublisherDOI

• An Interactive Image Editing System Using an Uncertainty-Based Confirmation Strategy

  IEEE Access · 2020-01-01 · 2 citations

  articleOpen access

  We propose an interactive image editing system that has a confirmation dialogue strategy using an entropy-based uncertainty calculation on its generated images with Deep Convolutional Generative Adversarial Networks (DCGAN). DCGAN is an image generative model that learns an image manifold of a given dataset and enables continuous change of an image. Our proposed image editing system combines DCGAN with a natural language interface that accepts image editing requests in natural language. Although such a system is helpful for human users, it often faces uncertain requests to generate acceptable images. A promising approach to solve this problem is introducing a dialogue process that shows multiple candidates and confirms the user's intention. However, confirming every editing request creates redundant dialogues. To achieve more efficient dialogues, we propose an entropy-based dialogue strategy that decides when the system should confirm, and enables effective image editing through a dialogue that reduces redundant confirmations. We conducted image editing dialogue experiments using an avatar face illustration dataset for editing by natural language requests. Through quantitative and qualitative analysis, our results show that our entropy-based confirmation strategy achieved an effective dialogue by generating images desired by users.

  PublisherOA PDFDOI

• Using Reinforcement Learning to Manage Communications Between Humans and Artificial Agents in an Evacuation Scenario.

  The Florida AI Research Society · 2020-01-01

  article
  Publisher

• Human swarm interaction using plays, audibles, and a virtual spokesperson

  2020-04-22 · 3 citations

  article

  This study explores two hypotheses about human-agent teaming: 1. Real-time coordination among a large set of autonomous robots can be achieved using predefined "plays" which define how to execute a task, and "audibles" which modify the play on the fly. 2. A spokesperson agent can serve as a representative for a group of robots, relaying information between the robots and human teammates. These hypotheses are tested in a simulated game environment: a human participant leads a search-and-rescue operation to evacuate a town threatened by an approaching wildfire, with the object of saving as many lives as possible. The participant communicates verbally with a virtual agent controlling a team of ten aerial robots and one ground vehicle, while observing a live map display with real-time location of the fire and identified survivors. Since full automation is not currently possible, two human controllers control the agent's speech and actions, and input parameters to the robots, which then operate autonomously until the parameters are changed. Designated plays include monitoring the spread of fire, searching for survivors, broadcasting warnings, guiding residents to safety, and sending the rescue vehicle. A successful evacuation of all the residents requires personal intervention in some cases (e.g., stubborn residents) while delegating other responsibilities to the spokesperson agent and robots, all in a rapidly changing scene. The study records the participants' verbal and nonverbal behavior in order to identify strategies people use when communicating with robotic swarms, and to collect data for eventual automation.

  PublisherDOI

• Predicting Ratings of Real Dialogue Participants from Artificial Data and Ratings of Human Dialogue Observers

  Language Resources and Evaluation · 2020 · 1 citations

  1st authorCorresponding
  • Computer Science
  • Artificial Intelligence
  • Computer Science
  Publisher

• Evaluation of Off-the-shelf Speech Recognizers Across Diverse Dialogue Domains

  Language Resources and Evaluation · 2020 · 10 citations

  1st authorCorresponding
  • Computer Science
  • Computer Science
  • Natural Language Processing
  Publisher

Recent grants

• EAGER: Reinforcement Learning of Multi-Party Negotiation Dialogue Policies

  NSF · $160k · 2014–2017

• RI: Small: Reinforcement Learning for Realistic Statistical Spoken Dialogue Systems - Beyond Slot-Filling Applications

  NSF · $450k · 2011–2015

Frequent coauthors

• David Traum

  88 shared

• Ron Artstein

  32 shared

• David DeVault

  22 shared

• Giota Stratou

  Keysight Technologies (United States)

  20 shared

• Fabrizio Morbini

  20 shared

• Louis‐Philippe Morency

  19 shared

• Anton Leuski

  University of Southern California

  17 shared

• Alexandros Papangelis

  16 shared

Education

• Ph.D., Institute for Language, Cognition and Computation

  University of Edinburgh

• M.S., Natural Language Processing and Speech Research Group

  Educational Testing Service

Awards & honors

• 2018 Annual SIGdial Meeting on Discourse and Dialogue (SIGDI…
• 2016 Interservice/Industry Training, Simulation and Educatio…
• 2015 International Conference on Affective Computing and Int…
• 2015 Annual SIGdial Meeting on Discourse and Dialogue (SIGDI…
• 2015 International Conference on Interactive Digital Storyte…