Paper: Mitigating Topic Bias when Detecting Decisions in Dialogue

by sodestream Project • Thursday 29 July 2021 • Permalink

Mladen Karan, Prashant Khare, Patrick Healey, and Matthew Purver
In Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue (pp. 542-547), 2021

Abstract

This work revisits the task of detecting decision-related utterances in multi-party dialogue. We explore performance of a traditional approach and a deep learning-based approach based on transformer language models, with the latter providing modest improvements. We then analyze topic bias in the models using topic information obtained by manual annotation. Our finding is that when detecting some types of decisions in our data, models rely more on topic specific words that decisions are about rather than on words that more generally indicate decision making. We further explore this by removing topic information from the train data. We show that this resolves the bias issues to an extent and, surprisingly, sometimes even boosts performance.

 Download