Towards Fluid Human-Agent Collaboration: From Dynamic Collaboration Patterns to Models of Theory of Mind Reasoning

Florian Schröder^1,2*Fabian Heinrich^1,2Stefan Kopp^1,2

• ¹Bielefeld University, Bielefeld, Germany
• ²Center of Cognitive Interaction Technology, Bielefeld University, Bielefeld, North Rhine-Westphalia, Germany

Collaborating in real-life situations rarely follows predefined roles or plans, but is established on-the-fly and flexibly coordinated by the interacting agents. We introduce the notion of fluid collaboration (FC), marked by frequent changes of the tasks partners assume or the resources they consume in response to varying requirements or affordances of the environment, tasks, or other agents. FC thus necessitates dynamic, action-oriented Theory of Mind (ToM) reasoning to enable agents to continuously infer and adapt to others' intentions and beliefs in real-time.In this paper, we discuss how FC can be enabled in human-agent collaboration. We introduce Cooperative Cuisine (CoCu), an interactive environment inspired by the game Overcooked! that facilitates human-human and human-agent collaboration in dynamic settings. We report results of an empirical study on human-human collaboration in CoCu, showing how FC can be measured empirically and that humans naturally engage in dynamically established collaboration patterns with minimal explicit communication and relying on efficient mentalizing. We then present an approach to develop artificial agents that can effectively participate in FC. Specifically, we argue for a model of dynamic mentalizing under computational constraints and integrated with action planning. We present first steps in this direction by addressing resource-rational and action-driven ToM reasoning.