Adaptation Mechanisms in Human-Agent Interaction: Effects on User’s Impressions and Engagement

2 Adaptation is a key mechanism in human-human interaction. In our work, we aim at endowing 3 embodied conversational agents with the ability to adapt their behaviour when interacting with a 4 human interlocutor. With the goal to better understand what are the main challenges concerning 5 adaptive agents, we investigated the effects on user’s experience of three adaptation models for 6 a virtual agent. The adaptation mechanisms performed by the agent take into account user’s 7 reaction and learn how to adapt on the ﬂy during the interaction. Agent’s adaptation is realised 8 at several levels (i.e., at behavioural, conversational and signal level) and focuses on improving 9 user’s experience along different dimensions (i.e., user’s impressions and engagement). In our 10 ﬁrst two studies, we aim to learn agent’s multi-modal behaviours and conversational strategies to 11 optimise dynamically user’s engagement and impressions of the agent, by taking them as input 12 during the learning process. In our third study, our model takes as input both the user’s and the 13 agent’s past behaviour and predicts the agent’s next behaviour. Our adaptation models have been 14 evaluated through experimental studies sharing the same interacting scenario, with the agent 15 playing the role of a virtual museum guide. These studies showed an impact of the adaptation 16 mechanisms on user’s experience of the interaction and their perception of the agent. Interacting 17 with an adaptive agent vs a non-adaptive agent tended to be more positively perceived. Finally, 18 the effects of people’s a-priori about virtual agents found in our studies highlight the importance 19 to take into account user’s expectancies in human-agent interaction.


INTRODUCTION
During an interaction, we communicate through multiple behaviours.Not only speech, but also our facial expressions, gestures, gaze direction, body orientation, etc. participate to the message being communicated (Argyle, 1972).Both interactants are active participants in an interaction and adapt their behaviours to each other.This adaptation arises on several levels: we align ourselves linguistically (vocabulary, syntax, level of formality), but we also adapt our non-verbal behaviours (e.g., we respond to the smile of our interlocutor, we imitate their posture, their gestural expressiveness), our conversational strategies (e.g., to be perceived warmer or more competent), etc (Burgoon et al., 2007).This multi-level adaptation can have several functions, such as reinforcing engagement in the interaction, emphasising our relationship with others, showing empathy, managing the impressions we give to others (Lakin and Chartrand, 2003;Fischer-Lokou et al., 2011;Gueguen et al., 2009).The choice of verbal and non-verbal behaviours and their temporal realisation are markers of adaptation.
Embodied Conversational Agents, ECAs, are virtual entities with a human-like appearance that are endowed with communicative and emotional capabilities (Cassell et al., 2000).They can display a wide range of multi-modal expressions to be active participants in the interaction with their human interlocutors.
They have been deployed in various human-machine interactions where they can act as tutor (Mills et al., 2019), health support (Zhang et al., 2017;Lisetti et al., 2013;Rizzo et al., 2016), companion (Sidner et al., 2018), museum guide (Swartout et al., 2010;Kopp et al., 2005), etc.Studies reported that ECAs are able to take into account their human interlocutors and show empathy (Paiva et al., 2017), display backchannels (Bevacqua et al., 2008), build rapport (Huang et al., 2011;Zhao et al., 2016).Since its relevance in human-human interaction, adaptation could be exploited to improve natural interactions with ECAs.It seems thus important to investigate whether an agent adapting to user's behaviours could provoke similar positive outcomes in the interaction.
The majority of works in this context developed models learnt from existing databases of human-human interaction and did not consider the dynamics of adaptation mechanisms during an interaction.We are interested in developing an ECA that exploits how the interaction is currently going and is able to learn in real-time what is the best adaption mechanism for the interaction.
In this paper we report 3 studies where an ECA adapts its behaviours by taking into account user's reaction and by learning how to adapt on the fly during the interaction.
The goal of the different studies is to answer two broad research questions: "Does adapting an ECA's behaviours enhance user's experience during interaction?";"How does an ECA which adapts its behaviour in real-time influence the user's perception of the agent?"User's experience can involve many factors and can be measured by different dimensions, such as user's engagement and user's impressions about the ECA (Burgoon et al., 2007).In our 3 studies that we report in this paper, we implemented three independent models where agent's adaptation is realised at several levels and focuses on improving user's experience along different dimensions: 1. Agent's adaptation at a behavioural level: the ECA adapts its behaviours (e.g., gestures, arms rest poses, smile) in order to maximise user's impressions about agent's warmth or competence, the two fundamental dimensions of social cognition (Fiske et al., 2007).This model is described in Section 7; 2. Agent's adaptation at a conversational level: the ECA adapts its communicative strategies to elicit different levels of warmth and competence, in order to maximise user's engagement.This model is described in Section 8; 3. Agent's adaptation at a signal level: the ECA adapts its head and eyes rotation and lip corners movement in function of user's signals in order to maximise user's engagement.This model is described in Section 9.
Each adaptation mechanism has been implemented in the same architecture that allows an ECA to adapt to the non-verbal behaviours of the user during the interaction.This architecture includes a multi-modal analysis of user's behaviour using the Eyesweb platform (Camurri et al., 2004), a dialogue manager (Flipper (van Waterschoot et al., 2018)) and the ECA GRETA (Pecune et al., 2014).The architecture has been adapted to each model, and evaluated through experimental studies.The ECA played the role of a virtual

Sample et al.
Adaptation Mechanisms in Human-Agent Interaction guide at the Science Museum of Paris.The scenario used in all the evaluation studies is described in Section 6.
Even though these 3 models have been implemented in the same architecture and tested on the same scenario, they have not been developed in order to do comparative studies.The main goal of this paper is to frame them in the same theoretical framework (see Section 2) and have insights about each of these different adaptation mechanisms, to better understand what are the main challenges concerning these models and to suggest further improvements for an adaptation system working on multiple levels.This paper is organised as follows: in Section 2 we review the main theories about adaptation which our work relies on, in particular Burgoon and colleagues' work; in Section 3 we present an overview of existing models that focus on adapting the ECA's behaviour according to user's behaviour; in Section 4 we specify the dimensions we focused on in our adaptation models; in Section 5 we present the general architecture we conceived to endow our ECA with the capability of adapting its behaviour to user's reactions in real-time; in Section 6 we describe the scenario we conceived to test the different adaptation models; in Sections 7, 8 and 9 we report the implementation and evaluation of each of the three models.More details about them can be found in our previous papers (Biancardi et al., 2019b,a;Dermouche and Pelachaud, 2019).We finally discuss the results of our work and possible improvements in Sections 10 and 11 respectively.

BACKGROUND
Adaptation is an essential feature of interpersonal relationship (Cappella, 1991).During an effective communication, people adapt their interaction patterns to one another (e.g., dancers synchronise their movements, people adapt their conversational style in a conversation).These patterns contribute to define and maintain our interpersonal relationships, by facilitating smooth communication, fostering attraction, reinforcing identification with an in-group, increasing rapport between communicators (Giles et al., 1991;Bernieri et al., 1988;Chartrand and Bargh, 1999;Gallois et al., 2005).
There exist several adaptation patterns, differing according to their behaviour type (e.g., the modality, the similarity to the other interlocutor's behaviour, etc), their level of consciousness, whether they are well decoded by the other interlocutor, and according to their effect on the interaction (Toma, 2014).
Cappella and colleagues (Cappella, 1981) consider an additional characteristic, that is, adaptation can be asymmetrical (unilateral), when only one partner adapts to the other, or symmetrical (mutual), like in the case of interaction synchrony.
In line with these criteria, in some examples of adaptation people's behaviours become more similar to one another.This type of adaptation is often unconscious and reflects reciprocity, or convergence.
According to Gouldner (Gouldner, 1960), reciprocity is motivated by the need to maintain harmonious and stable relations.It is contingent (i.e., one person's behaviours are dependent upon the other's) and transactional (i.e., it is part of an exchange process between two people).
In other cases, adaptation can include complementarity, or divergence: this occurs when the behaviour of one person differs from but complements that of the other person.
Several theories focus on one or more specific characteristics of adaptation and highlight different factors that drive people's behaviours.They can be divided into 4 main classes, according to the perspective they follow to explain adaptation.
The first class of theories includes biologically based models (e.g., (Condon and Ogston, 1971), (Bernieri et al., 1988)).These theories state that individuals exhibit similar patterns to one another.These adaptation patterns have an innate basis, as they are related to satisfaction of basic needs like bonding, safety, social organisation.Their innate bases make them universal and involuntary but they can be influenced as well by environmental and social factors.
Following a different perspective, arousal-based and affect-based models (e.g., (Argyle and Dean, 1965), (Altman et al., 1981), (Cappella and Greene, 1982)) support the role of internal emotional and arousal states as driving factors of people's behaviours.These states determine approaching or avoiding behaviours.This group of theories explains the balance between compensation and reciprocity.
Social-norm models (e.g., (Gouldner, 1960), (Dindia, 1988)) do not consider the role of physiological or psychological factors but argue for the importance of social phenomena as guiding forces.These social phenomena are for example the in-group or out-group status of the interactants, their motivation to identify with one another, their level of affiliation or social distance.
The last class of theories includes communication-and cognitive-based models (e.g., (Andersen, 1985), (Hale and Burgoon, 1984)), which focus on the communicative purposes of the interactants and on the meaning the behavioural patterns convey.While adaption happens mainly unconsciously, it may happen that the process of interpersonal adaptation may be strategic and conscious (Gallois et al., 2005;Giles et al., 1991).
The majority of these theories have been studied by Burgoon and colleagues (Burgoon et al., 2007).In particular, they examined fifteen previous models and considered the most important conclusions from the previous empirical research.From this analysis they came out with a broader theory, the Interaction Adaptation Theory (IAT).This theory states that we alter our behaviour in response to the behaviour of another person in conversations (Infante et al., 2010).IAT takes into account the complexities of interpersonal interactions by considering people's needs, expectations, desires and goals as precursors of their degree and form of adaptation.IAT is a communication theory made of multiple theories, and which focuses on sender's and receiver's process and patterns.For example, when IP and A almost match, IAT predicts behavioural patterns such as reciprocity and convergence.When A is more negatively valenced than IP, the model predicts compensation and avoiding behaviours.
In the work presented in this paper we rely on Burgoon's IAT theory.Indeed, our adapting ECA has an Interactional Position (IP), resulting from its Desires (D) and Expectations (E).In particular, the agent's Desire D is to maximise user's experience, and its Expectations E are about user's reactions to its behaviours.
In our different models of adaptation mechanisms, agent's Desire D refers either to give the best impression to the user or to maximise user's engagement (see Section 4).Consequently, the Expectations E refer to user's reaction reflecting their impressions or engagement level in response to the agent's behaviour.The behaviour that will be performed by the ECA depends on the relation between THE agent's IP and THE user's reaction (Actual behaviour A).
In addition, we explore different ways the ECA can adapt to user's reactions.On one hand, we focus on theories that consider adaptive behaviours more broadly than a mere matching, that is, adaptation as responding in appropriate ways to a partner.The ECA will choose its behaviours according to the effect they have on user's experience (see Section 7).In Study 2 (see Section 8), our adaptive agent follows the same perspective but by adapting its communicative strategies.On the other hand, we try to simulate a more unconscious and automatic process working at a motoric level: the agent adapts at a signal level (see Study 3, Section 9).

STATE OF THE ART
In this section, we present an overview of existing models that focused on adapting ECAs' behaviour according to user's behaviour in order to enhance the interaction and user's experience along different dimensions such as engagement, rapport, interest, liking etc.These existing models predicted and generated different forms of adaptation, such as backchannels, mimicry, voice adaptation, and were applied on virtual agents or robots.
Several works were interested in understanding the impact of adaptation on user's engagement and rapport building.Some of them did so through the production of backchannels.Huang et al. (2010) developed an ECA able to produce backchannels to reinforce the building of rapport with its human interlocutor.The authors used Conditional Random Fields (CRF) (Lafferty et al., 2001) to automatically learn when listeners produce visual backchannels.The prediction was based on three features: prosody (e.g., pause, pitch), lexical (spoken words) and gaze.Using this model, the ECA was perceived more natural; it also created more rapport with its interlocutor during the interaction.Schröder et al. (2015) developed a sensitive artificial listener able to produce backchannels.They developed a model that predicted when an ECA should display a backchannel and with which intention.The backchannel could either be a smile, nod, and vocalisation or an imitation of human's smile and head movement.Participants who interacted with an ECA displaying backchannels were more engaged compared to when no backchannels were shown.
Other works focused on modelling ECAs able to mimic their interlocutors' behaviours.Bailenson and Yee (2005) studied the social influence of mimicry during human-agent interaction (they referred at this as the chameleon effect).The ECA mimicked the user's head movements with a delay up to 4 seconds.
An ECA showing mimicry was perceived as more persuasive and more positive than an ECA showing no mimicry at all.Raffard et al. (2018) also studied the influence of ECAs mimicking their interlocutors' head and body posture with some delay (below 4 seconds).Participants with schizophrenia and healthy participants interacted with an ECA that mimicked them or not.Both groups showed higher behaviour synchronisation and reported an increase of rapport in the mimicry condition.Another study involving mimicry was proposed in (Verberne et al., 2013) in order to evaluate if an ECA mimicking user's head movements would be liked and trusted more than a non-mimicking one.This research question was investigated by running two experiments in which participants played a game involving drivers handling over the car control to the ECA.While results differed depending on the game, the authors found that liking and trust were higher for a mimicking ECA than for a non-mimicking one.
Reinforcement learning methods for optimising agent's behaviours according to user's preference have been used in different works.For example, Liu et al. (2008) endowed a robot with the capacity to detect in real time the affective states (liking, anxiety and engagement) of children with autism spectrum disorder and to adapt its behaviour to children's preferences of activities.The detection of children's affective states Frontiers was done by exploiting their physiological signals.A large database of physiological signals was explored to find their interrelation with the affective states of the children.Then, an SVM-based recogniser was trained to match children's affective state to a set of physiological features.Finally, the robot learned the activities that children preferred to do at a moment based on the predicted liking level of the children using QV-learning (Wiering, 2005).The proposed model led to an increase in the reported liking level of the children towards the robot.Ritschel et al. (2017) studied the influence of agent's personality on user's engagement.They proposed a reinforcement learning model based on social signals for adapting the personality of a social robot to user's engagement level.User's engagement was estimated from their multi-modal social signals such as gaze direction and posture.The robot adapted its linguistic style by generating utterances with different degrees of extroversion using a Natural Language Generation approach.
The robot that adapted its personality through its linguistic style increased user's engagement but the degree of user's preference toward the robot depended on the on-going task.Later on the authors applied similar approach to build a robot that adapts to the sense of humour of its human interlocutor (Weber et al., 2018).
Several works have been conducted in the domain of education where an agent, being physical as a robot or virtual as an ECA, adapted to the learner's behaviour.These works reported that adaptation is generally linked with an increase of the learner's engagement and performance.For example, Gordon et al. (2016) developed a robot acting as a tutor for children learning a second language.To favour learning, the robot adapted its behaviours to optimise the level of the children's engagement which was computed from their facial expressions.A reinforcement learning algorithm was applied to compute the robot's verbal and non-verbal behaviour.Children showed higher engagement and learned more second-language words with the robot that adapted its behaviours to children's facial expression, compared to the non-adaptive robot.Woolf et al. (2009) manually designed rules to adapt the facial expressions of a virtual tutor according to the student's affective state (e.g., frustrated, bored or confused).For example, if the student was delighted, respectively sad, the tutor might look pleased, respectively sad.Results showed that when the virtual tutor adapted its facial expressions in response to the student's ones, the latter maintained higher levels of interest and reduced boredom when interacting with the tutor.
Other works looked at adapting the activities undertaken by an agent during an interaction to enhance knowledge acquisition and reinforce engagement.In (Ahmad et al., 2017), a robot playing games with children was able to perform three different types of adaptations, game-based, emotion-based, and memorybased, that relied respectively on: i) the game state, ii) emotion detection from child's facial expressions, and iii) face recognition mechanisms and remembering child's performance.In the first category of adaptation, a decision making mechanism was used to generate a supporting verbal and non-verbal behaviour.For example, if the child performed well, the robot said "Wow, you are playing extra-ordinary" and showed positive gestures such as thumbs up.The emotion-based adaptation mapped the child's emotions to a set of supportive dialogues.For example, when detecting the emotion of joy the robot said: "You are looking happy, I think you are enjoying the game".For memory adaptation the robot adapted its behaviour after recognising the child and retrieving the child's game history such as their game performance and results.Results highlighted that emotion-based adaptation resulted in the highest level of social engagement compared to memory-based adaptation.Game adaptation did not result in maintaining long-term social engagement.Coninx et al. (2016) proposed an adaptive robot able to change activities during an interaction with children suffering of diabetes.The aim of the robot was to reinforce children's knowledge for managing their disease and well-being.Three activities were designed to approach the diabetes-learning problem from different perspectives.Depending on the children's motivation the robot switched between the three proposed activities.Adapting activities in the course of the interaction led to a high level of children's engagement toward the robot.Moreover, this approach seemed promising for setting up long-term child-robot relationship.
In a task-oriented interaction, Hemminahaus and Kopp (2017) presented a model to adapt the social behaviour of an assistive robot.The robot could predict when and how to guide the attention of the user depending on the interaction contexts.The authors developed a model that mapped interactional functions such as motivating the user, guiding them, onto low-level behaviours executable by the robot.The high-level functions were selected based on the interaction context as well as the attentive and emotional states of the user.Reinforcement learning was used to predict the mapping of these functions onto lower-level behaviours.The model was evaluated in a scenario in which a robot assisted the user in solving a memory game by guiding their attention to the target objects.Results showed that users were able to solve the game faster with the adaptive robot.
Other works focused on voice-adaptation during social interaction.Voice-adaptation is based on acousticprosodic entrainment that occurs when two interactants adapt their manner of speaking, such as their speaking rate, tone, or pitch, to each other.Levitan (2013) found that voice-adaptation improved spoken dialogue systems performance and user's satisfaction.Lubold et al. (2016) studied the effect of voiceadaptation on social variables such as rapport and social presence.They found that social presence was significantly higher with a social voice-adaptive speech interface than with purely social dialogue.
In most of previous works, the adaptation mechanisms that have been implemented measured their influence on user's engagement through questionnaires.They did not include them as a factor of the adaptation mechanisms.In our first two studies reported in this paper, we aimed to learn agent's multimodal behaviours and conversational strategies to optimise dynamically user's engagement and their impressions of the ECA, by taking them as input during the learning process.
Moreover, in most existing works the agent's predicted behaviour depended exclusively on the user's behaviour and ignored the interaction loop between the ECA and the user.In our third study, we took into account this interaction loop, i.e., our model takes as input both, the user's and the agent's past behaviour and predicts the agent's next behaviour.Another novelty presented in our work is to include the agent's communicative intentions along with its adaptive behaviours.

DIMENSIONS OF STUDY
In our studies we focused on adaptation in human-agent interaction, by using user's reactions as the input for agent's adaptation.In particular, we took into account two main dimensions, that are user's impressions of the ECA and user's engagement during the interaction.
These two dimensions play an important role during human-agent interactions, as they influence the acceptability of the ECA by the user and the willingness of interacting again with it (Bergmann et al., 2012;Bickmore et al., 2013;Cafaro et al., 2016).In order to engage the user, it is important that the ECA displays appropriate socio-emotional behaviours (Pelachaud, 2009).In our case, we were interested in whether and how the ECA could affect user's engagement by managing the impressions it gave to them.In particular, we considered user's impressions of the two main dimensions of social cognition, i.e., warmth and competence (Fiske et al., 2007).Warmth includes traits like friendliness, trustworthiness, sociability, while competence includes traits like intelligence, agency and efficacy.In human-human interaction, several studies showed the role of non-verbal behaviours in conveying different impressions of warmth and competence.In particular, communicative gestures, arms rest poses and smiling behaviour have been found to be associated with different degrees of warmth and/or competence (Duchenne, 1990;Cuddy et al., 2008; Frontiers  Maricchiolo et al., 2009;Biancardi et al., 2017a).In the context of human-agent interaction, we can control and adapt the non-verbal behaviours of the ECA during the flow of the interaction.Following Burgoon's IAT theoretical model, our adapting ECA has thus the Desire D to maintain user's engagement (or impressions) during the interaction.Since the ECA aims to be perceived as a social entity by its human interlocutor, the agent's Expectancy E is that adaptation can enhance the interaction experience.In our work we are interested in whether adapting at a behavioural or conversational level (i.e., the agent's warmth and competence impressions) and/or at low-level (i.e., the agent's head and eyes rotation and lip corners movement) could affect user's engagement.Even though the impact of agent's adaptation on user's engagement has already been the object of much research (see Section 3), here we use user's engagement as a real-time variable given as input for the agent's adaptation.

ARCHITECTURE
In this Section we present the architecture we conceived to endow the ECA with the capability of adapting its behaviour to user's reactions in real-time.The architecture consists of several modules (see Figure 1).One module extracts information about user's behaviours using a Kinect and a microphone.This information is interpreted in terms of speech (what the user has uttered) and user's state (e.g., their engagement in the interaction).This interpreted information is sent to a dialog manager that computes the communicative intentions of the ECA, that is, what it should say and how.Finally, the animation of the ECA is computed on the fly and played in real-time.The agent's adaptation mechanisms are also taken into account when computing its verbal and non-verbal behaviours.The architecture is general enough to allow for customisation of its different modules according to the different adaptation mechanisms and goals of the agent.
In more details, the 4 main parts of the architecture are: 1. User's Analysis.The EyesWeb platform (Camurri et al., 2004) allows the extraction in real-time of: (1) user's non-verbal signals (e.g., head and trunk rotation) starting from the Kinect depth camera skeleton data; (2) user's facial muscular activity (Action Units AUs (Ekman et al., 2002)), by running the OpenFace framework (Baltrušaitis et al., 2016) This is a provisional file, not the final typeset article These low-level signals are processed by EyesWeb and other external tools, such as machine learning pre-trained models (Dermouche and Pelachaud, 2019;Wang et al., 2019) to extract high-level features about the user, such as their level of engagement.
2. Dialog Model.In this module the dialog manager Flipper (van Waterschoot et al., 2018) selects the dialog act that the agent will perform, as well as the communicative intention of the agent (i.e., how to perform that dialog act).
3. Agent's Behaviour.Agent's behaviour generation is performed by GRETA, a software platform supporting the creation of socio-emotional embodied conversational agents (Pecune et al., 2014).The Agent's Behaviour module is made of two main modules: the Behaviour Planner receives the communicative intentions of the ECA from the Dialog Model module as input and instantiates them into multi-modal behaviours; the Behaviour Realiser transforms the multi-modal behaviours into facial and body animations to be displayed on a graphics screen.
4. Adaptation Mechanism.Since the ECA can adapt its behaviours at different levels, the Adaptation Mechanism module is implemented in different parts of the architecture, according to the type of adaptation the ECA performs.That is, the adaptation can affect the communicative intentions of the ECA or it can occur during the behaviour realisation at the animation level.In the first two models presented in this paper, the Adaptation Mechanism module is connected to the Dialog Model module, while for the third model it is connected to the Agent's Behaviour module.

SCENARIO
Each type of adaptation has been investigated by running human-agent interaction experiments at the Science Museum of Paris.In the scenario conceived for these experiments, the ECA, called Alice, played the role of a virtual guide of the museum.
The experiment room included a questionnaires space, including a desk with a laptop and a chair; an interaction space, with a big TV screen displaying the ECA, a Kinect 2 placed on the top of the TV screen and a black tent behind the chair where the participant sat; a control space, separated from the rest of the room by 2 screens, including a desk with the computer running the system architecture.The interaction space is shown in Figure 2.
The experiments were completed in three phases: 1. Before the interaction began, the participant sat at the questionnaires space, read and signed the consent form, and filled out a first questionnaire (NARS, see below).Then they moved to the interaction space, where the experimenter gave the last instructions [5 min]; 2. During the interaction phase, the participant stayed right in front of the TV screen, between it and the black tent.They wore a headset and was free to interact with the ECA as they wanted.During this phase, the experimenter stayed in the control space, behind the screens [3 min]; 3.After the interaction, the participant came back to the questionnaires space and filled out the last questionnaires about their perception of the ECA and of the interaction.After that, the experimenter proceeded with the debriefing [5 min].
Before the interaction with the ECA, we asked participants to fill out a questionnaire about their a-priori about virtual characters (NARS): an adapted version of NARS scale from Nomura et al. (2006) was used.
Items of the questionnaire included for example how much participants would feel relaxed talking with a virtual agent, or how much they would like the idea that virtual agents made judgements.
Figure 2. The interaction space in the experiment room.The participants were sitting in front of the TV screen displaying the ECA.On the left, 2 screens separated the interaction space from the control space.
The interaction with the ECA lasted about 3 minutes.It included 26 steps.A step included one dialog act played by the ECA and participant's potential reaction/answer.The dialog scenario was built so the ECA drove the discussion.The virtual guide provided information on an exhibit that was currently happening in the museum.It also asked some questions about participants' preferences.Purposely, we limited the possibility for participants to take the lead of the conversation as we wanted to avoid any error due to automatic speech understanding.More details about the dialog model can be found in (Biancardi et al., 2019a).

STUDY 1: ADAPTATION OF AGENT'S BEHAVIOURS
At this step we aim to investigate adaptation at a high level, meant as convergence of the agent's behaviours according to user's impressions of the ECA.
The goal of this first model is to make the ECA learn the verbal and non-verbal behaviours to be perceived as warm or competent by measuring and using user's impressions as reward.

Architecture
The general architecture described in Section 5 has been modified in order to contain a module for the detection of user's impressions, and a specific set of verbal and non-verbal behaviours from which the ECA could choose.
The modified architecture of the system is depicted in Figure 3.In the following Section we give more details about the modified modules.

User's Analysis: User's Impressions Detection
User's impressions can be detected from their non-verbal behaviours, in particular their facial expressions.The User's Analysis module is integrated with a User's Impressions Detection module that takes as input a stream of user's facial Action Units (AUs) (Ekman et al., 2002) and outputs the potential user's impressions about the level of warmth (or competence) of the ECA.
A trained Multilayer Perceptron Regression (MLP) model is implemented in this module to detect the impressions formed by users' about the ECA.The MLP model was previously trained with a corpus including face video recordings and continuous self-report annotations of warmth and competence given by participants watching the videos of the NoXi database (Cafaro et al., 2017).The self-report annotations being considered separately, the MLP model was trained twice, one for warmth and one for competence.
More details about this model can be found in (Wang et al., 2019).In this model the adaptation of the ECA concerns the impressions of warmth and competence given to the user.The inputs of the Adaptation Mechanism module are the dialog act to be realised (coming from the Dialog Model module) and the user's impression of agent's warmth or competence (coming from the User's Analysis module).The output is a combination of behaviours to realise the dialog act, chosen from a set of possible verbal and non-verbal behaviours to perform.
To be able to change the agent's behaviour according to detected participant's impressions, a machine learning algorithm is applied.We follow a reinforcement learning approach to learn which actions the ECA should take (here verbal and non-verbal behaviours) in response to some events (here user's detected impressions).We rely on a Q-learning algorithm for this step.More details about it can be found in (Biancardi et al., 2019b).
The set of verbal and non-verbal behaviours, from which the Q-learning algorithm selects a combination to send to the Behaviour Planner of the Agent's Behaviour module, includes: • Type of gestures.The ECA could perform ideational (i.e., related to the content of the speech) or beat (i.e., marking speech rhythm, not related to the content of the speech) gestures or no gestures.
• Arms rest poses: in the absence of any kind of gesture, these rest poses could be performed by the ECA: akimbo (i.e., hands on the hips), arms crossed on the chest, arms along its body, or hands crossed on the table.
• Smiling.During the animation, the ECA could decide whether or not to perform smiling behaviour, characterised by the activation of AU6 (cheek raiser) and AU12 (lip puller up).
• Verbal behaviour.The ECA could modify the use of youand we-words, the level of formality of the language, the length of the sentences.These features have been found to be related to different impressions of warmth and competence (Pennebaker, 2011;Callejas et al., 2014).

Experimental Design
The adaptation model described in the previous Subsection 7.1.2has been evaluated by using the scenario described in Section 6.Here we describe the experimental variables manipulated and measured during the experiment.

Independent Variable.
The independent variable manipulated in this experiment, called Model, concerns the use of the adaption model and includes 3 conditions: • Warmth, when the ECA adapts its behaviours according to user's impressions of agent's warmth, with the goal to maximise these impressions; • Competence, when the ECA adapts its behaviours according to user's impressions of the agent's competence, with the goal to maximise these impressions; • Random, when the adaptation model is not exploited and the ECA randomly chooses its behaviour, without considering user's reactions.

Measures.
The dependent variables measured after the interaction with the ECA are: • User's perception of agent's warmth (w) and competence (c): participants were asked to rate their level of agreement about how well each adjective described the ECA (4 adjectives concerning warmth, 4 concerning competence, according to Aragonés et al. (2015)).Even though only one dimension was manipulated at a time, we measured user's impressions about both of them in order to check whether the manipulation of one dimension can affect the impressions about the other (as already found in literature (Rosenberg et al., 1968;Yzerbyt et al., 2005;Judd et al., 2005)).
• User's experience of the interaction (exp): participants were asked to rate their level of agreement about a list of items adapted from (Bickmore et al., 2011).

Hypotheses.
We hypothesised that: H1: When the ECA is in the Warmth condition, that is, when it adapts its behaviours according to user's impressions of agent's warmth, it will be perceived as warmer compared to the Random condition; H2: When the ECA is in the Competence condition, that is, when it adapts its behaviours according to user's impressions of agent's competence, it will be perceived as more competent compared to the Random condition; H3: When the agent ECA its behaviours, that is, in either Warmth or Competence conditions, this will improve user's experience of the interaction, compared to the Random condition.

Analysis and Results
The visitors (24 women and 47 men) of the Carrefour Numérique of the Cité des sciences et de l'industrie of Paris were invited to take part in our experiment.28% of them were in the range 18-25 years old, 18% were in the range 25-36, 28% in the range 36-45, 15% in the range of 46-55 and 11% were over 55 years old.Participants were randomly assigned to each condition with 25 participants assigned to the Warmth condition, 27 to the Competence condition and 19 to the Random one.
We computed Cronbach's alphas on the scores of the 4 items about w and the 4 about c: good reliability was found for both (α = 0.85 and α = 0.81 respectively).Then, we computed the mean of these items in order to have one w score and one c score for each participant and we used them for our analyses.
Since NARS scores got an acceptable degree of reliability (α = 0.69), we computed the overall mean of these items for each participant and divided them into 2 groups, "high" and "low", according to whether they obtained a score higher than the overall mean or not, respectively.Participants were almost equally distributed into the two groups (35 in the "high" group, 36 in the "low" group).Chi-square tests for Model, age and sex were run to verify that participants were equally distributed across these variables, too (all p > 0.5).

Warmth Scores.
The w means were normally distributed (Shapiro test's p = 0.07) and their variances homogeneous (Bartlett tests' ps for each variable were > 0.44).We run a 3x5x2x2 between-subjects ANOVA, with Model, age, sex and NARS as factors.
No effects of age or sex were found.A main effect of NARS was found (F (1, 32) = 4.23, p < 0.05).A post-hoc test specified that the group who got high scores in NARS gave higher ratings about the agent's w (M = 3.65, SD = 0.84) than the group who got low scores in NARS (M = 3.24, SD = 0.96).
Although we did not find any significant effect, w scores were on average higher in the Warmth and Competence conditions than in the Random condition.Mean and standard error of w scores are shown in The c means were normally distributed (Shapiro test's p = 0.22) and their variances homogeneous (Bartlett tests' ps for each variable were > 0.25).We run a 3x5x2x2 between-subjects ANOVA, with Model, age, sex and NARS scores as factors.
We did not find any effect of age, sex or NARS.A significant main effect of Model was found (F (2, 32) = 3.22, p = 0.047, η 2 = 0.085).In particular, post-hoc tests revealed that participants in the Competence condition gave higher scores about the agent's c than participants in the Random condition (M C = 3.3, M R = 2.76, p-adj = 0.05).

User's experience Scores.
The exp items' means were not normally distributed but their variances were homogeneous (Bartlett tests' ps for each variable were > 0.17).We run non-parametric tests for each item and each variable.
Even if we did not find any statistically significant effect, on average items' scores tended to be higher in Warmth and Competence conditions than in Random condition.The Q-learning algorithm ended up selecting (for each participant) one specific combination of verbal and non-verbal behaviours from the 84% ± 7 and 82% ± 7 of the interaction, for Warmth and Competence conditions respectively.In the Warmth condition the rest pose Akimbo was the most selected one (χ 2 = 8.05, p < 0.01), and we found a tendency to use Ideational gestures (p > 0.05).In the Competence condition the Verbal Behaviour aiming at eliciting low warmth and high competence (formal language, long sentences, use of you-words) was the most selected one (χ 2 = 3.86, p < 0.01).

Discussion
The results show that participants' ratings tended to be higher in the conditions in which the ECA used the adaptation model, compared to when it selected its behaviour randomly.In particular, the results indicate that we successfully manipulated the impression of competence when using our adaptive ECA.Indeed, higher competence was reported in the Competence condition compared to the Random one.No a-priori effect was found.
On the other hand, we found an a-priori effect on warmth but no significant effect of our conditions (just a positive trend for both Competence and Warmth conditions).People with high a-priori about virtual agents gave higher ratings about the agent's warmth than people with low a-priori.
We could hypothesise some explanations for these results.First, we did not get effects of our experimental conditions on warmth ratings since people were more anchored into their a-priori and it was hard to change them.Indeed, people's expectancies have already been found to have an effect on user's judgements about ECAs (Burgoon et al., 2016;Biancardi et al., 2017b;Weber et al., 2018).The fact that we found this effect only for warmth judgements could be related to the primacy of warmth judgements over competence (Wojciszke and Abele, 2008).Then, it could have been easier to elicit impressions of competence since we found no a-priori effect on competence.This could be explained as people might expect that it is easier to implement knowledge in an ECA rather than social behaviours.
User's experience of the interaction was not affected by agent's adaption.During the debriefing many participants expressed their disappointment about the agent's appearance, the quality of the voice synthesizer and the animation, described as "disturbing", "creepy", as well as the limitations of the conversation (participants could only answer to ECA's questions).These factors could have reduced any other effect of the independent variables.Indeed, the agent's appearance and the structure of the dialog were the same across conditions.If participants mainly focused on these elements, they could have paid less attention to ECA's verbal and non-verbal behaviour (the variables that were manipulated and we were interested in), which thus did not manage to affect their overall experience of the interaction.

STUDY 2: ADAPTATION OF COMMUNICATIVE STRATEGIES
At this step we investigate adaptation at a higher level than the previous one, namely the communicative strategies of the ECA.In particular, we focus on the agent's self-presentational strategies, that is, different techniques to convey different levels of warmth and competence towards the user (Jones and Pittman, 1982).Each strategy is realised in terms of the verbal and non-verbal behaviour of the ECA, according to (Biancardi et al., 2017a;Callejas et al., 2014;Pennebaker, 2011).
While in the previous study we investigated whether and how adaptation could affect user's impressions of the agent, we here focus on whether and how adaptation can affect user's engagement during the interaction.The goal of this second model is thus to make the ECA learn the communicative strategies that improve user's engagement, by measuring and using user's engagement as reward.

Architecture
The general architecture described in Section 5 has been modified in order to contain a module for the detection of user's engagement, and a communicative intention planner for the choice of the agent's self-presentational strategy.
The modified architecture of the system is depicted in Figure 5.In the following subsection we give more details about the modified modules.that the user is enjoying the interaction (Castellano et al., 2009).Eyebrows are equally important: for example, Corrigan et al. (2016) claimed that "frowning may indicate effortful processing suggesting high levels of cognitive engagement".Head/trunk signals are detected in order to measure user's attention level.
According to Corrigan et al. (2016), attention is a key aspect of engagement: an engaged user continuously gazes at relevant objects/persons during the interaction.We approximate user's gaze with the user's head and trunk orientation.

Adaptation Mechanism: Communicative Intention Management.
During its interaction with the user, the agent has the goal of selecting its self-presentational strategy (e.g., to communicate verbally and non-verbally a given dialog act with high warmth and low competence).
The agent can choose its strategy among a given set of 4 strategies inspired from Jones and Pittman's taxonomy (Jones and Pittman, 1982): • Ingratiation: the ECA has the goal to convey positive interpersonal qualities and elicit impressions of high warmth towards the user, without considering its level of competence; • Supplication: the ECA has the goal to present its weaknesses and elicit impressions of high warmth and low competence;

Frontiers
• Self-promotion: the ECA has the goal to focus on its capabilities and elicit impressions of high competence, without considering its level of warmth; • Intimidation: the ECA has the goal to elicit impressions of high competence by decreasing its level of warmth.
The verbal behaviour characterising the different strategies is inspired from the works of Pennebaker (2011) and Callejas et al. (2014).In particular, we took into account the use of youand we-words, the level of formality of the language, the length of the sentences.
The choice of agent's non-verbal behaviour is based on our previous studies described in (Biancardi et al., 2017a,b).So, for example, if the current agent's self-presentational strategy is Supplication and the next dialog act to be spoken is introducing a topic, then the agent would say "I think that while you play there are captors that measure tons of stuffs!" accompanied by smiling and beat gestures.Conversely, if the current agent's self-presentational strategy is Intimidation and the next dialog act to be spoken is the same, then the agent would say "While you play at video games, several captors measure your physiological signals."accompanied by ideational gestures without smiling.
To be able to change the agent's communicative strategy according to the detected participant's engagement, we applied a reinforcement learning algorithm to make the ECA learn what strategy to use.Specifically, a multi-armed bandit algorithm (Katehakis and Veinott Jr, 1987) was applied.This algorithm is a simplified setting of reinforcement learning which models agents evolving in an environment where they can perform several actions, each action being more or less rewarding for them.The choice of the action does not affect the state (i.e., what happens in the environment).In our case, the actions that the ECA could perform are the verbal and non-verbal behaviours corresponding to the self-presentational strategy the ECA aims to communicate.The environment is the interaction with the user, while the state space is the set of dialog acts used at each speaking turn.The choice of the action does not change the state (i.e., the dialog act used during the actual speaking turn), but rather it acts on how this dialog act is realised by verbal and non-verbal behaviour.More details about the multi-armed bandit function used in our model can be found in (Biancardi et al., 2019a).

Experimental Design
The adaptation model described in the previous Section 8.1.2was evaluated by using the scenario described in Section 6.Here we describe the experimental variables manipulated and measured during the experiment.

Independent Variable.
The design includes one independent variable, called Communicative Strategy, with 6 levels determining the way the ECA chooses the strategy to use: 1. Adaptation: the ECA uses the adaptation model and thus selects one self-presentational strategy at each speaking turn, by using user's engagement as reward; 2. Random: the ECA chooses a random behaviour at each speaking turn; 3. Ingr static: the ECA always adopts the Ingratiation strategy during the whole interaction; 4. Suppl static: the ECA always adopts the Supplication strategy during the whole interaction; 5. Self static: the ECA always adopts the Self-promotion strategy during the whole interaction; 6. Intim static: the ECA always adopts the Intimidation strategy during the whole interaction.condition also liked the ECA more than participants in the Intim static condition (z = 2.87, p-adj < 0.05).
No differences were found between the scores of the participants in the Adaptation condition and those of the other participants, for any of the items measuring exp.
The exp scores are also affected by participants' a-priori about virtual agents (measured through the NARS questionnaire).In particular, participants who got high scores in the NARS questionnaire were more satisfied by the interaction (U = 910.5,p < 0.05), were more motivated to continue the interaction (U = 998, p = 0.001) and perceived the agent as less closed to a computer(U = 1028, p < 0.001) than people who got low scores in the NARS questionnaire.
Another interesting result concerns the effect of age on participants' satisfaction (H(4) = 15.05,p < 0.01): people in the age range 55+ were more satisfied than people of any other age range (all p-adj < 0.05).
On the whole, these results do not allow us to validate H2a, but agent's adaptation was found to have at least an effect on its level of warmth (H2b).

Verbal cues of engagement
During each speaking turn, the user was free to reply to the agent's utterances.We consider as a user's verbal feedback any type of verbal reply to the ECA, from a simple backchannel (e.g., "ok","mm") to a longer response (e.g., giving an opinion about what the ECA said).In general, participants who did not give much verbal feedback (i.e., less than 13 replies to the agent's utterances over all the speaking turns) answered positively to the ECA when it asked whether they wanted to continue to discuss with it, compared to the participants who gave more verbal feedback (OR = 4.27, p < 0.05).In addition, we found that the participants who did not give much verbal feedback liked the ECA more compared to those who talked a lot during the interaction (U = 36.5,p < 0.05).However, no differences in any of the dependent variables were found according to Communicative Strategy.

Discussion
First of all, regarding H1, the only statistically significant results concern the perception of agent's warmth.
The ECA was rated as colder when it adopted Intim static strategy, compared to the other conditions.This supports the thesis of the primacy of warmth dimension (Wojciszke and Abele, 2008) and it is in line This is a provisional file, not the final typeset article with the positive-negative asymmetry effect described by (Peeters and Czapinski, 1990), who argues that negative information has generally a higher impact in person perception than positive information.In our case, when the ECA displayed cold (i.e., low warmth) behaviours (i.e., in Intim static condition), it was judged by participants with statistically significant lower ratings of warmth.Regarding the other conditions (Ingr static, Supp static, Self static, Adaptation and Random)), they elicited warmer impressions in the user, but there was not one strategy better than the others in this regard.The fact that also the Self static elicited the same level of warmth than the others could reflect an halo effect (Rosenberg et al., 1968): the behaviours displayed to appear competent influenced its warmth perception in the same direction.
Regarding H2, the results do not validate our hypothesis H2a that the interaction would be improved when the ECA managed its impressions by adapting its strategy according to user's engagement.When analysing scores for exp items, we found that participants were more satisfied by the interaction and they liked the ECA more when the ECA wanted to be perceived as warm (i.e., in Ingr static condition), compared to when it wanted to be perceived as cold and competent (i.e., in Intim static condition).An hypothesis is that, since the ECA was perceived warmer in Ingr static condition, it could have positively influenced the ratings of the other items, like user's satisfaction.Concerning H2b about a possible effect of agent's adaptation on user's perception of its warmth and competence, it is interesting to see that when the ECA adapted its self-presentational strategy according to user's overall engagement, it was perceived as warm.This highlights a link between agent's adaptation, user's engagement and warm impression: the more the ECA adapted its behaviours, the more the user was engaged and the more she/he perceived the ECA as warm.

STUDY 3: ADAPTATION AT A SIGNAL LEVEL
At this step, we are interested in low-level adaptation at the signal level.We aim to model how the ECA can adapt its signals to user's signals.Thus, we make the ECA predict the signals to display at each time step, according to those displayed by both the ECA and the user during a given time window.For sake of simplicity, we consider a subset of signals, namely lip corners movement (AU12), gaze direction and head movement.To reach our aim, we follow a two-steps approach.At first, we need to predict which signals due to adaptation to user's behaviours should be displayed by the ECA at each time step.The prediction of signals adaptation is learned on human-human interaction.The ECA ought to communicate its intentions as well as to adapt to user's signals.Then, the second step of our approach consists in blending the predicted signals linked to the adaptation mechanism with the non-verbal behaviours corresponding to the agent's communicative intentions.We describe in further details our algorithm in subsection 9.1.2.

Architecture
The general architecture described in Section 5 has been modified in order to contain a module for predicting the next social signal to be merged with the agent's other communicative ones.The modified architecture of the system is depicted in Figure 6.In the following we explain the modified modules.More details about these modules can be found in (Dermouche and Pelachaud, 2019).Behaviour module where they are merged with the behaviours of the ECA related to its communicative intents.

Agent's Behaviour: Behaviour Realiser
We have updated the Behaviour Realiser so the ECA not only communicates its intentions but also adapts its behaviours in real time to user's behaviours.This module blends the predicted signals linked to the adaptation mechanism with the non-verbal behaviours corresponding to its communicative intentions that have been outputted using the GRETA agent platform (Pecune et al., 2014).More precisely, the dialogue module Flipper sends the set of communicative intentions to the Agent's Behaviour module.This module computes the multi-modal behaviour of the ECA and sends it to the Behaviour Realiser that computes the animation of the ECA's face and body.Then, before sending each frame to be displayed by the animation player, the animation computed from the communicative intentions is merged with the animation predicted by the Adaption Mechanism module.This operation is repeated at every frame.

Sample et al.
Adaptation Mechanisms in Human-Agent Interaction

Experimental Design
The adaptation model described in the previous Section was evaluated by using the scenario described in Section 6.Here we describe the experimental variables manipulated and measured during the experiment.9.2.1 Independent Variable.
We manipulated the type of Low-level adaptation of the ECA by considering five conditions: • Random: when the ECA did not adapt its behaviour; • Head: when the ECA adapted its head rotation according to the user's behaviour; • Lip Corners: when the ECA adapted its lip corners puller movement (AU12) according to the user's behaviour; • Eyes: when the ECA adapted its eyes rotation according to the user's behaviour; • All: when the ECA adapted its head and eyes rotation and lip corners movement, according to the user's behaviour.
We tested these five conditions with a between-subjects design.

Measures.
The dependent variables measured after the interaction with the ECA were user's engagement and the perceived friendliness of the ECA.
User's engagement was evaluated using the I-PEFiC framework (van Vugt et al., 2006) that encompasses user's engagement and satisfaction during human-agent interaction.This framework considers different dimensions regarding the perception of the ECA (in terms of realism, competence and relevance) as well as user's engagement (involvement and distance) and user's satisfaction.We adapted the questionnaire proposed by Van Vugt and colleagues to measure the behaviour of the ECA along these dimensions (van Vugt et al., 2006).The perceived friendliness of the ECA was measured through the adjectives kind, warm, agreeable and sympathetic of the IAS questionnaire (Wiggins, 1979).
As for the other two studies, we also measured the a-priori attitude of participants towards virtual agents through the NARS questionnaire.

Hypotheses.
Previous studies (Liu et al., 2008;Woolf et al., 2009;Levitan, 2013) found that users' satisfaction about their interaction with an ECA is greater when the ECA adapts its behaviour to user's one.From these results, we could expect that the user would be more satisfied about the interaction when the ECA adapted its low-level signals according to their behaviours.We also assumed that the ECA adapting its lip corner puller (that is related to smiling) would be perceived as friendlier.Thus, our hypotheses were: H1Head: when the ECA adapted its head rotation, the users would be more satisfied with the interaction compared to the users interacting with the ECA in the Random condition.
H2aLips: when the ECA adapted its lip corners movement (AU12), the users would be more satisfied with the interaction compared to the users interacting with the ECA in the Random condition.
H2bLips: when the ECA adapted its lip corners movement (AU12), it would be evaluated as friendlier compared to the ECA in the Random condition.
H3Eyes: when the ECA adapted its eyes rotation, the users would be more satisfied with the interaction compared to the users interacting with the ECA in the Random condition.
H4aAll: when the ECA adapted its head and eyes rotations and lip corners movement, the users would be more satisfied with the interaction compared to the users interacting with the ECA in the Random condition.
H4bAll: when the ECA adapts its head and eyes rotations and lip corners movement, it would be evaluated as friendlier compared to the ECA in the Random condition.

Analysis and Results
101 participants (55 females), almost equally distributed among the 5 conditions, took part of our experiment.95% of participants were native French speakers.32% of them were in the range 18-25 years old, 17% were in the range 25-36, 21% in the range 36-45, 18% in the range of 46-55 and 12% were over 55 years old.For each dimension of the user's engagement questionnaire, as well as for that about the perceived friendliness of the ECA, Cronbach's αs were > 0.8; we then computed the mean of the scores in order to have one score for each dimension.Mean and standard deviation of each measured dimension for each of the five conditions are showed in Table 4.In Head condition, we could not find differences between the conditions.We conclude that the hypothesis H1Head is rejected.
In the Lip Corners condition, compared to participants in the Random condition, participants in the Lip Corners condition were more involved (W = 98.5, p-adj < .05).We can also note that the ECA was evaluated as more positive on the relevance dimension (W = 104.5,p-adj < .05).We can conclude that the hypotheses H2aLips and H2bLips are not validated, but the adaptation of lip corners movement still has a positive effect on other dimensions related to user's engagement.
In the Eyes condition, participants were satisfied with the ECA as with the ECA in the Random condition.
Thus, the hypothesis H3Eyes is rejected.
In the All condition, the ECA was evaluated as friendlier (W = 104.5,p-adj < .05)than the ECA in the Random condition.So, H4aAll is supported, while H4bAll is rejected.

Adaptation Mechanisms in Human-Agent Interaction
Results of the NARS questionnaire indicated that 40%, respectively 30% and 30%, of participants had a positive, respectively neutral and negative, attitude toward virtual agents.An ANOVA test was performed to study the influence of participants' a-priori toward virtual agents on their engagement in the interaction.Participants' prior attitude toward ECAs had a main effect on participants' distance (F (1, 93) = 5.13, p < .05)).Results of pairwise comparisons with Bonferroni adjustment highlighted that participants with prior negative attitude were less engaged (more distant (p-adj< .05)and less involved (p-adj< .05))than those with prior positive attitude.

Discussion
The results of this study showed that participants engagement and perception of ECA's friendliness were positively impacted when the ECA adapted its low-level signals.These results were significant only when the ECA adapted its lip corners movement (AU12) to user's behaviour (mainly their smile), that is, in the Lip Corners and All conditions.In the case of head and eyes rotation adaptation, we found a trend on some dimensions but no significant differences compared to the Random condition.These results could be caused by the adopted evaluation setting where ECA and user faced each other.During the interaction, most participants gazed at the ECA without doing any postural shift or even changing their gaze and head direction.They were mainly still and staring at the ECA.The adaptive behaviours, i.e., head and eyes rotation of the ECA computed from user's behaviours, remained constant throughout the interaction.They reflected participants' behaviours (that were not moving much).Thus, in the Head and Eyes adapting conditions the ECA showed much less expressiveness and may have appeared much less lively, which may have impacted participants' engagement in the interaction.

GENERAL DISCUSSION
In our studies, we applied the Interaction Adaptation Theory (see Section 2) on the ECA.That is, our adapting ECA had the Requirement R that it needed to adapt in order to have a successful interaction.
Its Desire D was to maximise user's experience by eliciting a specific impression towards the user, or maintaining user's engagement.Finally, its expectations E were that the user's experience would be better when interacting with an adaptive ECA.All these factors rely on the general hypothesis that the user expects to interact with a social entity.According to this hypothesis, the ECA should adapt its behaviour like humans do (Appel et al., 2012).
We have looked at different adaptation mechanisms through three studies, each focusing on a specific type of adaptation.In our studies we found these mechanisms impacted user's experience of the interaction and their perception of the ECA.Moreover, in all three studies, interacting with an adaptive ECA vs a non-adaptive ECA tended to be more positively perceived.More precisely, manipulating agent's behaviours (Study 1) had an impact on user's perception of the ECA while low-level adaptation (Study 3) positively influenced user's experience of the interaction.Regarding managing conversational strategies (Study 2), the ECA was perceived warmer when it managed those that increased user's engagement vs when it did not change them all along the interaction.
These results suggest that the IAT framework allows enhancing human-agent interaction.Indeed, the adaptive ECA shows some improvement in the quality of the interaction and the perception of the ECA in terms of social attitudes.
However, not all our hypotheses were verified.This could be related to the fact that we based our framework on the general hypothesis that the user expects to interact with a social entity.The ECA did not take into account the fact that also the user had their specific Requirements, Desires and Expectations, the expectancy to interact with a social agent.Yet, the ECA did not check if the user still considered it as a interaction in real-time.Evaluation studies showed a tendency towards a positive impact of the adaptive ECA on user's experience and perception of the ECA, encouraging us to continue to investigate in this direction.
One limitation of our models is their reliance on the interaction scenario.Indeed, to obtain good performances of adaptation models using reinforcement learning algorithms, a scenario including an adequate number of steps is required.In our case, the agent ended up selecting a specific combination of behaviours only during the late part of the interaction.A longer interaction with more steps would allow an adaptive agent using reinforcement learning algorithms to better learn.Another possibility would be to have participants interacting more than once with the virtual agent.This latter would require adding a memory adaptation module (Ahmad et al., 2017).This would also allow for checking whether the same user prefers the same behaviour and/or conversational strategies of the agent over several interactions.Similarly, regarding adaptation models reflecting user's behaviour, the less the user moves during the interaction, the less the agent's expressivity level is.The interaction scenario should be designed in order to elicit user's participation, including strategies to tickle users when they become too still and non reactive.For example, one could use a scenario including a collaborative task where both agent and user would interact with different objects.In such a setting, though it would require to extend our engagement detection module to include joint attention, we expect the participants would also perform much more head movements that in turn could be useful for a better low-level adaptation of the agent.
In the future, our work could be improved and explored along further axes.We list three of them here.
First, the three models presented in this paper were implemented and evaluated independently from each other.It could be interested to merge the three adaptation mechanisms in a broader model and investigate the impacts of agent's adaptation along different levels at the same time.Second, in our studies, the agent adapted its behaviours to the user's ones without considering if the relationship between the behaviours of the dyad showed any specific interaction patterns.In particular, we have not made explicit if the agent's behaviour should either match, reciprocate, complement, compensate or mirror their human interlocutor's behaviour (Burgoon et al., 2007).Also, we have not measured any similarities, synchronisation or imitation between user's and agent's behaviour when we analysed the data of our studies.Since adaptation may be signalled through a larger variety of behaviour manifestations during an interaction, more adaptation mechanisms could be implemented.One last important direction for future work concerns the improvement of the interaction with the user.This would reduce possible secondary effects of uncontrolled variables, such as user's expectancies, and allow for better studying the effects of agent's adaptation.We aim to improve the agent's conversational skills, to ensure conversation repairs and interruptions and by letting the user choose the topic of conversation (e.g., from a set of possible ones) and drive the discussion.In addition to these improvements, user's expectancies should also be better modelled by taking into account Expectancy Violation Theory in addition to Interaction Adaptation Theory.
Three main interrelated factors contribute to IAT.Requirements (R) refer to the individual beliefs about what is necessary in order to have a successful interaction.R are mainly driven by biological factors, such as survival, safety, affiliation.Expectations (E) refer to what people expect from the others based on social norms or knowledge coming from previous interactions.E are mainly influenced by social factors.Finally, Desires (D) refer to individual's goals and preferences about what to get out of the interaction.D are mainly influenced by person-specific factors, such as temperament or cultural norms.These three factors are used to predict an individual's Interactional Position (IP).This variable derives from the combination of R, E and D, and represents the individual's behavioural predisposition that will influence how an interaction will work.The IP would not necessarily correspond to the partner's Actual behaviour performed in the interaction (A).The relation between IP and A will determine the type of adaptation during the interaction.

Figure 1 .
Figure 1.System architecture: in the User's Analysis module user's non-verbal and verbal signals are extracted and interpreted; user's reaction is sent to the Dialog Model module which computes the dialog act to be communicated by the ECA.The Agent's Behaviour module instantiates the dialog act into multi-modal behaviours to be displayed by the ECA.The Adaptation Mechanism module adapts the agent's behaviour to user's behaviour.Its placement in the architecture depends on the specific adaptation mechanism that is implemented.

Figure 3 .
Figure 3.The modified system architecture used in Study 1.In particular, the User's Analysis module contains the model to detect user's impressions from facial signals.The Impressions Management module contains the Q-learning algorithm.

Figure 4 .
Figure 4. Competence means for each level of Model.* stands for p<0.05.

Frontiers 7 .
3.4 Performance of the adaptation model.

Figure 5 .
Figure 5.The modified system architecture used in Study 2. In particular, the User's Analysis Module contains the model to detect user's engagement from facial and head/trunk signals.The Communicative Intention module uses reinforcement learning to select the agent's self-presentational strategy.

8. 1
.1 User's Analysis: User's Engagement Detection.The User's Analysis module is integrated with a User's Engagement Detection module that continuously computes the overall user's engagement at the end of every speaking turn.The computational model of user's engagement is based on the detection of facial signals and head/trunk signals, that are indicators of engagement.In particular, smiling is usually considered an indicator of engagement, as it may show

Figure 6 .
Figure 6.The modified system architecture used in Study 3. In particular, the User's Analysis module detects user's low-level signals such as head and eyes rotations and lip corners activity.The Adaptation Mechanism module exploits the IL-LSTM model for selecting the agent's low-level signals.In the Agent's Behaviour module, the Behaviour Realiser is customised in order to take into account the agent's communicative behaviours and signals coming from the IL-LSTM module in real-time.

Table 1 .
Mean and standard deviation of w and c scores for each level of Model.

Table 2 .
Mean and SD values of warmth scores for each level of Communicative Strategy.

Table 3 .
Mean and SD values of competence scores for each level of Communicative Strategy.No significant differences among the conditions were found.

Table 4 .
Mean ± standard deviation of each dimension of the questionnaires (each row of the table), for each of the five conditions (each column).* indicates that the score is significantly different compared to the Random condition (p-adj < .05).
As our data were not normally distributed (Shapiro test's p < 0.5), we used unpaired Wilcoxon test (equivalent to t-test) to measure how participants ratings differed between the Random condition and each of the other conditions.