Janet Wessler
Patrick Gebhard
Sigal Zilcha-Mano- 1Affective Computing Group, Cognitive Assistants, German Research Center for Artificial Intelligence, Saarbruecken, Germany
- 2Psychotherapy Research Lab, Department of Psychology, University of Haifa, Haifa, Israel
1 Introduction
1.1 Parameters and measurement of interactional synchrony
Movement synchrony is the temporal coordination of postures and gestures between interaction partners (1). In contrast to mimicry — which is defined as an unconscious imitation of specific gestures and expressions (e.g., both interaction partners scratch their head) — synchronous movements can take the same or different form as long as they are time-coupled (e.g., one partner scratches the head and the other the leg).
Five general criteria can be used to describe interactional synchrony (2). First, synchrony takes place in a certain context, which could for example be affiliative or competitive. Second, synchrony occurs within a certain modality, e.g. movements or verbal expressions. Third, within a modality, interactants use specific resources for synchrony, e.g., postures or gestures within the movement modality. Fourth, synchrony is characterized by entrainment, that is, who is leading or following the synchrony. Fifth, synchrony occurs with different time lags, from 0 seconds and perfect matching up to a few seconds. However, the exact parameters of time lag for synchrony vary between studies, for example 5 seconds (3) or 2.5 seconds (4). Thus, currently, it remains an open question which time lag of synchrony is the most appropriate for certain contexts and modalities (2, 5). Moreover, Scheidt and colleagues (2) specified eight dimensions which are specific to movement synchrony movement, that is, its spatial directions (e.g., towards or far from the other), its amplitude, sinuosity, duration, event structure, phase shifts, frequency in a given time period, and the content of synchrony (e.g., same versus different body parts).
Investigating such temporal dynamics of nonverbal behaviors between human interaction partners is complex and typically involves immense human resources and time efforts. Participants are typically video-recorded, and the interactions are annotated by blind coders. In recent years, developments in technology partially allowed for automatic analysis of certain behaviors using software like OpenFace (6), OpenPose (7) or Movement Energy Analysis (MEA; 8–11). Through certain algorithms, such software quantifies the amount of nonverbal behavior which is present in the videos, thus facilitating the measurement of synchrony profoundly. In psychotherapeutic research, movement synchrony is typically measured by MEA (3, 8). Computer algorithms calculate the movement energy as the amount of pixel changes between picture frames. Then, cross-correlations between this quantified movement energy are calculated to find covariations in the movements between interactants within a time-lag of between 0 sec and 10 sec. Then, these synchrony values are compared against levels of so-called pseudo-synchrony, in which random subsets of the interaction are paired and cross-correlations are computed.
1.2 Effects of synchrony on the dyadic relationship
In the past few years, a great interest emerged in measuring movement synchrony between the patient and therapist in psychotherapy because it is assumed to represent an important component of the therapeutic relationship, or even serve as its underlying mechanism (12). In the Interpersonal synchrony (In-sync) model of psychotherapy (12) movement synchrony between interaction partners and inter-brain coupling between them mutually influence each other positively. The process of alignment takes place within a time frame of a few milliseconds to 10s via a simple association between perception and motor movements. The inter-brain coupling then facilitates higher cognitive processes that involve language and reasoning. These processes lead to alliance formation and strengthening — a positive working relationship between patient and therapist.
So far research had assumed that the synchrony-outcome relationship is linear, that is, more synchrony leads to better outcomes, even though this relationship is small (13, 14). However, recent arguments claim that this might not be always the case (1). However, recent arguments claim that this might not be always the case (15). More synchrony could also lead to adverse, unwanted effects. This is due to the fact that humans are not only driven by a need to affiliate and belong (e.g. 16–18), but similarly by a need for uniqueness, autonomy, and distinguishness (e.g., 19). Thus, when patients are rather in search for independence from others, the feeling of being in sync and aligned with the therapist might have adverse effects on the therapeutic outcome (1).
Several studies showed a positive association between movement synchrony and working alliance and positive treatment outcomes during psychotherapy (3, 8, 20–22). However, whereas in some studies the effect reached significance (e.g., 3, 23), in others it did not (e.g., 7, 24). Yet other studies reported that no association exists, or even that higher synchrony is associated with a poorer process and outcome of treatment (25–27).
Focusing on the association between the strength of the patient-therapist working alliance and the level of synchrony between them, a similar pattern of mixed results emerges. Many of the studies examining the association between synchrony and alliance found a positive association, so that higher synchrony was associated with stronger alliance; in some, the effect reached significance (e.g., 20, 23, 28), whereas in others, it did not (e.g., 29). Furthermore, three studies found a negative association (25, 30, 31), with a stronger alliance being associated with lower levels of synchrony.
In summary, the existing line of research remains rather inconclusive about which parameters of movement synchrony exactly drive its positive effects, if it has always positive effects, and which factors might drive negative effects of movement synchrony — more conclusive studies are needed (2).
One of the main drawbacks of the current studies is that they are mainly correlational in nature. The manipulation of synchrony in psychotherapeutic settings usually is not feasible, either from an ethical point of view or due to the study designs. Moreover, it would have to involve excessive training of confederates to be able to react with standardized levels of synchrony towards every participant.
However, in order to investigate causal relationships instead of (cross-)correlations between nonverbal behaviors and higher-level outcomes of the interaction, it is necessary to manipulate certain parameters of nonverbal behaviors. For example, mimicry research has relied on human confederates to manipulate the amount of mimicry between interactants (32). Although human confederates should be extensively trained before conducting such studies, a certain variance between interactions can occur, confederates might not be blind, and an experimental manipulation of fine-grained parameters like the time-lag of the imitation is difficult (see 33). Due to these challenges, mimicry research has employed socially interactive agents (SIAs; 34) in order to keep the mimicry manipulation as controlled as possible (33, 35). In general, using virtual reality has many advantages for psychological experiments (36). SIAs have a physical representation, which is similar to but does not equal humans. They can detect social signals like smiles and movements, analyze them and react with similar behaviors themselves with a certain time-lag, that is, engage in mimicry. The reasoning behind such research is that some equivalence between humans and computers is assumed in human-computer interaction (37), manipulating nonverbal behaviors of SIAs in human-SIA interaction could serve as a proxy for studying consequences of nonverbal behaviors in human-human interactions. Thus, it was for example possible to experimentally test the effects of time-lags of 1 sec vs. 3 sec mimicry from either ingroup are outgroup SIAs on interpersonal outcomes like liking and trust (33). Still, other research on interpersonal coordination like movement synchrony has not yet explored the possibility of using SIAs as an experimental tool. There is a need for a virtual testing environment to experimentally test synchrony-related hypotheses (36). With the proposed conceptual model, we aim at creating an experimental framework using SIAs which then can be used to fill these research gaps.
2 Building an experimental toolkit with SIAs
As an experimental toolkit for exploring synchrony and related concepts, we propose a highly parameterisable real-time architecture using a SIA. To model the SIA’s synchrony behavior, we will use both theory-driven and data-driven approaches. For example, data collected from thousands of therapeutic sessions can be used for machine learning (38, 39). This real-time architecture then allows to configure and support relevant aspects such as context (e.g., competitive or affiliative), modality (speech, movement), resources (head, arm), entrainment (leading, following), and time lag (delay in milliseconds). Figure 1 shows an overview of the proposed experimental toolkit.
Figure 1 Overview of the architecture and modules of the experimental toolkit for studying synchrony between humans and socially interactive agents. SIA obtained from Charamel, https://www.charamel.com.
The toolkit allows to study synchrony with an SIA in a fully automated mode up to fully human controlled (wizard). In each mode, synchrony parameters (Figure 1, middle) can be manipulated at any time. The automated mode relies on the following additional technological SIA components (34) (Figure 1, right side), such as social signal analysis, automatic speech recognition (ARS), natural language understanding (NLU), large language models (LLM) (40), GUI modules for questionnaires, and user models (affect, medical) which drive together the dialogue and behavior models for a (configurable) real-time behavior rendering of the SIA. LLMs are used to create context information which is integrated into existing authoring frameworks (e.g., 41) for SIAs for further adaptation and verifications by experts. In each mode, relevant timing information, such as user (reactions), module processing time, and the participants’ signals (voice, video) are recorded for further data analysis.
When investigating movement synchrony with SIAs, time delay, and intensity are essential aspects that are carefully controlled by the proposed experimental toolkit. Controlling the time delay involves thoroughly understanding important parameters and the data and information processing pipeline. In the autonomous mode, a significant movement (e.g., head nodding) has to be detected by social signal classifiers before any observable animation as a reaction of the SIA is shown. Usually, there is a defined configurable movement threshold. The movement range (e.g., head tilt angle from 2 – 10 degrees) correlates to the intensity (e.g., weak - strong head nodding). Usually, the detection takes between 50 – 200 ms depending on which modalities (face, head, body, or any combination of them) are involved and which movement thresholds are defined. After that period, a suitable action is selected, and after a defined delay is shown by the SIA. The delay is usually between 500 – 2000 ms (or longer), and the observable reaction of the SIA usually takes as long as the detected social signal but can be altered concerning timing and intensity. The minimal system intrinsic reaction time is between 100 – 400 ms + 20 ms (max. approximated) inter software module delay. In general, an important toolkit feature is accurately logging such events for future analysis. This is especially important if a human wizard does the detection of social signals and the selection of the movement modality.
3 Exemplary clinical study
Let us imagine depressive patients use the toolkit in a daily diary study in which they interact with the SIA for about 10 minutes over 8 days via an App on a mobile device at home (42). The SIA asks the patient about his/her day and how s/he feels. The dialogue model represents the questions based on a specific therapy model. While listening, the toolkit analyses the patient’s social signals (e.g., smiles, head movements) and recognizes the speech content via ASR and NLU. At best, the ASR and NLU are trained for the specific requirements to improve their accuracy. LLMs (retrained for the specific situation) are used for an automatic contextual annotation that might be useful at later stages for an automatic generation of synchrony behavior. On random days, patients are exposed to the no-delay condition: the SIA synchronizes with the patient’s movements within typical technological limits (between 200-1000 milliseconds). On the other days, patients are randomly exposed to the 3 sec-delay, 5 sec-delay, or 7 sec-delay synchrony conditions: the SIA synchronizes with the patient’s movements after a delay of 3, 5, or 7 sec. Synchronizing here means that the SIA might not mimic the exact movement of the patient, but, for example, might move their arm when the patient moves the head (i.e., modality switch). The agent’s behavior is driven by rule-based, learned, or hybrid computational behavior models or controlled by a wizard (WOZ control interface). Afterward, the patient will indicate their alliance with the SIA during the interaction in a questionnaire, including their general rapport and how much they trusted them.
Moreover, before and after the eight days of interaction a patient’s symptoms will be assessed to measure the interaction outcome. Thus, we can test the hypothesis if there is an optimal level of synchrony for its positive effects to occur in this setting (5). Moreover, we can explore whether the optimal level is individual-specific or more universal. If the optimal level is universal, it will be possible to train therapists (real and virtual) to synchronize with their patients at this optimal level. If the optimal level is individual-specific, it will be possible to provide individuals with feedback to raise awareness of their optimal level. It will also be possible to provide therapists with training on how to identify the optimal level of each of their patients and how to optimize the synchrony level accordingly.
4 Possible future research questions
With the proposed experimental toolkit, psychotherapeutic researchers have the possibility to experimentally investigate various types of currently open research questions. For example, one important question to investigate is how specific timing lags of synchrony influence the working alliance between a SIA as a therapist and the user. One possibility here is that the relationship between time lag and working alliance is invertedly U-shaped (Wessler et al., under review): very low (e.g., 1 sec) and very high (e.g., 8 sec) time lags lead to lower levels of working alliance compared to moderate time lags (e.g., 3-5 sec). Also, such a curvilinear relationship might be shifted by interindividual preferences and interpersonal tendencies. One such interpersonal tendency in the context of synchrony could be a person’s attachment style (16, 18). While anxiously-attached individuals might prefer rather short time delays, avoidant individuals might prefer longer time delays for a positive working alliance with their therapist. Even more, such preferences might be sensible to change throughout the course of psychotherapy. Results of possible future studies testing such questions would help to clarify the theorizing about interactional synchrony in therapeutic settings and to derive practical recommendations for psychotherapists.
5 Conclusion
Here we outline the necessity to build an experimental toolkit to further investigate the open research questions about the relationship between movement synchrony and therapy-related outcomes. We propose an experimental toolkit in which certain parameters of synchrony behaviors of SIAs can be specifically manipulated in a controlled manner, such as time-lag. Such a toolkit would facilitate the resource-intensive character of synchrony research, enable experimental manipulations, and help to derive practical recommendations.
Author contributions
JW: Conceptualization, Project administration, Writing – original draft. PG: Conceptualization, Software, Supervision, Writing – review & editing. SS: Conceptualization, Supervision, Writing – review & editing.
Funding
The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work is partly funded by the German Research Foundation (DFG) within the DEEP project (funding code 392401413) and by the German Federal Ministry for Education and Research (BMBF) within the UBIDENZ project (funding code 13GW0568D). The publication of this paper was also supported by European Research Council (ERC) under the European Union’s Horizon Research and Innovation Programme (grant agreement no.101123661 SynSig).
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1. Mayo O, Gordon I. In and out of synchrony—Behavioral and physiological dynamics of dyadic interpersonal coordination. Psychophysiology (2020) 57(6):e13574. doi: 10.1111/psyp.13574
2. Scheidt CE, Pfänder S, Ballati A, Schmidt S, Lahmann C. Language and movement synchronization in dyadic psychotherapeutic interaction–a qualitative review and a proposal for a classification. Front Psychol (2021) 12:696448. doi: 10.3389/fpsyg.2021.696448
3. Ramseyer F, Tschacher W. Nonverbal synchrony in psychotherapy: coordinated body movement reflects relationship quality and outcome. J Consult Clin Psychol (2011) 79(3):284. doi: 10.1037/a0023419
4. Altmann U. (2011). Investigation of movement synchrony using windowed cross-lagged regression, in: Analysis of Verbal and Nonverbal Communication and Enactment. The Processing Issues: COST 2102 International Conference, Budapest, Hungary, September 7-10, 2010, Revised Selected Papers. 335–45. Springer: Berlin Heidelberg. doi: 10.1007/978-3-642-25775-9_31
5. Wessler J, Loschelder DD, Fendel JC, Friese M. The too-much-mimicry effect: Strong (vs. subtle) mimicry impairs liking and trust in distributive negotiations. J Nonverbal Behav (2023). doi: 10.1007/s10919-023-00446-5
6. Amos B, Ludwiczuk B, Satyanarayanan M. Openface: A general-purpose face recognition library with mobile applications. CMU School Comput Sci (2016) 6(2):20.
7. Cao Z, Simon T, Wei SE, Sheikh Y. (2017). Realtime multi-person 2d pose estimation using part affinity fields, in: Proceedings of the IEEE conference on computer vision and pattern recognition, (New York City, USA: Institute for Electrical and Electronics Engineers). pp. 7291–9. doi: 10.1109/TPAMI.2019.2929257
8. Ramseyer F, Tschacher W. Synchrony: A core concept for a constructivist approach to psychotherapy. Constr Hum Sci (2006) 11(1-2):150–71.
9. Altmann U. Synchronisation des nonverbalen Verhaltens. Wiesbaden: Springer Fachmedien: Wiesbaden (2013) 7–17. doi: 10.1007/978-3-531-19815-6
10. Fujiwara K, Kimura M, Daibo I. Rhythmic features of movement synchrony for bonding individuals in dyadic interaction. J Nonverbal Behav (2020) 44(1):173–93. doi: 10.1007/s10919-019-00315-0
11. Kleinbub JR, Ramseyer FT. rMEA: An R package to assess nonverbal synchronization in motion energy analysis time-series. Psychother Res (2021) 31(6):817–30. doi: 10.1080/10503307.2020.1844334
12. Koole SL, Tschacher W. Synchrony in psychotherapy: A review and an integrative framework for the therapeutic alliance. Front Psychol (2016) 7:862. doi: 10.3389/fpsyg.2016.00862
13. Wiltshire TJ, Philipsen JS, Trasmundi SB, Jensen TW, Steffensen SV. Interpersonal coordination dynamics in psychotherapy: A systematic review. Cogn Ther Res (2020) 44:752–73. doi: 10.1007/s10608-020-10106-3
14. Atzil-Slonim D, Soma CS, Zhang X, Paz A, Imel ZE. Facilitating dyadic synchrony in psychotherapy sessions: Systematic review and meta-analysis. Psychother Res (2023) 67(4):1–20. doi: 10.1080/10503307.2023.2191803
15. Zilcha-Mano S. How getting in sync is curative: insights gained from research in psychotherapy. Psychol Rev [Online first] (2024).
16. Bowlby J. Developmental psychiatry comes of age. Am J Psychiatry (1988) 145(1):1–10. doi: 10.1176/ajp.145.1.1
17. Baumeister RF, Leary MR. The need to belong: desire for interpersonal attachments as a fundamental human motivation. psychol Bull (1995) 117(3):497. doi: 10.1037/0033-2909.117.3.497
18. Mikulincer M, Shaver PR. Attachment, group–related processes, and psychotherapy. Int J Group Psychother (2007) 57(2):233–45. doi: 10.1521/ijgp.2007.57.2.233
19. Deci EL, Ryan RM. Self-determination research: Reflections and future directions. In: Deci EL, Ryan RM, editors. Handbook of self-determination research. Rochester: University of Rochester Press (2002). p. 431–41.
20. Ramseyer F, Tschacher W. Nonverbal synchrony of head-and body-movement in psychotherapy: different signals have different associations with outcome. Front Psychol (2014) 5:979. doi: 10.3389/fpsyg.2014.00979
21. Ramseyer F, Tschacher W. Movement coordination in psychotherapy: Synchrony of hand movements is associated with session outcome. A single-case study. Nonlinear Dyn Psychol Life Sci (2016) 20(2):145–66. doi: 10.7892/boris.92657
22. Cohen K, Ramseyer FT, Tal S, Zilcha-Mano S. Nonverbal synchrony and the alliance in psychotherapy for major depression: disentangling state-like and trait-like effects. Clin psychol Sci (2021) 9(4):634–48. doi: 10.1177/2167702620985294
23. Altmann U, Schoenherr D, Paulick J, Deisenhofer AK, Schwartz B, Rubel JA, et al. Associations between movement synchrony and outcome in patients with social anxiety disorder: Evidence for treatment specific effects. Psychother Res (2020) 30(5):574–90. doi: 10.1080/10503307.2019.1630779
24. Prinz J, Boyle K, Ramseyer F, Kabus W, Bar-Kalifa E, Lutz W. Within and between associations of nonverbal synchrony in relation to Grawe’s general mechanisms of change. Clin Psychol Psychother (2021) 28(1):159–68. doi: 10.1002/cpp.2498
25. Reich CM, Berman JS, Dale R, Levitt HM. Vocal synchrony in psychotherapy. J Soc Clin Psychol (2014) 33(5):481–94. doi: 10.1521/jscp.2014.33.5.481
26. Ramseyer FT. Exploring the evolution of nonverbal synchrony in psychotherapy: The idiographic perspective provides a different picture. Psychother Res (2020) 30(5):622–34. doi: 10.1080/10503307.2019.1676932
27. Schoenherr D, Strauss B, Paulick J, Deisenhofer AK, Schwartz B, Rubel JA, et al. Movement synchrony and attachment related anxiety and avoidance in social anxiety disorder. J Psychother Integr (2021) 31(2):163. doi: 10.1037/int0000187
28. Shapira N, Atzil-Slonim D, Tuval-Mashiach R, Shapira O. Measuring linguistic synchrony in psychotherapy. In: Proceedings of the Eighth Workshop on Computational Linguistics and Clinical Psychology (Seattle, USA: Association for Computational Linguistics) (2022) 158–76. doi: 10.18653/v1/2022.clpsych-1.14
29. Paulick J, Deisenhofer AK, Ramseyer F, Tschacher W, Boyle K, Rubel J, et al. Nonverbal synchrony: A new approach to better understand psychotherapeutic processes and drop-out. J Psychother Integr (2018) 28(3):367. doi: 10.1037/int0000099
30. Aafjes-van Doorn K, Porcerelli J, Müller-Frommeyer LC. Language style matching in psychotherapy: An implicit aspect of alliance. J Couns Psychol (2020) 67(4):509. doi: 10.1037/cou0000433
31. Gernert CC, Nelson A, Falkai P, Falter-Wagner CM. Synchrony in psychotherapy: High physiological positive concordance predicts symptom reduction and negative concordance predicts symptom aggravation. Int J Methods Psychiatr Res (2023) 33(1):e1978. doi: 10.1002/mpr.1978
32. Chartrand TL, Bargh JA. The chameleon effect: The perception–behavior link and social interaction. J Pers Soc Psychol (1999) 76(6):893. doi: 10.1037/0022-3514.76.6.893
33. Hale J, Hamilton AFdC. Testing the relationship between mimicry, trust and rapport in virtual reality conversations. Sci Rep (2016) 6:35295. doi: 10.1038/srep35295
34. Lugrin B. Introduction to socially interactive agents. In: The Handbook on Socially Interactive Agents: 20 years of Research on Embodied Conversational Agents, Intelligent Virtual Agents, and Social Robotics Volume 1: Methods, Behavior, Cognition (New York, NY, USA: Association for Computing Machinery) (2021). p. 1–20. doi: 10.1145/3477322.3477324
35. Bailenson JN, Yee N. Digital chameleons: Automatic assimilation of nonverbal gestures in immersive virtual environments. Psychol Sci (2005) 16(10):814–9. doi: 10.1111/j.1467-9280.2005.01619.x
36. Pan X, Hamilton AFDC. Why and how to use virtual reality to study human social interaction: The challenges of exploring a new research landscape. Br J Psychol (2018) 109(3):395–417. doi: 10.1111/bjop.12290
37. Reeves B, Nass C. The media equation: How people treat computers, television, and new media like real people Vol. 10. Cambridge, UK: Cambridge University Press (1996).
38. Zilcha-Mano S, Dolev T, Leibovich L, Barber JP. Identifying the most suitable treatment for depression based on patients’ attachment: Study protocol for a randomized controlled trial of supportive-expressive vs. supportive treatments. BMC Psychiatry (2018) 18:1–9. doi: 10.1186/s12888-018-1934-1
39. Zilcha-Mano S, Shahar B, Fisher H, Dolev-Amit T, Greenberg LS, Barber JP. Investigating patient-specific mechanisms of change in SET vs. EFT for depression: study protocol for a mechanistic randomized controlled trial. BMC Psychiatry (2021) 21(1):1–11. doi: 10.1186/s12888-021-03279-y
40. Bubeck S, Chandrasekaran V, Eldan R, Gehrke J, Horvitz E, Kamar E, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712. (2023). doi: 10.48550/arXiv.2303.12712
41. Gebhard P, Mehlmann G, Kipp M. Visual scenemaker—a tool for authoring interactive virtual characters. J Multimodal User Interf (2012) 6(1-2):3–11. doi: 10.1007/s12193-011-0077-1
42. Gebhard P, Schneeberger T, Dietz M, André E, Bajwa NUH. (2019). Designing a mobile social and vocational reintegration assistant for burn-out outpatient treatment, in: Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents, (New York, NY, USA: Association for Computing Machinery). pp. 13–5.
Keywords: synchrony, psychotherapy, working alliance, socially interactive agents, machine learning, experimental design
Citation: Wessler J, Gebhard P and Zilcha-Mano S (2024) Investigating movement synchrony in therapeutic settings using socially interactive agents: an experimental toolkit. Front. Psychiatry 15:1330158. doi: 10.3389/fpsyt.2024.1330158
Received: 30 October 2023; Accepted: 16 January 2024;
Published: 05 February 2024.
Edited by:
Steven Fernandes, Creighton University, United StatesReviewed by:
Warren Mansell, The University of Manchester, United KingdomCopyright © 2024 Wessler, Gebhard and Zilcha-Mano. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Janet Wessler, janet.wessler@dfki.de