Multi-Modal Anchoring in Infants and Artificial Systems
-
1
Bielefeld University, Cognitive Interaction Technology – Excellence Cluster, Emergentist Semantics, Germany
-
2
Bielefeld University, Technical Faculty; Applied Informatics, Germany
-
3
Bielefeld University, Germany
-
4
Bielefeld University, Cognitive Interaction Technology for Medicine (CITmed), Germany
Abstract
Whereas the facilitating function of intersensory redundancy in the integration of different senses into a multi-modal representation of objects and events is unquestioned, semantic factors only recently are under scope of investigation in infant studies. Arbitrary artificial connections such as between colors and tones can be integrated at the age of 7 months when they are presented in temporal synchrony. Our studies are motivated by the synesthesia research in adults suggesting that emotional and semantic congruence of certain colors and tones (red and high tones, green and low tones) can be perceived. We pursue thus the question of whether such congruence might facilitate the integration of colors and tones in infancy as well. We report the findings of an infant habituation study in which we tested whether congruent stimuli are differentiated at the age of 4 months already. Results indicate a discrimination of a green/low event from a red/high event, possibly motivated by a facilitating factor inherent in the colors and tones. We discuss the role of a proto-semantic similarity in certain colors and tones and propose a synesthetic connection between them from early on in infancy. Based on this evidence, we emphasize an important role of semantic factors that have to be considered as crucial for cross-modal anchoring in artificial systems.
I. INTRODUCTION
Intersensory redundancy fulfills a fundamental function in the development of perceptual systems [1,2]. When they are presented in a temporally synchronous way, infants integrate arbitrary auditory-visual features as colors and tones at the age of 7 months, but not earlier [3]. However, to our knowledge until now it has not been investigated whether a more semantically motivated connection in the sense of adult semiotics would lead to an earlier integration of the senses, e.g., the reason why warning signals are red and of a high pitch (rather than red and low pitch), because we perceive them in convergence. In our study with adults, we found that red and high pitch are perceived to a large extent negatively [4,6]. Green, on the other hand, is related to positive associations [5,6], thus resembling to low pitch in its positivity [6].
II. INFANT HABITUATION STUDY
In this study, we examined whether infants integrate congruent color-tone connections at the age of 4 months, earlier than so far investigated [3]. Congruence is defined as eliciting similar emotions and associations in adults [6].
A. Method
1) Participants: Fourteen 4-month-olds (7 male, 7 female, range 116–134 days, M = 125 days). All infants were full-term, with normal hearing and vision, raised in monolingual German families and with no family history of color-blindness.
2) Stimuli: The visual stimuli consisted of four geometric objects of different shape with low semantic content (Fig 1). They were presented by a hand performing a looming forward movement in all films. This presentation was synchronous to either a high (2500 Hz) or low (475 Hz) sinus tone.
3) Procedure: The infants viewed 3 RED color-HIGH tone events out of 4 possible events (Fig. 1). When the infant looked away for 1 sec, the next habituation trial was shown. The habituation phase ended when the infant’s looking times decreased in 3 consecutive habituation film trials by 50% relative to the first 3 habituation trials. When the criterion was reached, 5 test trials were shown (Fig. 2): One familiar and one unfamiliar object (both red-high events), two uni-modal changes (one GREEN-high event and one red-LOW event), and a bimodal change (a GREEN-LOW event). The test trials were presented in a randomized order and counterbalanced across participants.
B. Results
We compared infants’ looking durations to each test event with the familiar test event. Wilcoxon tests showed longer looking times for the GREEN-LOW event in contrast to the familiar RED-HIGH event (M = 14.8, SD = 11.8; Z(10) = -2.40, p = .016). No other test event was discriminated from the familiar test event (Fig. 3).
C. Discussion
These results suggest that already 4-month-old infants form arbitrary color-tone connections as they differentiate a green/low tone event from a red/high tone event. Two explanations may account for this early discrimination: Infants might have distinguished the bimodal change in contrast to the habituation event, which is easier to detect than a uni-modal change. An alternative explanation is that the discrimination was facilitated by the underlying (congruent) semantics in the green-low tone event. Infants may exhibit a certain predisposition to form these connections that may perseverate into adulthood. The source of this initial proto-semantic knowledge may be seen in infant synesthesia [7], suggesting a cross-talk of sensory cortical areas. We therefore propose that a perceived congruence may support the formation of a connection between a color and a tone. Further studies could focus this problem, within a preferential looking paradigm that is sensible to the recognition of an intermodal relationship, determining the innate biases in the sense of infant synesthesia.
III. CONCLUSION
This study sheds light on how color and tone associations might develop from infancy by revealing certain predispositions in color perception that may be biologically based, as [5] and [7] suggest investigating.
Based on the provided results and concepts we motivate the development of an artificial multi-modal anchoring system that uses the temporally synchronous changes in attribute values as a basis [8]. Functional dependencies between attributes can thus be uncovered and used to facilitate the correspondence detection in ambiguous situations. This is a first step towards content-based strategies for cross-modal fusion. However, our infant study implies such processes on even earlier levels of representation.
Acknowledgements
This research was supported by the German Research Council (DFG), Center of Excellence “Cognitive Interaction Technology” (CITEC, EC 277), Bielefeld University, Germany.
References
[1] L. E. Bahrick & R. Lickliter, “Intersensory redundancy guides attentional selectivity and perceptual learning in infancy,” in Developmental Psychology, 36(2), 2000, 190-201.
[2] D. J. Lewkowicz, “The development of intersensory temporal perception: an epigenetic systems/limitation view,” Psychological Bulletin, 126(2), 2000, 281-308.
[3] L. E. Bahrick, “The development of infants’ sensitivity to arbitrary intermodal relations,” Ecological Psychology, 6(2), 1994, 111-123.
[4] A. J. Elliot, & M. A. Maier, “Color and psychological functioning”, Current Directions in Psychological Science, 16(5), 2007, 250-245.
[5] A. C Moller, A. J. Elliot, & M. A. Maier, “Basic hue-meaning associations”, Emotion, 9, 2009, 898–902.
[6] S. Fischer, K. J. Rohlfing, & M. Piefke, “Emotional Semantics of Colors and Tones”, in prep.
[7] F. Spector & D. Maurer (2009), “Synesthesia-A New Approach to Understanding the Development of Perception”, Developmental Psychology, 45(1), 175- 189.
[8] D. Schulze, S. Wachsmuth, & K. Rohlfing, “Automatic Enhancement of Correspondence Detection in an Object Tracking System”, in Proc. of the ESANN 2011, 423-428.
Keywords:
Artificial systems,
Cross-modal perception,
infants,
Multi-modal anchoring
Conference:
IEEE ICDL-EPIROB 2011, Frankfurt, Germany, 24 Aug - 27 Aug, 2011.
Presentation Type:
Poster Presentation
Topic:
Development and emergence
Citation:
Fischer
S,
Schulze
D,
Borggrebe
P,
Piefke
M,
Wachsmuth
S and
Rohlfing
K
(2011). Multi-Modal Anchoring in Infants and Artificial Systems.
Front. Comput. Neurosci.
Conference Abstract:
IEEE ICDL-EPIROB 2011.
doi: 10.3389/conf.fncom.2011.52.00034
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
02 Sep 2011;
Published Online:
02 Sep 2011.
*
Correspondence:
Mrs. Silke Fischer, Bielefeld University, Cognitive Interaction Technology – Excellence Cluster, Emergentist Semantics, Bielefeld, Germany, silke.fischer@ifs.uni-hannover.de