Evolutionary robotics: model or design?
- Laboratory of Autonomous Robotics and Artificial Life, Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
In this paper, I review recent work in evolutionary robotics (ER), and discuss the perspectives and future directions of the field. First, I propose to draw a crisp distinction between studies that exploit ER as a design methodology on the one hand, and studies that instead use ER as a modeling tool to better understand phenomena observed in biology. Such a distinction is not always that obvious in the literature, however. It is my conviction that ER would profit from an explicit commitment to one or the other approach. Indeed, I believe that the constraints imposed by the specific approach would guide the experimental design and the analysis of the results obtained, therefore reducing arbitrary choices and promoting the adoption of principled methods that are common practice in the target domain, be it within engineering or the life sciences. Additionally, this would improve dissemination and the impact of ER studies on other disciplines, leading to the establishment of ER as a valid tool either for design or modeling purposes.
For about 25 years, evolutionary robotics (ER) has aroused much interest for its ability to automatically synthesize robots and their behavior, and to model the emergence of adaptive abilities within a Darwinian selection process (Lipson, 2005; Floreano et al., 2008). Soon after its introduction, ER demonstrated a great potential for synthesizing efficient robot controllers that exploit the properties of a fine-grained sensorimotor coordination. Similar results were difficult to obtain with traditional engineering/AI approaches, mainly due to the lack of precise models to deal with the inherent stochasticity and uncertainty of the robot–environment interaction. However, the continuous development of better robots with more precise and informative sensors, together with more advanced techniques and control approaches, has eroded the advantage that ER first manifested, at least in the single-robot domain (Thrun et al., 2005; Siegwart et al., 2011; Zucker et al., 2011). At the same time, the scope of ER studies significantly broadened, in the attempt to provide solutions to the control problems of ever more complex robotic systems (Paul et al., 2006; Baldassarre et al., 2007), as well as to address problems relevant to cognitive sciences and evolutionary biology (see, for instance, Tuci et al., 2011; Wischmann et al., 2012). Unfortunately, this broadening has not led to the establishment of ER as a mature discipline. In my opinion, the causes are to be found in the observation that ER studies often oscillate between incompatible lines of action, in which both modeling and design approaches are present. It is my conviction that this problem is a by-product of the strong bio-inspiration that informs ER studies. In advocating a resurgence of ER in mainstream robotics, Stanley (2011) states that “when applied to robotics, evolutionary algorithms should produce artifacts that remind us of nature, which provides our primary inspiration for running evolution in the first place.” If this is acceptable when the target is producing robots that display “the robustness and fluidity of organisms in nature” (Stanley, 2011), it also bears the risk of promoting studies that veer between providing design solutions and modeling biological phenomena. Indeed, it is clear that much work in ER is affected by the difficulty of disentangling modeling biological systems from designing artificial ones. I believe that sensible progress can be achieved only through a strong commitment to one or the other approach, in order to convey clear-cut messages and unlock the full potential of ER. It is important to acknowledge that the need to shift toward a sharper distinction between modeling and design has recently started to be recognized in the community, as also demonstrated by several studies that maintain either one or the other stance. This paper seeks to reinforce this attitude by providing a selection of studies that should be considered as best practice (see Section Design and Section Model). Far from being a comprehensive review of the field, the handful of selected papers contains several methodological aspects that should be followed in future studies to properly exploit the potential of ER in modeling biological systems or designing artificial ones. The interested reader can refer to recent reviews highlighting the variety of methodologies and problems addressed in ER (Floreano and Keller, 2010; Bongard, 2013; Bongard and Lipson, 2014; Doncieux and Mouret, 2014).
As mentioned above, ER has been proposed principally as an automatic design methodology for robotic systems, following the idea that the experimenter would be exempted from arbitrary, sub-optimal choices in the definition of the robot controller, and leaving to the evolutionary machinery the burden of finding the best controller for a given performance metric. In this respect, it is tempting to consider the evolutionary algorithm as a black-box optimization tool for the robotics problem at hand, following the misconception that biological evolution always produces optimal designs. However, as noted by Doncieux and Mouret (2014), ER is not black-box optimization: there exist several design choices that interact with the optimization process, and determine its fate. In other words, the definition of an ER experiment requires several choices that de facto shift the engineering problem from the robotic system to the evolutionary setup (Trianni and Nolfi, 2011). Each choice may introduce selective pressures and/or alter the search space (Doncieux and Mouret, 2014), and needs to be performed with a principled methodological approach. Without a strong commitment to produce an engineering methodology, the success of ER as a design tool is bounded by the ingenuity of the experimenter. It is therefore necessary to abandon the approach in which ER should work just because it is a model of natural evolution, and instead focus on providing design guidelines and/or task-independent methods that make ER suitable for engineering robotic systems. Trianni and Nolfi (2011) identify four different aspects that need to be correctly engineered in ER. In the following, I report these four aspects and review some related work.
The robot morphology and sensorimotor configuration constitutes the interface between the external world and the control system. This also includes the processing of raw sensor readings (e.g., feature extraction from the camera images), of actuator outputs (e.g., redundant encodings), as well as the definition of communication protocols. The correct engineering of the robot configuration can lead to improved performance: Trianni and Nolfi (2011) demonstrate how an appropriately defined communication protocol can lead to better scalability in a synchronization task with respect to the naive protocol previously defined (Trianni and Nolfi, 2009). Similarly, Fehérvári et al. (2013) show the performance difference related to varying sensorimotor configuration in the evolution of a coordinated motion behavior. These studies demonstrate that an appropriately defined sensorimotor configuration can ease the evolutionary process in finding optimal solutions. Pushing the concept further, Pfeifer et al. (2007) introduced morphological computation as the ability of the physical body to perform actual computations in support of the organism adaptive behavior [see also Paul (2006)]. It is therefore possible – and sometimes advisable – to co-evolve the robot morphology and control system, because certain aspects of the brain–body–environment dynamics can be devolved to appropriately defined morphologies and can result in more robust and efficient systems. For instance, Bongard (2010) demonstrates that the advantage of placing the robot’s morphology under evolutionary optimization increases with task complexity.
The genotype-to-phenotype mapping represents the link between the evolutionary algorithm and the robotic system. The choice of the mapping clearly influences the efficiency of the evolutionary optimization. The most common approach is the usage of a direct encoding that provides a one to one correspondence between genotype and phenotype. However, indirect encodings may prove particularly beneficial for complex tasks presenting some regularity (Clune et al., 2011). Mapping genotype to phenotypes is also challenging in the collective robotics context, where it is necessary to specify the behavior of a group/team of robots. Lichocki et al. (2013) demonstrate how to engineer an evolutionary experiment when a single genotype encodes the control system for the whole team. Instead, when genotypes define the controllers of individual robots, it is necessary to appropriately assemble the groups from multiple genotypes. Here, genetic relatedness has a bearing in the evolution of collective behaviors, as shown by Waibel et al. (2009) in a foraging task.
The fitness function and the evolutionary algorithm determine the way in which potential solutions are retained or discarded, and how the search space is explored. In ER, much attention has been dedicated to the definition of the fitness function, while algorithms have been mostly mediated from research in evolutionary computation. Nelson et al. (2009) provide a comprehensive review of the different methods employed to define the fitness function. Improvement on conventional methods can be achieved by techniques that enhance the ability to search the space of all potential solutions. Trianni and López-Ibáñez (2014) discuss the usage of multi-objective optimization in ER, and identify the related advantages. Lehman and Stanley (2011) propose “novelty search” as a methodology of avoiding deception from ill-defined fitness functions, and to explore the solution space more widely. Mouret and Doncieux (2012) join the advantages of task-specific metrics and novelty search in a multi-objective optimization paradigm, showing its superiority with respect to other approaches to maintain diversity during task-dependent evolution. For a broad review of selective pressures in ER (not limited to the fitness function and the evolutionary algorithm), see Doncieux and Mouret (2014), in which a broad division is proposed among techniques to refine the goal of the evolutionary optimization and techniques to support the search process.
The ecological context represents the possible task variations that need to be tackled. This normally corresponds to varying starting positions of the robots, as well as varying parameters of the task environment. The way in which the ecological context is explored during the evolutionary optimization may produce selective pressures that have a bearing on the flexibility and robustness of the generated solutions [see also the already mentioned review by Doncieux and Mouret (2014)]. For instance, Ampatzis et al. (2008) show that communication and cooperation evolve solely as a result of the ecological conditions encountered by the evolving robots. The ecological context also has a bearing on the so called “reality gap” problem, that is, the difficulty in transferring controllers evolved in simulation to the real world. Recent studies propose to alternate evolution in simulation with tests on the real robot, in order to reduce the effect of simulation errors (Bongard et al., 2006; Koos et al., 2013). Francesca et al. (2014) frame this problem – and more generally, the generalization problem in ER – as the bias–variance trade-off well studied in machine learning. They propose the automatic optimization of probabilistic finite state machines, which allow us to inject some “bias” from the experimenter and conversely to reduce the variance in the obtained results, obtaining more robust solutions, which also efficiently transfer to real robots. A radically different approach to deal with the ecological context is to resort to embodied evolution, having the evolutionary algorithm running on a group of physical robots while they perform their task. Bredeche et al. (2012) introduce a task-independent method for environment-driven adaptation, in which selective pressures are determined solely by the ability of genotypes to diffuse within the robot population. Haasdijk et al. (2014) propose an approach to embodied evolution in which both environment-driven adaptation and task-specific selective pressures are present. In both systems, it is possible to continuously adapt to unknown and dynamically changing environmental conditions.
Modeling has always been an important component of evolutionary computation studies when applied to the behavior of artificial agents. Modeling studies were originally developed within the Artificial Life and Adaptive Behaviors communities. Here, robots are considered as artificial organisms immersed in a synthetic ecology, and are often referred to as “animats” (Meyer and Wilson, 1991). In this context, ER offers the unique possibility to run evolutionary experiments in silico, where robotic organisms undergo a Darwinian selection process that shapes their morphology and behavior within the given ecological context. In principle, ER allows us to identify the causal relationship between selective pressures and adaptive traits, thanks to the possibility of having complete control over the evolutionary process. In this context, the modeling effort is focused on the evolutionary dynamics. Wischmann et al. (2012) showed that evolutionary drift can determine the communication strategy adopted by a population of robots engaged in a foraging scenario, and showed the existence of a trade-off between communication efficiency and robustness from invasion of different strategies. Mitri et al. (2011) studied a population of evolving robots competing for food, and demonstrated that genetic relatedness influences the reliability of the emitted signal: the higher the genetic relatedness, the lower the signal ambiguity. In non-homogeneous groups, genetic relatedness can compensate for the costs of cooperation, as theorized by Hamilton (1964). An experimental demonstration of Hamilton’s theory has been provided by Waibel et al. (2011) by evolving robots for cooperation and carefully controlling the genetic relatedness. Olson et al. (2013) provided an experimental demonstration of the emergence of swarming behavior as a means to confound predators. When the perceptual accuracy of predators is reduced by the presence of multiple aggregated agents, swarming spontaneously evolves in the prey population. By co-evolving predators and prey, Olson et al. (2013) also found that predators evolve a refined perceptual system to reduce the confusion effect, similar to that observed in nature for predators specialized in hunting of swarming animals. Elfwing and Doya (2014) propose an artificial ecology in which polymorphic mating strategies emerge as a result of opposing pressures to collect energy and mate with conspecifics, and demonstrate the evolutionary stability of polymorphism and its relation to the availability of energy resources in the environment. Finally, Auerbach and Bongard (2014) studied the influence of environmental features on the morphological complexity of virtual creatures in which the whole body plan and its motion control were under evolutionary pressure for efficient locomotion. They conclude that the environment plays a crucial role in determining the complexity of the body plan, which is varied to match the requirements imposed by the complexity of the locomotion problem.
In contrast with the evolutionary studies mentioned above, ER can be exploited to automatically generate animats with relevant behavioral and cognitive abilities. In this case, artificial evolution serves just as the optimization process, and could in principle be replaced by any other method of synthesizing the animat. In other words, the modeling effort focuses on the animat itself, and not on the process of obtaining it. One of the most influential approaches in this respect consists in minimizing the complexity of the animat and its environment in order to focus only on the phenomenon of interest. Beer (2003) introduced the concept of “minimally cognitive behavior”, that is, “the simplest behavior that raises issues of genuine cognitive interest”. In this study, Beer (2003) shows how a categorical perception problem can be solved through the coupled brain–body–environment dynamics displayed by a minimalistic agent. The task requires the agent to categorize the shape of 2D objects, which can be either circles or diamonds. The minimalistic agent moves on a line to catch or avoid objects while they fall, and perceives them solely through proximity sensors that give no direct information on their shape. Categorization is operationally defined as the ability to catch circles and avoid diamonds. The study is a prototypical case in which a supposedly complex cognitive ability is demonstrated in a very simplified system, which however preserves the relevant brain–body–environment interactions that pertain to biological systems (Beer, 2003). Following this study, a number of similar minimalistic approaches have been proposed to account for the emergence of cognition in artificial systems (Barandiaran, 2006; Dale and Husbands, 2010; Buhrmann et al., 2013; Iizuka et al., 2013). In few cases, a specific parallel with the target biological system has been proposed. Froese and Di Paolo (2010) showed that structured interaction between agents can be achieved without postulating the need for internalization of the other’s mental state. They replicated a minimalist experiment of perceptual crossing performed with human subjects (Auvray et al., 2009), showing that evolved agents could solve the problem solely on the basis of the interaction dynamics. Vickerstaff and Di Paolo (2005) present a model of path integration displayed by desert ants (Cataglyphis fortis). Contrary to previous approaches, they make no a priori assumption about the neural encoding of path information, and instead use ER to build a complete brain–body–environment model. The results show adherence with observations of the ants’ behavior, such as systematic navigation errors and local search of the target destination.
The minimally cognitive behavior approach is not the only way of exploiting ER as a modeling tool. The relevance of ER models is discussed by Seth (2007), who proposes a model of action selection grounded on sensorimotor processes, and shows that apparently irrational behavior is the result of action selection mechanisms evolved in response to the peculiar ecological conditions. The importance of ER models of adaptive behavior is revealed also by an attentive analysis beyond qualitative observation. For instance, Matsuda et al. (2014) explore deliberative decision-making in an ER model, showing that robust deliberation is linked to the coupled effects of continuous learning abilities and rich sensorimotor experiences. Finally, Sellers and colleagues exploited the strengths of ER to model plausible gaits of extinct species, from the Australopithecus afarensis (Sellers et al., 2005) to sauropod dinosaurs (Sellers et al., 2013). Starting from fossil traces, a 3D musculoskeletal simulation is built and its parameters optimized to maximize speed and/or minimize energy costs, allowing Sellers et al. to test hypothesis about the plausibility of different gaits and the existence of mechanisms to support the generation and accumulation of forces.
The selected studies presented in this paper demonstrate that ER has a strong potential both for the design of complex robotic systems and for the study of biological ones. However, this potential can be easily dissipated without a clear commitment to produce high-quality studies that focus on the relevant aspects of either modeling or design. To maximize the potential of ER, it is useful to recall a distinction proposed by Mitri et al. (2012) about the usage of (evolutionary) robotics to model biological systems, which may be either exploratory or hypothesis-driven. In the former, experiments are conducted to understand the possibilities offered by robotics and ER as a modeling tool, with the goal of providing existence proofs and producing novel testable hypotheses (Harvey et al., 2005). In the latter, the experimental effort is dedicated toward testing specific hypotheses, and the whole experimental design is conceived for this purpose. I maintain that a similar dichotomy should be established also for the design approach, in which exploratory studies investigate the potential for evolutionary methods to provide solutions to a specific control problem, while hypothesis-driven studies are focused toward methodological aspects as well as the demonstration of the advantages and disadvantages of ER with respect to other approaches. In my opinion, exploratory studies are the most susceptible to find themselves in the limbo between modeling and design: they suffer a high risk of adding very little to the current knowledge and are likely to pass unnoticed by either community. Exploratory studies should therefore be restricted to the extension of ER into novel territories [e.g., automatic design of robots with soft/unconstrained bodies, see Hiller and Lipson (2012)]. It is under these conditions that exploration can prove beneficial, as it can allow us to unlock the potential of both the design methodology and the target domain. Otherwise, hypothesis-driven focused studies should be preferred.
From a design perspective, ER would benefit from further studies tailored to characterizing the influence of certain design choices on the expected performance. It is necessary to isolate the different aspects that need to be defined and study their effect to evolutionary efficiency [e.g., as done with the effects of genetic relatedness and selection level on task performance by Waibel et al. (2009)]. This would better delineate the features of ER as a design tool, and would produce guidelines, which benefit developers who wish to exploit ER for their applications. Novel algorithmic solutions and design methods should also be proposed, and systematically contrasted with the state of the art, possibly in the context of a well-conceived benchmarking exercise (Clune et al., 2011; Mouret and Doncieux, 2012; Trianni and López-Ibáñez, 2014). Benchmarking would also be useful with respect to other control approaches, in order to identify the benefits and drawbacks of ER against the methodologies developed in other domains. Given the wide-ranging scope of ER studies, multiple benchmarking exercises could be proposed for different applications, and the proposed techniques tested against one or more test suites. Finally, hybridization of ER with other control approaches could prove beneficial, above all to provide solutions to application-specific problems.
For what concerns modeling, ER is especially fruitful when it is impossible or unpractical to run experiments directly with the biological system, either as laboratory or field work. In particular, the evolution of adaptive traits and cognitive responses is tightly linked to ecological and social conditions. These conditions are extremely difficult or impossible to be controlled and replicated with empirical studies, while they can be completely managed in evolutionary simulations (Adami, 2006). Therefore, ER can prove highly beneficial for testing general hypotheses about behavioral mechanisms and evolutionary dynamics. A tenet of ER is the relevance of situatedness and embodiment in (the evolution of) behavior and cognition. Therefore, experimental studies should refrain of tackling issues where situatedness and embodiment have a minor impact. Reducing the complexity of the evolving agents allows us to focus only on the relevant issues [e.g., minimally cognitive behaviors à la Beer (2003)]. In this respect, it is not always necessary to refer to a real robotic system, but simulated agents can be sufficient as long as situatedness and embodiment are properly accounted for [e.g., the predator–prey interaction studied by Olson et al. (2013)]. Real robotic systems instead are mostly useful when the physical body and the way in which the world is perceived through physical sensors may have a bearing on the phenomena under study [see a similar discussion by Mitri et al. (2012)].
Whether the animat approach is suitable for modeling biological phenomena is a highly debated issue [see the target article by Webb (2009) and the responses from several authors in the same journal issue]. In the ER context, considerable caution is needed given that artificial evolution is a very simplified model of natural evolution. In my opinion, however, it is wrong to a priori proscribe ER as a modeling tool, but it is necessary to evaluate case by case whether the proposed ER model can be of some value. This will be facilitated by a stronger interaction with (evolutionary) biology, through a better knowledge of the related literature as well as through the use of proper language and suitable analytical tools. Indeed, by studying artificial systems with the tools normally employed for biological ones, it may be easier to identify differences and similarities, and in parallel attempt to determine those causal relationships that escape conventional research in biology [e.g., the relationship between genetic relatedness and signal reliability studied by Mitri et al. (2011)]. In behavioral studies, the experimental setup should ideally be replicable also with biological systems (Froese and Di Paolo, 2010), to attempt a generalization of the identified results and possibly a direct comparison (Morlino et al., 2014).
To conclude, I believe that there is much room for exciting research in ER, be it for modeling or design purposes. To maximize the impact of future studies over ER and other disciplines, ER practitioners are advised to frame their studies in a hypothesis-driven way, which appears to me the best approach for a successful research program.
Conflict of Interest Statement
The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Funding: Vito Trianni gratefully acknowledges the financial support provided by the Italian National Research Council (CNR) through the EUROCORES-EuroBioSAS Program of the European Science Foundation, within the context of the H2Swarm project.
Baldassarre, G., Trianni, V., Bonani, M., Mondada, F., Dorigo, M., and Nolfi, S. (2007). Self-organized coordinated motion in groups of physically connected robots. IEEE Trans. Syst. Man Cybern. B Cybern. 37, 224–239. doi:10.1109/TSMCB.2006.881299
Bredeche, N., Montanier, J.-M., Liu, W., and Winfield, A. (2012). Environment-driven distributed evolutionary adaptation in a population of autonomous robotic agents. Math. Comput. Model. Dyn. Syst. 18, 101–129. doi:10.1080/13873954.2011.601425
Clune, J., Stanley, K., Pennock, R., and Ofria, C. (2011). On the performance of indirect encoding across the continuum of regularity. IEEE Trans. Evol. Comput. 15, 346–367. doi:10.1109/TEVC.2010.2104157
Fehérvári, I., Trianni, V., and Elmenreich, W. (2013). “On the effects of the robot configuration on evolving coordinated motion behaviors,” in Proceedings of the 2013 IEEE Congress on Evolutionary Computation (CEC) (Cancun: IEEE), 1209–1216. Available from: http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6557703
Francesca, G., Brambilla, M., Brutschy, A., Trianni, V., and Birattari, M. (2014). Automode: a novel approach to the automatic design of control software for robot swarms. Swarm Intell. 8, 89–112. doi:10.1007/s11721-014-0092-4
Froese, T., and Di Paolo, E. (2010). Modelling social interaction as perceptual crossing: an investigation into the dynamics of the interaction process. Conn. Sci. 22, 43–68. doi:10.1080/09540090903197928
Iizuka, H., Ando, H., and Maeda, T. (2013). Extended homeostatic adaptation model with metabolic causation in plasticity mechanism–toward constructing a dynamic neural network model for mental imagery. Adapt. Behav. 21, 263–273. doi:10.1177/1059712313488426
Matsuda, E., Hubert, J., and Ikegami, T. (2014). A robotic approach to understanding the role and the mechanism of vicarious trial-and-error in a t-maze task. PLoS ONE 9:e102708. doi:10.1371/journal.pone.0102708
Olson, R., Hintze, A., Dyer, F., Knoester, D., and Adami, C. (2013). Predator confusion is sufficient to evolve swarming behaviour. J. R. Soc. Interface 10, 20130305–20130305. doi:10.1098/rsif.2013.0305
Sellers, W., Cain, G., Wang, W., and Crompton, R. (2005). Stride lengths, speed and energy costs in walking of Australopithecus afarensis: using evolutionary robotics to predict locomotion of early human ancestors. J. R. Soc. Interface 2, 431–441. doi:10.1098/rsif.2005.0060
Tuci, E., Ferrauto, T., Zeschel, A., Massera, G., and Nolfi, S. (2011). An experiment on behavior generalization and the emergence of linguistic compositionality in evolving robots. IEEE Trans. Auton. Ment. Dev. 3, 176–189. doi:10.1109/TAMD.2011.2114659
Wischmann, S., Floreano, D., and Keller, L. (2012). Historical contingency affects signaling strategies and competitive abilities in evolving populations of simulated robots. Proc. Natl. Acad. Sci. U. S. A. 109, 864–868. doi:10.1073/pnas.1104267109
Keywords: evolutionary robotics, design methodology, modeling, complex systems, artificial life
Citation: Trianni V (2014) Evolutionary robotics: model or design? Front. Robot. AI 1:13. doi: 10.3389/frobt.2014.00013
Received: 04 August 2014; Accepted: 14 November 2014;
Published online: 03 December 2014.
Edited by:Alan Frank Thomas Winfield, University of the West of England, UK
Reviewed by:Nicolas Bredeche, Université Pierre et Marie Curie, France
Takashi Ikegami, The University of Tokyo, Japan
Alan Frank Thomas Winfield, University of the West of England, UK
Copyright: © 2014 Trianni. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Vito Trianni, Istituto di Scienze e Tecnologie della Cognizione, Consiglio Nazionale delle Ricerche, Via San Martino della Battaglia 44, Rome 00185, Italy e-mail: email@example.com