The Past, Present, and Future of Artificial Life

For millennia people have wondered what makes the living different from the non-living. Beginning in the mid-1980s, artiﬁcial life has studied living systems using a synthetic approach: build life in order to understand it better, be it by means of software, hardware, or wetware.This review provides a summary of the advances that led to the development of artiﬁcial life, its current research topics, and open problems and opportunities. We classify artiﬁcial life research into 14 themes: origins of life, autonomy, self-organization, adaptation (including evolution, development, and learning), ecology, artiﬁcial societies, behavior, computational biology, artiﬁcial chemistries, information, living technology, art, and philosophy. Being interdisciplinary, artiﬁcial life seems to be losing its boundaries and merging with other ﬁelds.


THE PAST
Google's Ngram Viewer (Michel et al., 2011) allows users to search the relative frequency of n-grams (short-words combinations, n ≤ 5) in time, exploiting the large database of Google Books that includes about 4% of all books ever written. Hiroki Sayama did a search for"artificial life" 1 , and the curve showed how the frequency jumps from 1986 and reaches a peak in 1997 before stabilizing. However, there is an even higher peak around 1821. "What were they doing in those days?" Hiroki tweeted. Well, Frankenstein, or The Modern Prometheus by Mary Shelley was published in 1818. That created a wave in literature until the end of the 1820s and had an impact for the rest of the nineteenth century, as people debated on the nature of life in view of the impressive technological and scientific advances of the age. What are the causes and conditions of life? Can we make living creatures?
We know that such questions were asked from the dawn of history. Consider, for instance, the artificial creatures found in the Greek, Mayan, Chinese, and Jewish mythologies, where human beings acquire the divine ability to make living creatures through magic. Other examples can be found during the middle ages, such as the automata created by al-Jazari (including the first programmable humanoid robot) and the legendary Albertus Magnus' brazen head (an automaton reputed to be able to answer any question) and its mechanical servant (which advanced to the door when anyone knocked and then opened it and saluted the visitor). Later on, during the Italian Renaissance, several automata were designed (Mazlish, 1995). Leonardo da Vinci's mechanical knight (a humanoid that could stand, sit, raise its visor and independently maneuver its arms) and its mechanical lion (which could walk forward and open its chest to reveal a cluster of lilies) are just two examples of this kind of automata. There is also a legend that 1 http://t.co/boMAxmjQ2c says that Juanelo Turriano created an automata called "The Stick Man." It begged in the streets, and when someone gave him a coin, he bowed. Through the modern age, automata became more and more sophisticated, based on and leading to advances in clockwork and engineering (Wood, 2002). Perhaps the most impressive of this period were the automata of Vaucanson. His first workshop was destroyed because the androids he wanted to build were considered profane. He later built a duck, which appeared to eat, drink, digest, and defecate. Other examples of modern automata are those created by Pierre Jaquet-Droz: the writer (made of 2500 pieces), the musician (made of 2500 pieces), and the draughtsman (made of 2000 pieces).
Questions related to the nature and purpose of life have been central to philosophy, and the quest of creating life has been present for centuries (Ball, 2011). Being able to imitate life with automata, can we understand better what makes the living alive? Hobbes (1651, p. 1) begins his Leviathan with: Nature (the art whereby God hath made and governs the world) is by the art of man, as in many other things, so in this also imitated that it can make an artificial animal. For seeing life is but a motion of limbs, the beginning whereof is in some principal part within, why may we not say that all automata (engines that move themselves by springs and wheels as doth a watch) have an artificial life? [our emphasis] Descartes also considered the living as being mechanical: life being similar to a clockwork (Descartes, 1677). Still, Descartes did not consider the soul to be mechanical, leading to dualism.
Nevertheless, in spite of these many antecedents, it is commonly accepted [see, for example, Bedau (2003)] that it was not until 1951 that the first formal artificial life (ALife) model was created, when von Neumann (1951) was trying to understand the fundamental properties of living systems. In particular, he was interested in www.frontiersin.org self-replication, a fundamental feature of life. Collaborating with Stanislaw Ulam at Los Alamos National Laboratory, von Neumann defined the concept of cellular automata and proposed a self-replicating formal system, which was aimed at being computationally universal (Turing, 1936) and capable of open-ended evolution (von Neumann, 1966;Mange et al., 2004). Simpler alternatives to von Neumann's "universal constructor" were later proposed by Codd (Hutton, 2010) and Banks (1971). Langton (1984) then proposed simpler self-replicating "loops," based on Codd's ideas but without universality 2 . Popularization and further development of cellular automata continued in the 1970s and 1980s, the best known examples being Conway's Game of Life (Berlekamp et al., 1982), and Wolfram's elementary cellular automata (Wolfram, 1983). A contemporary of von Neumann, Barricelli (1963) developed computational models similar to cellular automata, although focusing on evolution.
In parallel to these studies by von Neumann and others, cybernetics studied control and communication in systems (Wiener, 1948;Gershenson et al., 2014). Cybernetics and systems research described phenomena in terms of their function rather than their substrate, so similar principles were applied to animals and machines alike. Langton (1984) suggested that life should be studied as property of form, not matter. This resonates with the cybernetic approach, so it can be said that ALife has strong roots in cybernetics. Moreover, central concepts such as homeostasis (Ashby, 1947a(Ashby, , 1960Williams, 2006) and autopoiesis (Varela et al., 1974;Maturana and Varela, 1980) were developed within and inspired by cybernetics (Froese and Stewart, 2010). A couple of examples, Walter (1950Walter ( , 1951 built robotic "tortoises" (Holland, 1997), which can be classified as early examples of adaptive robotics. In the 1960s, Beer (1966) developed a model for organizations based on the principles of living systems. Beer's ideas were implemented in Chile during the Cybersyn project (Miller Medina, 2005) in the early 1970s.
It is clear that life does not depend only on its substrate. Take, for example, Kauffman's blender thought experiment (Kauffman, 2000): imagine you take the biosphere, place it in a giant blender, and press MAX. For some time, you would have the same molecular diversity. However, without its organization, the complex molecules of the biosphere would soon decay and their diversity would be lost. Living systems organize flows of matter, energy, and information to sustain themselves. Life cannot be studied without considering this organization, as one cannot distinguish molecules, which are part of a living organization from those that are not. There have been several advances, but there is still much to discover about the realm of the living.
ALife has been closely related to artificial intelligence (AI), since some of their subjects overlap. As Bedau (2003, p. 597) stated: "living and flourishing in a changing and uncertain environment requires at least rudimentary intelligence." However, the former is particularly focused on systems, which can mimic nature and its laws and therefore it is more related to biology, while the latter is mainly focused on how human intelligence can be replicated, and therefore, it is more related to psychology. Moreover, they differ in their modeling strategies. On the one hand, most traditional AI models are top-down specific systems involving a complicated, centralized controller that makes decisions based on access to all aspects of global state. On the other hand, ALife systems are typically bottom-up (Maes, 1993), implemented as low-level agents that simultaneously interact with each other, and whose decisions are based on information about, and directly affect, only their own local environment (Bedau, 2003).
The research around these topics continued until 1987, the year in which Langton organized the first Workshop on the Synthesis and Simulation of Living Systems in Santa Fe, New Mexico, where the term "artificial life" was coined in its current usage. The event marked the official birth of the field. Incidentally, the scientific study of complex systems (Gershenson, 2008) also initiated roughly at the same time in the same place, the Santa Fe Institute. Figure 1 summarizes the "prehistory" of ALife, which begins with the ancient myths and stories and finishes with the formal creation of this area of research.

WHAT IS ARTIFICIAL LIFE?
The concept of artificial life can take different meanings. In its current usage, the term artificial life (ALife) was coined in the late 1980s by Langton (1989), who originally defined it as "life made by man rather than by nature," i.e., it is the study of man-made systems that exhibit behaviors characteristic of natural living systems. However, with time, Langton found fundamental problems with this definition, and redefined it as "the study of natural life, where nature is understood to include rather than to exclude, human beings and their artifacts" (Langton, 1998). He stated that human beings, and all that they do, are part of nature, and as such, a major goal of ALife should be to work toward removing "artificial life" as a phrase that differs in meaning in any fundamental way from the term "biology." Indeed, it is now quite common for biologists to use computational models, which would have been considered as ALife 20 years ago, but now they are part of mainstream biology (Bourne et al., 2005). Bedau (2007) defined contemporary artificial life as an interdisciplinary study of life and life-like processes, whose two most important qualities are that it focuses on the essential rather than the contingent features of living systems and that it attempts to understand living systems by artificially synthesizing simple forms of them. Three broad and intertwining branches of artificial life correspond to three different synthetic methods. "Soft" artificial life creates simulations or other purely digital constructions that exhibit life-like behavior (most ALife research is soft), "hard" artificial life produces hardware implementations of life-like systems, and "wet" artificial life synthesizes living systems from biochemical substances (Rasmussen et al., 2003(Rasmussen et al., , 2008. In this way, ALife attempts to synthesize properties of living systems in computers, machines, and molecules. Thus, ALife aims to understand biological life better by creating systems with life-like properties and developing novel forms of life. In a broad sense, artificial life can be understood as the synthesis and simulation of living systems, which actually has been the name of the international workshops and conferences organized since 1987. ALife has been an interdisciplinary research field (Langton, 1997;Adami, 1998;Dorin, 2014), bringing together biologists, philosophers, physicists, computer scientists, chemists, mathematicians, artists, engineers, and more. It has also been related to several fields, having a strong overlap with some of them, such as complexity (Bar-Yam, 1997;Mitchell, 2009), natural computing (de Castro, 2006, evolutionary computation (Baeck et al., 1997;Coello Coello et al., 2007), language evolution (Cangelosi and Parisi, 2002;Christiansen and Kirby, 2003), theoretical biology (Waddington, 1968a), evolutionary biology (Maynard Smith and Szathmáry, 1995), philosophy (Boden, 1996), cognitive science (Clark, 1997;Bedau, 2003;Couzin, 2009), robotics (Mataric andCliff, 1996), artificial intelligence (AI) (Steels and Brooks, 1995) 3 , behavior-based systems (Maes, 1993;Webb, 2000), game theory (Sigmund, 1993), biomimesis (Meyer, 1997;Carmena et al., 2001), network theory (Newman, 2003;Newman et al., 2006), and synthetic biology (Benner and Sismour, 2005), among others.

Frontiers in Robotics and AI | Computational Intelligence
Current ALife research can be classified into the 14 themes summarized in the rest of this section: origins of life, autonomy, self-organization, adaptation (evolution, development, and learning), ecology, artificial societies, behavior, computational biology, artificial chemistries, information, living technology, art, and philosophy. Figure 2 shows the number of papers published in the Artificial Life journal related to each of these themes since 1993. The first four themes focus more on properties of living systems. The next five themes study life at different scales. The last four are related to our understanding, uses, and descriptions we have of the living. This categorization is somewhat arbitrary, as several of the themes are entwined and overlapping. This also causes some of the topics to appear underrepresented, as related work has been mentioned in other subsections.

ORIGINS OF LIFE
ALife has had a close relationship with the community of scientists working on the origins of life. Similar to the subdivision of ALife into two rather distinct areas focused on either individual autonomy or population evolution, there have been two major theories about the origin of life, known as the metabolism-first and replicator-first approaches (Dyson, 1985;Pross, 2004). The former typically views the origin of life as related to the emergence of selfproducing and self-maintaining far-from-equilibrium structures, for example, based on the principles of autopoiesis (Ono et al., 2008), autocatalytic networks (Kauffman, 1986), and reactiondiffusion systems (Froese et al., 2012a). The latter approach, which has received more attention in mainstream science (Joyce, 2002), prefers to identify the origin of life with the beginning of evolution by natural selection (Tessera, 2009). Its classic formulation is the "RNA world" hypothesis (Gilbert, 1986), which has been generalized to the idea of natural selection in chemical evolution (Fernando and Rowe, 2007). In recent work, these two approaches can no longer be clearly distinguished as both autonomy and www.frontiersin.org FIGURE 2 | Popularity of different themes per year, as measured by papers published in the Artificial Life journal. Adaptation has been a dominant theme in the journal, as it includes evolution, development, and learning. Self-organization has not been that popular, but is a constant topic. Some themes are poorly represented, such as art, because artists usually choose different venues to publicize their work. Other themes have had peaks of popularity for different reasons, such as special issues.
evolution are thought to be necessary for life (Ruiz-Mirazo and Moreno, 2004). Metabolism-first approaches have accepted the necessity of an informational capacity to enable open-ended evolution, even if it is in terms of a prebiotic "composite" genome (Segré et al., 2000). Replicator-first approaches, on the other hand, had to make recourse to membrane boundaries and metabolic activity, for example, to give rise to individuated protocells capable of competition (Chen et al., 2004). More recently, a new debate has arisen about the role of movement and adaptive behavior in the origin of life (Hanczyc, 2011;Egbert et al., 2012;Froese et al., 2014), a topic that had long been ignored by both metabolismand replicator-first approaches. Indeed, one of the major open challenges in this area is to better understand the engineering of second-order emergence (Froese and Ziemke, 2009), that is, how to synthesize the underlying conditions for the emergence of an individual that, in interaction with its environment, gives rise to interesting behavior. Here, we therefore find the flipside of the problem faced by evolutionary robotics (see below); while models of the origin of life must somehow make its systems more interactive, robotics has to somehow make their systems more autonomous. It is likely that attempts at integrating biological autonomy, adaptive behavior, and evolution into one model will continue to improve, which would at the same time mean an integration of the various subfields of ALife. This integration of life and Frontiers in Robotics and AI | Computational Intelligence mind on various timescales is also supported by ongoing developments in the philosophy of mind and cognitive science, which is increasingly realizing the many ways in which mind is inseparable from a living body (Thompson, 2007).
A key question related to the studies of the origin of life is the definition of life itself (Schrödinger, 1944;Haldane, 1949;Margulis and Sagan, 1995;Bedau, 2008;Lazcano, 2008), to be able to determine when it began. Some argue that one of the defining properties of living systems is autonomy.

AUTONOMY
Since its beginnings, the field of ALife has always been closely associated with the concepts of biological autonomy and autopoiesis (Bourgine and Varela, 1992). The term "autopoiesis" was coined by the biologists Maturana and Varela (1980) to characterize a bounded network of processes that self-maintains its organization such that it is identifiable as a unity in the chemical domain. They created a computer model that can be considered as one of the first examples of ALife (Varela et al., 1974), and which has given rise to a tradition of computational autopoiesis in the field (McMullin, 2004). The precise definition of autopoiesis continues to be debated, and even Maturana and Varela were not always in agreement with each other (Froese and Stewart, 2010). Although the core idea seems to be that living beings are not only selforganizing but are also self-producing, they owe their existence as individual material entities to their ongoing internal (metabolic) and relational (regulatory) activities. This idea is sometimes formalized as operational closure, which can be defined as a network of processes in which each process enables, and is enabled by, at least one other process in that network. Varela (1979) used this concept to abstract autopoiesis from the specificities of the chemical domain so as to derive a concept of autonomy in general. In this way, Varela was able to describe other biological systems, such as the nervous system and the immune system, as being autonomous, even if they did not chemically self-produce. Relatedly, this concept of autonomy has been used to describe the self-sustaining dynamics of social interaction (De Jaegher and Froese, 2009). However, there is a concern that this abstraction makes us overlook what is essential to life itself, which has prompted some researchers to develop a more concrete theory of biological autonomy. For example, Ruiz-Mirazo and Moreno (2004) propose that "basic autonomy" is the capacity of a system to manage the flow of matter and energy through it so that it can regulate internal self-constructive and interactive exchange processes under far-from-equilibrium thermodynamic conditions. This conception of autonomy, as referring to processes of selfproduction, must be distinguished from the term's common use in robotics, where it is employed more loosely as the capacity of a system to move and interact without depending on remote control by an operator (Froese et al., 2007). Nevertheless, it is the strong sense of autonomy that allows us to talk about a system as being an individual that acts in relation to its intrinsic goals, i.e., of being a genuine agent (Barandiaran et al., 2009), rather than being a system whose functions are heteronomously defined from the outside. This has implications for how we should think about the so-called "ALife route to artificial intelligence" (Steels, 1993;Steels and Brooks, 1995). An important first step along this route was the development of behavior-based robotics, rather than micromanaging all aspects of a system's behavior, as was common practice in good old-fashioned AI and still is in industrial robotics, behavior (see below) began to be seen as an emergent property of the robot-environment as a whole (Brooks, 1991).
Living systems need a certain degree of autonomy. This implies that they have certain control over their own production. This can be achieved through the process of self-organization.
Self-replication can be seen as a special case of self-organization, as a replicator has to conserve and duplicate its organization by itself. Examples from von Neumann to Langton have been already mentioned, although there have been several more (Sipper, 1998).
Another special case of self-organization is self-maintenance, which is related to homeostasis (Ashby, 1947a(Ashby, , 1960Williams, 2006) and has been studied in relation to artificial chemistries Ikegami, 1999, 2001) (see below).
Some of these robots have taken inspiration from insect swarms. Their self-organization has served as an inspiration in computational intelligence (Bonabeau et al., 1999;Prokopenko, 2014a). More recently, these studies have been extended toward cognitive science (Trianni and Tuci, 2009;Gershenson, 2010). This kind of research is also related to collective intelligence (Hutchins, 1995) and the evolution of language (Steels, 2003).

ADAPTATION
Adaptation can be defined as "a change in an agent or system as a response to a state of its environment that will help the agent or system to fulfill its goals" (Gershenson, 2007). Adaptation is a central feature of living systems and is essential for autonomy and survival. One of the major criticisms of AI has been its lack of adaptability, as it traditionally attempted to predict and control rather than to adapt (Gershenson, 2013a), while part of ALife has focused on bringing adaptability to AI (Maes, 1993;Steels and Brooks, 1995). Still, both adaptability and predictability are desirable properties in natural and artificial systems.

www.frontiersin.org
Adaptation can occur at different time scales (Jablonka and Lamb, 2006;Gershenson, 2010). At a slow scale (several lifetimes), adaptation is called evolution. At a medium scale (one lifetime), adaptation is called development (including morphogenesis and cognitive development). At a fast scale (a fraction of a lifetime), adaptation is called learning. Adaptation at one or more scales has been a central topic in ALife, as shown by Figure 2.

Evolution
Computer science has exploited artificial evolution extensively, initially with genetic algorithms (Holland, 1975;Mitchell et al., 1992;Mitchell and Forrest, 1993) 4 , which were generalized in the field of evolutionary computation (Baeck et al., 1997;Coello Coello et al., 2007), an important part of computational intelligence (Prokopenko, 2014a). The main purpose of using evolutionary algorithms is to search suitable solutions in problem spaces that are difficult to explore with more traditional heuristic methods.
ALife systems such as Tierra (Ray, 1993) and Avida (Ofria, 1999;Ofria and Wilke, 2004) have been used to study the evolution of "digital organisms," using a formal framework, which has brought fruitful advances in the understanding features of living systems such as robustness (Lenski et al., 1999), the evolution of complexity , the effect of high-mutation rates (Wilke et al., 2001), the evolution of complex organisms (Lenski et al., 2003), mass extinctions (Yedid et al., 2012), and ecological networks (Fortuna et al., 2013).
In hard ALife, evolution has been used also for further removing the influence of the designer with the development of evolutionary robotics (Cliff et al., 1993;Eiben, 2014), e.g., the use of evolutionary algorithms in the automated design of a robot's cognitive architecture, which could simply be initialized as a generic dynamical system (Beer, 1995). This approach continues to be a popular tool for the ALife community (Nolfi and Floreano, 2000;Harvey et al., 2005;Vargas et al., 2014), but it has become evident that replacing the human designer by artificial evolution does not spontaneously lead to the emergence of agents in the strong sense discussed above (Froese and Ziemke, 2009). One response has been to apply insights from organisms to better design the internal organization of artificial agents such that they can spontaneously re-organize, for example, by incorporating some capacity for homeostatic adaptation and habit formation (Di Paolo, 2003). Initial attempts followed Ashby (1960) proposal of ultrastability, but the problem of heteronomous design quickly resurfaced. It is still an important open challenge to enable more profound forms of internal adaptation in these agents without pre-specifying the underlying mechanisms and/or their goals (Iizuka et al., 2013;Izquierdo et al., 2013;Egbert and Cañamero, 2014).

Development
Artificial development is, on one hand, inspired by the developmental processes and cellular growth seen in nature (biological development), and on the other hand, it is interested in studying developmental processes related to cognition (cognitive development). Chavoya (2009) defined "biological artificial development" as the study of computer models of cellular growth, with the objective of understanding how complex structures and forms can emerge from a small group of undifferentiated initial cells. These systems have been traditionally divided into two groups: (1) those that are based on self-organizing chemical processes in and between cells, and (2) those that follow a grammatical approach. Turing (1952) seminal paper on the chemical basis of morphogenesis is probably the earliest work belonging to the first group. In that paper, Turing used a set of differential equations to propose a reaction-diffusion model, which led him to suggest that an initially homogeneous medium might develop a structured pattern (such as certain radial and dappling patterns observed in the skin of many animals) due to an instability of the homogeneous equilibrium, triggered by small random disturbances. Later on, Gierer and Meinhardt (1972) presented a model similar to Turing's. They proposed that pattern formation was the result of local self-activation coupled with lateral inhibition. The most famous result of their theory is the simulation of seashell patterns (Meinhardt, 2003). Regarding those systems that follow a grammatical approach, Lindenmayer (1971) proposed the socalled L-Systems, which are a formal grammar with a set of symbols and a set of rewriting rules. They were introduced as a mathematical formalism for modeling development of simple multicellular organisms. These systems were applied to modeling the development of plants and trees (Prusinkiewicz et al., 1990). In 1996, Dawkins (1996) introduced his famous work on "biomorphs" to illustrate how evolution might induce the creation of complex designs by means of micro-mutations and cumulative selection. His results include biomorphs that resemble tree-like structures, insects, crustaceans, and mammals. More recently, Stanley (2007) proposed a novel abstraction of natural development, called "compositional pattern producing networks" (CPPNs). This model allowed him to demonstrate the existence of intrinsic properties found in natural development, such as bilateral symmetry and repeating patterns with and without variation. There has also been interest in creating software platforms as tools for experimenting with simulated developmental processes. For example, Stewart et al. (2005) created the METAMorph open source software, which allows researchers to manually design genetic regulatory networks and visualize the resulting morphological growth process.
The artificial life community has also been interested in creating computational models of cognitive development. Mareschal and Thomas (2006) defined them as formal systems that track the changes in information processing taking place as a behavior is acquired. Several approaches have been taken to tackle this problem, such as neural networks [e.g., Shultz et al. (1995); Parisi and Schlesinger (2002)], dynamical systems theory [e.g., Thelen and Smith (1996)], cognitive architectures [e.g., Anderson (1993); Simon (1998); Jones et al. (2000)], and Bayesian networks [e.g., Xu and Tenenbaum (2000); Schlesinger and Parisi (2001)]. For recent reviews of this topic, see Elman (2005) and Schlesinger and McMurray (2012).

Learning
Learning is a fundamental aspect of adaptive behavior for living organisms. Although there is no agreed definition, it can be conceived as a change in an organism's capacities or behavior brought about by experience (Wilson and Keil, 1999). In the context of artificial life, several approaches have been taken to model learning, some of which have influenced the field of machine learning (Bishop, 2006).
Artificial neural networks (Rojas, 1996;Neocleous and Schizas, 2002) are a well known approach to learning, which are inspired by the structure and functional aspects of biological neural networks.
There have been several other ALife approaches to learning in conjunction with other themes, e.g., behavior or evolution (Izquierdo et al., 2008).

ECOLOGY
At a high level of abstraction, ecological studies in ALife can be described as interactions between individuals from different species and with their environment.
Coevolution involves species interaction across generations, having strong relations with ecology. Sims's creatures (Sims, 1994) are one example of coevolution. These creatures compete for a resource and evolve interesting morphologies and behaviors. A relevant topic within coevolution is the "red queen effect" (Dave and Miller, 1995) where species evolution affects the fitness of other species, leading to "arm races" (Nolfi and Floreano, 1998), which can promote the evolution of complex traits.
At a global level, the living properties of biospheres have been studied. Perhaps the best known example is Daisyworld (Watson and Lovelock, 1983;Lenton and Lovelock, 2000). ALife models can study how regulation can occur as a consequence of multiple ecological interactions (McDonald-Gibson et al., 2008).
ALife ecological models, including cellular automata and agentbased (Grimm et al., 2005), have been used already in ecology for applications such as resource management (Bousquet and Page, 2004) and land-use models (Matthews et al., 2007), where models have to include also the social dimension.

ARTIFICIAL SOCIETIES
Societies are defined by the interactions of individuals of the same species. The computational modeling of social systems has become very popular because it enables the systematic exploration of possibilities of social interaction, which are very difficult to achieve with complex societies (Gilbert and Conte, 1995;Epstein Axtell, 1996;Gershenson, 2001;Epstein, 2006).
For example, the evolution of cooperation has been a popular research topic (Burtsev and Turchin, 2006). Mainly based on game theory (Nowak, 2006), one of the most studied problems of cooperation is the prisoner's dilemma (Santos et al., 2006(Santos et al., , 2008. This approach has also been used to study multilevel selection (Traulsen and Nowak, 2006;Powers et al., 2011).
Central to human societies, the evolution of language and communication has been widely studied, beginning within the ALife community (Cangelosi and Parisi, 2002;Kirby, 2002;Steels, 2012). The evolution of language can be seen as a special case of semiotics, i.e., the problem of how meaning is acquired, which is also studied within ALife (Emmeche, 1991;Rocha, 1998;Ziemke and Sharkey, 2001) and closely related with philosophy (Gershenson, 2002).
Language is also a part of culture, which is beginning to be modeled within computational anthropology (Axtell et al., 2002).
The modeling of societies has led to the development of popular ALife games, such as Creatures (Grand, 2001) and The Sims (Wikipedia, 2014).

BEHAVIOR
Some of the differences between artificial intelligence and artificial life can be seen in their contrasting views of and approaches to synthesizing behavior. Put in somewhat simplified terms, AI reduces behavior to something that is specified to take place inside an agent independently and on its own terms. This internal processing is often implemented in terms of a sense-model-plan-act architecture, which means that the agents behavior has more to do with logical inferences based on internal representations rather than with interacting with the world in real time. This traditional view was widely criticized from scientific, engineering, and philosophical perspectives. These have agreed that the structure of behavior is primarily to be conceived, designed, and analyzed in terms of the dynamics of a closed sensorimotor loop (Braitenberg, 1986;Brooks, 1991;Cliff, 1991;Dreyfus, 1992;Clark, 1997;Pfeifer and Scheier, 1999;Pfeifer et al., 2007a). This has led to the study of adaptive behavior, mainly based on ethology (Maes, 1993;Meyer, 1997). This widespread paradigm shift made it evident that the contributions of the body and of the environment cannot be ignored, which is why this research is often referred to as embodied and situated (or embedded) cognition (Varela et al., 1991). Since the 1990s this paradigm has continued to grow in popularity (Wheeler, 2005;Chemero, 2009;Robbins and Aydede, 2009;Beer, 2014b), so much that the next step is to disentangle the many versions that have been proposed (Kiverstein and Clark, 2009). ALife has benefited from this paradigm shift because it has always preferred to study the conditions of emergence to pre-specified behavior, and because it has closely linked the notion of life with biological embodiment and its environment. As cognitive science is in the process of continuing its theoretical development from an embodied to a so-called "enactive" approach, which pays particular attention to the properties of the living such as autopoiesis, autonomy, and sense-making (Weber and Varela, 2002;Thompson, 2007;).Therefore, we can expect that ALife will take the place of AI as the most important synthetic discipline of cognitive science. It is ALife, not traditional AI, which has the tools in order to investigate the general principles of the biologically embodied mind (Di Paolo, 2003;Pfeifer et al., 2007b;Froese and Ziemke, 2009). At the same time, given the increasing interest in the science of consciousness, it is likely that these efforts www.frontiersin.org will be complemented by a growing emphasis on synthesizing and using new kinds of immersive and life-like human-computer interfaces to explore life-and mind-as-it-could-be from the first-person perspective (Froese et al., 2012b).

COMPUTATIONAL BIOLOGY
Theoretical biology (Waddington, 1968b) preceded ALife in the abstract study of living systems. In return, ALife has contributed to theoretical biology with the development of computational models and tools.
Computers have enabled the study of complex systems (i.e., having many non-linearly interacting components), in a similar way as microscopes enabled microbiology (Pagels, 1989). Systems biology (Kitano, 2002) has also required computers to study the complexity of biological systems at different scales, overlapping with ALife in several aspects. The transmission, storage, and manipulation of information at different scales are essential features of living systems, and several ALife models focus on one or more of these.
Cellular automata were already mentioned (Wolfram, 1986;Wuensche, 1992). Similar models have been used to study other aspects of biology. For example, Kauffman (1969) proposed random Boolean networks as models of genetic regulatory networks (Aldana-González et al., 2003;Gershenson, 2004). Studying ensembles of such networks, the functional effects of topologies, modularity, degeneracy, and other structural properties can be measured (Gershenson, 2012), providing insights into the nature of adaptability and robustness. These models of genetic regulatory networks have been useful for theoretical biology, as they have demonstrated the role of criticality in evolution (Balleza et al., 2008) and suggested that a possible evolutionary mechanism for obtaining this criticality (Torres-Sosa et al., 2012).
The study of biological neural networks led to the proposal of several models of distributed computation (Rojas, 1996). Some of these have been used in ALife for the evolution, development, or learning of artificial "brains" with different applications.
In a similar way, the computational study of immune systems (Bersini, 1992;Forrest et al., 1994) has led to developments in computer security and optimization (Burke et al., 2014).

ARTIFICIAL CHEMISTRIES
Artificial chemistries are used to study questions related to the origin of life from chemical components, as well as prebiotic and biochemical evolution (Dittrich et al., 2001). This is because chemical components are considered non-living, while they form living organisms. Perhaps the first computer simulation of the formation of a simple protocell consisting of a metabolic network and a boundary was that which introduced the concept of autopoiesis (Varela et al., 1974;Maturana and Varela, 1980;McMullin, 2004).
Artificial chemistries have been extended to include evolution (Hutton, 2002) and are closely related with self-organization (Sayama, 2008).

INFORMATION
It has been argued that living systems lie at the "edge of chaos" (Langton, 1990;Kauffman, 1993), i.e., they require a balance between stability/robustness and change/adaptability. How to find this balance? More generally, how are we to measure organization and self-organization? And adaptability, homeostasis, autonomy, or even autopoiesis? There have been several proposals, but still there is no agreement on how the properties of living systems should be measured.
A recent attempt has been to use information theory (Shannon, 1948;Prokopenko et al., 2009) to measure different properties of living systems. In this context, the field of guided self-organization is emerging (Prokopenko, 2009(Prokopenko, , 2014bAy et al., 2012;Polani et al., 2013), combining tools and concepts from information theory, self-organizing systems, and ALife.
For example, following Ashby's law of requisite variety (Ashby, 1956), autopoiesis can be seen as the ratio of the complexity of a system over the complexity of its environment (Fernández et al., 2014). This implies that a living system requires a higher complexity than its environment to have a certain degree of autonomy. This view shifts the definition of life from "all or nothing" to a continuous transition between the non-living and living.

LIVING TECHNOLOGY
There have been hundreds of papers published on applications of ALife (Kim and Cho, 2006). More recently, the term "living technology" has been used to describe technology that is based on the core features of living systems (Bedau et al., , 2013. Living technology is adaptive, robust, autonomous, and self-organizing. Living technology can be classified as primary and secondary [Bedau et al. (2009), p. 91]. Primary living technology is constructed from non-living components, while secondary living technology depends on living properties already present in its elements.
An example of primary living technology would be the design of protocells (Rasmussen et al., 2008) or artificial cells (Gibson et al., 2010) for applications such as cleaning pollution, generating energy, and improving health.
A broad area of application of secondary living technology lies within socio-technical systems (Helbing et al., 2012;Gershenson, 2013c). Governments, economies, and cities will be more efficient if they are "living," i.e., exhibiting some of the key properties of living systems, potentially bringing numerous benefits to society.
ALife has the capacity to improve technologies, but also technologies have contributed to ALife. For example, there has been substantial ALife research based on the Internet, which facilitates the study of, e.g., interactive evolution (Taylor, 2014), which has also led to some artistic applications, e.g., Picbreeder (Secretan et al., 2008).

ART
Within artificial intelligence, methods have been developed to model creativity (Boden, 1998). This has also been the case in ALife (Rinaldo, 1998;Whitelaw, 2004), where computational methods such as evolutionary computation have been used for creating artwork (McCormack and d'Inverno, 2012;Antunes et al., 2014), mainly within design, the visual arts, and music.

Frontiers in Robotics and AI | Computational Intelligence
There have been several exhibitions dedicated to ALife art, such as the Ars Electronica Festival 1993, with many artists producing works within this movement (Penny, 2010). The VIDA Art and Artificial Life International Awards (Tenhaaf, 2008) began in 1999 and has been active since, supporting and promoting ALife art.
The interaction between the scientific and artistic ALife communities has been marginal and could be enhanced. Still, they are far more interconnected than is sometimes the case between the sciences and the humanities.

PHILOSOPHY
Artificial life has dealt with several philosophical questions (Boden, 1996). An ontology is required to discuss what life is. Epistemology is needed for understanding living systems (Pattee, 1995), but also artificial creatures can have their own epistemology (Beer, 2014a). ALife has also contributed to the philosophical discussions related to the nature of emergence (Bedau and Humphreys, 2008). Furthermore, building living systems has ethical implications (Bedau and Parke, 2009).
In particular, one unresolved question in the philosophy of artificial life is the status of the modeled phenomena. In the case of wet ALife, the synthetic creation of a living system logically implies the creation of an actual life form. But what about simulation models of living systems? Some researchers argue that since life is a property of the systemic organization of a material phenomenon (such as autopoiesis), and not identical with the material phenomenon per se, we should also treat modeled life as real life. This position is known as "strong" ALife. Still, it could also be argued that even though life is expressed by a certain systemic organization, it nevertheless requires a concrete material realization in order to be considered real life. On this view, modeled life is just that -a model and not real life. An intuitive way to understand this position ("weak" ALife) is to consider what happens when we run a program that is simulating the molecular structure of water. Although the formal organization of the molecules in the model is the same as of real water, the computer running the simulation does not get wet! Therefore, it becomes understandable why many researchers do not assign to their modeling results the same status as empirical data, that is, data obtained from wet ALife or other physical experiments. Yet, due to the complexity of most models, running a computer simulation can provide us with new insights, some of which may, in fact, be unattainable without actually running the simulation. In other words, models are not just computerized versions of thought experiments, they are "opaque" thought experiments (Di Paolo et al., 2000). This interpretation also connects the field of ALife with a long tradition in continental philosophy of mind that is currently gaining popularity in cognitive science, i.e., phenomenology (Gallagher and Zahavi, 2008), which also relies on imaginative thought experiments (a method known as eidetic variation) to investigate the essential structure of life and mind (Froese and Gallagher, 2010). Of course, this more conservative and pragmatic interpretation of the status of ALife models will not convince those who see life as a purely abstract relational phenomenon, and therefore, realizable by digital computers. Fortunately, for most purposes of scientific investigation based on the use of artificial life tools, this still unresolved philosophical debate is somewhat tangential. No matter whether we treat our simulations as models or as actual realizations, the objective results we obtain from them remain the same.

THE FUTURE
How can systems be built with metabolism, heredity, and membranes at the same time? How can adaptation at multiple temporal and spatial scales be achieved? Is there an inherent limitation to computer simulations of open-ended evolution? How to integrate adaptivity and autonomy? How can ALife benefit society?
These and other questions have been asked within the ALife community. Bedau et al. (2000) distilled a list of 14 open problems: 1. Generate a molecular proto-organism in vitro. 2. Achieve the transition to life in an artificial chemistry in silico. 3. Determine whether fundamentally novel living organizations can exist. 4. Simulate a unicellular organism over its entire lifecycle. 5. Explain how rules and symbols are generated from physical dynamics in living systems. 6. Determine what is inevitable in the open-ended evolution of life. 7. Determine minimal conditions for evolutionary transitions from specific to generic response systems. 8. Create a formal framework for synthesizing dynamical hierarchies at all scales. 9. Determine the predictability of evolutionary consequences of manipulating organisms and ecosystems. 10. Develop a theory of information processing, information flow, and information generation for evolving systems. 11. Demonstrate the emergence of intelligence and mind in an artificial living system. 12. Evaluate the influence of machines on the next major evolutionary transition of life. 13. Provide a quantitative model of the interplay between cultural and biological evolution. 14. Establish ethical principles for artificial life.
There have been advances in all of these problems since 2000, but all of them remain open. As such, they continue to serve as guidelines for future ALife research.
A better understanding of life will allow us to make better decisions at all levels: managing ecological resources, regulating social interactions, planning urban systems, commercializing biotechnology, and more.
We are increasingly designing living systems: from husbandry in ancient times to molecular robots (Benenson et al., 2004) and synthetic biology in the present and near future. The complexity of living systems limits the scalability of the systems we can design. For example, electronic circuits are scalable because their interactions can be regulated. Even when there is a registry of standard biological parts 5 , it is difficult to isolate components. Moreover, unexpected chemical interactions bound the complexity of molecular machines because of limited scalability. Techniques based on evolution or self-organization have produced some advances, but there is much to do before we will be able to design living systems reliably. The interactions between components have been a limitation, as these generate novel information, which limits predictability (Gershenson, 2013b). Guiding these interactions has to be the way forward in the design of living systems.
The creation of artificial life is having deep implications in society and culture. The film "Mechanical Love" (Ambo, 2009;Gershenson et al., 2010) explores two implications: how pet robots can benefit human beings emotionally, and how artificial creatures, which look closely like human beings, generate an "uncanny valley" (Mori, 1970), i.e., discomfort because they look real but not real enough. As ALife progresses and its applications permeate into society, how will society be transformed as living artifacts are used? Will we still distinguish artificial from biological life?
As mentioned above, the methods and insights of ALife have been also permeating into biology, in the sense that computational modeling is now commonplace in all branches of biology. Will the successes of ALife imply its absorption into the mainstream study of life? That seems to be the case. If this tendency continues, soon ALife will no longer be "artificial."