Complexity Measures: Open Questions and Novel Opportunities in the Automatic Design and Analysis of Robot Swarms

1. Introduction

Metrics that quantify the complexity of a system and measure information processing are used in a wide range of scientific areas, including neuroscience, physics, and computer science. In the scientific literature, the word complexity is overloaded, as it may refer to the amount of effort needed to describe a system, or to create it, or also to quantify its structure both in terms of components and dynamical relations among its parts. For example, let us consider a swarm of robots: we may ask what is the complexity of a function describing the overall behavior of the swarm, or what is the complexity of the problem of optimally assigning tasks to the robots, or what is the complexity of each of the tasks. These objectives require different measures, each addressing a specific question. As a consequence, there is no unique and all-encompassing complexity measure: a plethora of metrics are available. Most come from information theory, which abstracts from specific system's details and focuses on information processing. While notable results have been attained, we believe that the potential of these methods has still to be fully exploited in the automatic design of robot swarms and in the analysis of their behaviors.

In automatic design methods, the design problem is cast into an optimization problem that is solved either off-line or on-line, i.e., either before the swarm is deployed in its target environment or while the swarm is operating in it. A prominent example of automatic design is evolutionary robotics (ER), where the control software—typically an artificial neural network (ANN)—is optimized by means of an evolutionary algorithm (Nolfi and Floreano, 2000). A number of alternative methods depart from the classical ER by employing control software architectures other than ANNs and/or optimization techniques other than evolutionary computation (Watson et al., 2002; Hecker et al., 2012; Francesca et al., 2014; Gauci et al., 2014). A review of the main studies on automatic design of robot swarms—both off-line and on-line—is provided by Francesca and Birattari (2016).

The aim of this paper is to outline what we think are the most important open questions and to describe opportunities to use complexity measures for supporting the automatic design of swarms of robots and the analysis of their behaviors. In section 2, we provide an introduction to complexity measures. In section 3, we highlight the main contributions to the robotics field. In section 4, we illustrate our perspective and outline relevant open questions.

2. A Capsule Introduction to Complexity Measures

The notion of complexity is multifaceted. If, by the term “complex,” one means “difficult to predict,” then a suitable metric is provided by information theory with Shannon entropy (Shannon, 1948). Let us consider a simple system of which we observe the state at a given time. The observations can be modeled as a random variable X, which can assume values from a finite and discrete domain ${X}$. If the observation is $x\in {X}$, which has a probability P(x), then the amount of information carried by the observation of x is defined as $\frac{1}{logP(x)}=-logP(x)$¹. Shannon entropy is defined as the expected value of the information of all symbols: $H(X)=-\sum _{x\in {X}}P(x)logP(x)$. Intuitively, H(X) measures the amount of surprise—or, equivalently, the lack of knowledge—about the system; we may also observe that Shannon entropy measures the degree of disorder in a system or process. Many complexity measures are based on Shannon entropy. For example, the reciprocal influence between two parts of a system can be estimated by computing their mutual information, defined as I(X; Y) = H(X) + H(Y) − H(X, Y), where H(X, Y) is the joint entropy of the variables X and Y, defined on the basis of the joint probability P(x, y). I(X; Y) provides a measure of the information we can gain on a variable, by observing the other. Information-theoretic metrics are currently widely applied, as they have the property of being model independent and able to capture non-linear relations. In practice, probabilities are usually estimated through the observed frequencies.

When the objective is to measure the complexity of the description of a system, then algorithmic complexity may be used, as proposed by Kolmogorov (1965): the complexity of a string of symbols is defined as the length of the shortest program producing it. This measure is not computable in general, but approximations are available, such as the ones based on compression algorithms (Lempel and Ziv, 1976). Shannon entropy and Kolmogorov complexity are conceptually different (Teixeira et al., 2011). The former measures the average uncertainty of a random variable X, and so it estimates the difficulty of predicting the next symbol of a sequence received from a source. Conversely, Kolmogorov complexity measures the length of the minimal (algorithmic) description of a given sequence of symbols σ, therefore it estimates the difficulty of describing or reconstructing the sequence. However, they both capture the notion of compressibility of a signal and, in particular, they are null when X (resp. σ) is constant and maximal when X (resp. σ) is random.

Kolmogorov complexity also provides a theoretical framework for the principle known as Occam's razor that states that among all the possible explanations of a set of data, the simplest one is preferable. A similar argument supports the notion of stochastic complexity, proposed by Rissanen (1986), which is the shortest description of the data with respect to a given probabilistic model.

The term “complex” is often used for capturing the notion of structure or pattern observed in data or in the dynamics of a system, once random elements are discarded. This concept is also related to the extent to which correlations distribute across the parts of the system observed (Grassberger, 1986a). The intuition is that high complexity should be associated to conditions characterized by a mixture of order and disorder, structure and randomness, easily predictable dynamics and novelty. Along this line, several measures have been proposed (Grassberger, 1986a; Lindgren and Nordahl, 1988; Li, 1991; Crutchfield, 1994; Gell-Mann and Lloyd, 1996; Shalizi and Crutchfield, 2001). A survey on complexity metrics is out of the scope of this contribution and we refer the interested reader to prominent works on the subject (Grassberger, 1986a; Lindgren and Nordahl, 1988; Badii and Politi, 1999; Lloyd, 2001; Prokopenko et al., 2009; Lizier, 2013; Moore et al., 2018; Thurner et al., 2018; Valentini et al., 2018).

3. Complexity Measures in Robotics

A possibility for using information theory in robotics is enabled by the notion of sensory-motor coordination (Pfeifer and Scheier, 1997) which emphasizes the role of the loop between sensors and actuators in robots performing cognitive tasks. Sensory-motor coordination models can be described in terms of dynamical systems and control theory, which are suitable for analyses based on information theory. More specifically, the sensory-motor loop expresses both the effect that sensors have on actuators and the effect that actuators have on sensors. The former is mediated by the robot's control software; the latter by the environment. Information-theoretic measures can be used to study some properties of system dynamics (Islam and Murase, 2005; Lizier, 2013; Beer and Williams, 2015; Da Rold, 2018) and to characterize the information flow in the sensory-motor loop (Lungarella and Pfeifer, 2001; Lungarella et al., 2005; Lungarella and Sporns, 2006; Ay and Zahedi, 2014). Notably, this approach makes it possible to quantitatively study the relation between the robot and the environment (Beer, 1995, 2008; Smithers, 1995; Tarapore et al., 2004, 2006; Nehmzow, 2008; Schmidt et al., 2013; Butail et al., 2014; Izquierdo et al., 2015), which is fundamental in embodied systems (Pfeifer and Scheier, 2001). A sound and thorough treatise on the dynamics emerging from the interaction among robot, control program and environment is provided by Nehmzow (2008).

Information-theoretic measures typically used in these contexts are mainly based on Shannon entropy (Cover and Thomas, 2012) and range from mutual information (Lindgren, 2014) and transfer entropy (Schreiber, 2000) to predictive information (Grassberger, 1986b; Crutchfield and Young, 1989; Bialek et al., 2001; Martius et al., 2013). Predictive information (PI) has been successfully used to quantify properties of emergent behaviors and as an objective function for designing robots able to show the so-called self-organization dynamics (Der et al., 2008; Martius and Olbrich, 2015). The PI of a system is computed by extracting a time series from a set of variables, e.g., the robot's sensor readings. The PI of the system is then defined as the mutual information of the future and the past within the time series. If the robot's behavior is such that features of the environment (e.g., a light gradient) are exploited to achieve the robot's goals (e.g., phototaxis), then the sensory-motor loop tends to produce coordinated patterns. These patterns are captured by high values of PI computed across sensor time series. Conversely, maximizing PI produces self-organized behaviors (Ay et al., 2008).

To the best of our knowledge, Odagiri et al. (1998) were the first to study the complexity of an ANN controlling a robot and to highlight its correlation with the complexity of the environment in which it is evolved². In their work, the ANN controlling a Khepera robot is evolved in different environments characterized by different levels of complexity (obtained by construction) and the complexity of the ANN is evaluated in terms of non-zero weights in the network. While this way of estimating ANN complexity is common in machine learning and is not based on information theory, this paper provides a clear statement of the problem. Results show that the ANN complexity is correlated with the complexity of the environment and the authors suggest to use this metric to estimate the complexity of the environment “as seen by the robot.” A similar work achieving analogous conclusions has been proposed by Capi (2007). Yang and Anderson (2011) address the same problem by evolving artificial agents that have to reach a target cell in a grid with obstacles; robots are guided by means of a depth search algorithm. The authors propose a metric for estimating environmental complexity based on entropy and compressibility of the grid in which the agents move. A linear regression model on these two variables is then inferred and used to estimate the average number of steps required by the agent to reach the target.

As different complexity measures capture different features of a system, one should aim at producing a complexity fingerprint by computing several metrics, rather than identifying a single metric able to summarize all the relevant properties related to complexity (Roli et al., 2018). Teo and Abbass (2005) address this issue by proposing a multidimensional complexity measure, consisting of several different metrics by means of which a partial ordering can be defined.

A prominent question in natural and artificial evolution is whether complexity increases over generations and what is its relationship with fitness. A recent work (Joshi et al., 2013) addresses this issue in simulation by evolving artificial agents controlled by extended ANNs. The metrics used in this research are mutual information, predictive information and also integrated information (Balduzzi and Tononi, 2008). This latter metric has been proposed with the aim of capturing to what extent a system is able to integrate information coming from the environment. Agents controlled by a network of stochastic transition functions have been subject to artificial evolution with the goal of finding the exit of a maze in the shortest time. The outcome of the research is that complexity is positively correlated with fitness. More precisely, the minimal complexity among all the evolved ANN corresponding to any one fitness value is a quantity increasing with fitness. A measure of behavioral diversity in a group of robots has been proposed by Balch (2000) who defines a metric based on hierarchical clustering of behaviors across the group of robots. This information-theoretic approach enables us to quantitatively correlate the heterogeneity of the group of robots with performance.

Information-theoretic measures have also been used as task-agnostic merit factors for the design of coordinated behaviors in evolutionary robotics³. An example of this approach is the achievement of a coordinated behavior by maximizing the average mutual information between all pairs of robot motors (Olsson et al., 2005; Sporns and Lungarella, 2006; Salge and Polani, 2011; Sperati et al., 2014). The aim of using information-theoretic measures in this setting is to bias evolution toward robot control software that enables the robots to attain some useful emergent property, which can be exploited for the specific task at hand⁴. Of similar spirit are the works of Martius et al. (2013) and Ay et al. (2008), which present a principled approach to derive control software on the basis of integrated information for attaining self-organized and explorative behaviors. The coordination of a group of robots is the focus of work by Capdepuy et al. (2007), in which the multi-robot case is subjected to an information-theoretic analysis. Finally, we mention the work by Klyubin et al. (2008), where a new metric called empowerment is proposed with the aim of guiding evolution to producing robots that, all things being equal, choose actions that maximize the number of their future possible actions.

4. Open Questions and Opportunities for Future Research

Recent results attained by using information-theoretic and complexity measures in analysing and designing robot behaviors motivate further investigations. Some questions are still open and we believe that addressing them will improve swarm robotics, in particular. Here we focus on a specific subject that we think has still to be thoroughly addressed: the relation between complexity of individual robot, swarm and environment. As the observed behavior of a robot is the result of the interplay of control software, robot's physical features and environment, the complexity of an individual robot does not necessarily correspond to the complexity of its control software. Complex control software might produce simple behaviors because the robot cannot exploit any significant property of the environment or because the environment is overconstrained; conversely, simple control software may generate rather complex behaviors by exerting simple interactions in a complex environment. This, in turn, poses the question as to what extent the complexity of the environment can be assessed: is it possible to measure environment complexity without referring to a specific robot platform? If not, are there general approaches to assess relative complexity? Should one abandon the idea of measuring the complexity of the two entities separately and measure the complexity of the compound instead? The situation is of course more complicated in the case of robot swarms, as interactions among robots produce an emerging behavior at a higher level. The relation between the complexity of individual robot and the swarm is again to be studied: is it in general possible to attain any complexity level in the swarm independently of the individual complexity by operating on the interactions? If not, are there bounds on the complexity that can be computed? Concepts and methods from the physics of collective systems and statistical physics (Nicolis and Prigogine, 1977; Binney et al., 1992; Kauffman, 1993; Bar-Yam, 1997; Badii and Politi, 1999) are likely to provide useful tools for addressing these questions, especially for what concerns the relation between micro and macro levels, i.e., between robots and swarm. Nevertheless, the hypothesis that robots behave as particles is seldom verified and new advancements of these theories should be developed to properly address these questions. Furthermore, we may ask if meso-levels appear as intermediate structures between micro and macro levels, as in the case of the so-called sandwiched emergence (Lane, 2006); these meso-levels have indeed a bidirectional effect as they influence both upper and lower levels.

We believe that addressing these questions would lead to important improvements to the automatic design of robot swarms. Swarm robotics is a promising approach to coordinating large groups of robots (Dorigo et al., 2014; Yang et al., 2018), which has already attracted the attention of the wider scientific community (e.g., see Rubenstein et al., 2014; Werfel et al., 2014; Garattoni and Birattari, 2018; Slavkov et al., 2018; Yu et al., 2018; Li et al., 2019; Xie et al., 2019). As observed by Brambilla et al. (2013), an engineering approach to the development of robot swarms is still in its infancy (Winfield et al., 2005; Brambilla et al., 2012, 2015; Francesca et al., 2015; Francesca and Birattari, 2016; Khaluf et al., 2016). In automatic design, a space of possible design instances is explored by an optimization algorithm with the goal of maximizing an appropriate mission-specific performance measure. In the off-line case, the performance of candidate design instances explored by the optimization algorithm is assessed via computer simulations (Birattari et al., 2019). One of the main issues to be addressed in this case is the discrepancy between simulation and reality. This discrepancy—usually named the reality gap—is often the reason for performance drops when control software developed in simulation is deployed on real robots. Recently, Birattari et al. (2016) show that the reality gap is the cause of what has been named overdesign: in an off-line automatic design process, the performance in simulation steadily increases with design effort (e.g., iterations of the optimization algorithm), while the one in reality increases up to a certain level and then starts decreasing. Furthermore, the performance drop due to the reality gap is a relative problem: different design methods are affected to a different extent by the same difference between simulation and reality (Francesca et al., 2014) and a rank inversion can be observed: on the same mission, the control software produced by design method A might perform better than the one produced by design method B when they are evaluated in simulation; while it could be the other way around when the control software is ported to the robots. Overdesign and eventually performance drop and rank inversion have a statistical interpretation in terms of model identification: the choice of the structure of a statistical model implies—even implicitly—a trade-off between bias and variance, which represent two kinds of error affecting an estimator. Models of a relatively simple structure are characterized by high bias and tend to under-fit data, whilst those of a relatively complex ones are affected by a large variance and tend to overfit (Geman et al., 1992). As it has been recently shown, performance drop and rank inversion may be observed also in a simulation-only experiments (Ligot and Birattari, 2018, 2019): that is, when behaviors are designed using a simulation model and tested using another one. This indicates that performance drop and rank inversion should not be ascribed to some unique relationship between reality and the simulation environment used in the design process, but rather to an intrinsic characteristic of a design method—i.e., the complexity of the control software produced: high complexity exposes a method to the risk of overfitting the simulation environment used in the design process. Overfitting is commonly addressed by invoking the parsimony principle, stating that a model should be as simple as possible. In fact, this principle can be seen as a statistical formalization of the Occam's razor, usually expressed as “shave away all that is unnecessary” or “everything should be made as simple as possible, but not simpler” (Burnham and Anderson, 2002).

In the light of this last consideration, we believe that a principled estimation of the complexities of swarm, individual robots and environment may help address this issue by suggesting the best levels of complexity to attain in the design. Contrary to some established fields like statistics and machine learning in which the scientific community has agreed on the adoption of some complexity measures—e.g., and Akaike's information criterion (Akaike, 1973) and Vapnik-Chervonenkis dimension (Vapnik, 1995)—the swarm robotics community has not yet reached consensus on any complexity measure.

In a futuristic scenario, we imagine the possibility of measuring the complexity of a swarm robotics task and designing the control software accordingly. Besides the questions already stated, this perspective poses further issues, as measuring the complexity of a task is a non-trivial problem and in general it depends on the robot's model. We believe that information theory may provide a suitable framework for addressing these questions because it makes it possible to abstract from implementation details and focus on interactions and dynamics related to information processing. In Figures 1 and 2, an example of the use of complexity measures inside the automatic design flow is sketched.

FIGURE 1

Figure 1. Generic design cycle: the control software is deployed onto the robots, which are then evaluated; on the basis of the evaluation, some parameters of the control software are changed by means of an optimization method. Complexity metrics are fed into the evaluation block. Off-line design: robots are evaluated in simulation. The metrics can be used as a regularization component of the fitness function used in the evaluation block: besides a fitness component accounting for the performance of the swarm on the specific task, a further component can be added to penalize (i) individual robots or (ii) swarms that exhibit a complexity level not complying with the requirements. In the first case, the average predictive information (PI) of the robots can be taken, whilst in the second case the average pairwise mutual information can be used (see Figure 2). Alternatively, a multi-objective approach can be chosen, in which the swarm performance and the complexity metric are the two criteria considered in the optimization process. In this latter case, the user can choose a personal trade-off between performance and complexity. Mixed off-line and on-line design: an off-line design phase is first run, in which the complexity of the swarm (or the individuals, or both) is maximized. Subsequently, an on-line design process takes place for tuning the control software parameters with the goal of specializing the swarm to the task at hand and to keep the complexity in a given range.

FIGURE 2

Figure 2. An example of the evaluation of complexity metrics. We suppose that robots are controlled by a probabilistic finite state automaton, hence the swarm state is given by a tuple composed of symbols from a finite alphabet, each representing one state in the automaton (this choice is not a limitation, as it is possible to compute complexity metrics for any choice of the variables representing robots' state). Individual metrics can be computed by considering the time sequence of states for each robot, while global swarm measures are a function of the whole swarm state (or a portion of it). In the figure, the predictive information (PI) of each robot can be computed on the basis of the time sequence of robot states; conversely, as a swarm measure the average pairwise mutual information can be computed. Indeed, high values of this metric are expected to favor global coordinated behaviors (Sperati et al., 2014).

Besides the open questions already mentioned, we also envision room for improvement in the use of task-agnostic merit factors in automatic design of robot swarms. While some results have been attained (Sperati et al., 2008; Der and Martius, 2012; Mouret and Doncieux, 2012), we believe that the potential of this approach has still to be fully expressed. For example, we expect that combining complexity measures with task-specific objective functions—e.g., in a multi-objective framework—may lead to swarms characterized by fast re-calibration in case of environmental or task changes. In the longer term, we envision the possibility of tuning the complexity of an adaptive swarm depending on the—possibly changing—requirements. For example, highly complex control software may be employed with its complexity controlled during the design process or even dynamically during operation. A viable way to attain this could be either a priori designing the control software instances so as to tune their complexity by means of specific variables acting on their structure and parameters (internal regulation) or by properly calibrating the interactions between robot and environment, and between the robots of the swarm. This last kind of control (external regulation) may be achieved, for example, by properly choosing type and properties of environmental features (i.e., by tuning the robots' environmental niche) or by selecting which sensors and actuators the robots can use.

Apart from few preliminary investigations (Roli et al., 2013, 2015, 2018), there is still much room for application of information-theoretic and complexity measures in the analysis of robot behavior. Besides the use of such metrics during the design process (e.g., to assessing the complexity of the swarm or the individual robots), we envisage a real-world scenario in which the complexity level of a robot swarm is monitored in order to detect specific phases of its behavior (e.g., when a decision has to be collectively taken) and possible failures—e.g., when a large discrepancy between the expected and actual complexity of the swarm is observed it might be the case that some malfunctioning dynamics is taking place.

Author Contributions

AR coordinated the paper and drafted a preliminary version of the manuscript. All authors equally contributed to the ideas and the writing.

Funding

The project has received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreement No. 681872).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

AR is a member of the INdAM Research group GNCS. MB acknowledges support from the Belgian Fonds de la Recherche Scientifique – FNRS.

Footnotes

1. ^Usually, the logarithms are taken to the base 2, so as to express information in terms of bits.

2. ^The generation of environments with different degrees of complexity for testing or evolutionary purposes is not new. E.g., see Stirling et al. (2010).

3. ^The use of task-independent objective functions in artificial evolution is not a new idea—e.g., see Lehman and Stanley (2011). However, in this paper we are concerned only with approaches related to complexity and information-theoretic measures.

4. ^A similar approach is that of guided self-organization (Prokopenko, 2014).

References

Akaike, H. (1973). “Information theory as an extension of the maximum likelihood principle,” in Second International Symposium on Information Theory, eds B. Petrov and F. Csaki (Budapest: Akademiai Kiado), 267–281.

Google Scholar

Ay, N., Bertschinger, N., Der, R., Güttler, F., and Olbrich, E. (2008). Predictive information and explorative behavior of autonomous robots. Eur. Phys. J. B 63, 329–339. doi: 10.1140/epjb/e2008-00175-0

CrossRef Full Text | Google Scholar

Ay, N., and Zahedi, K. (2014). “On the causal structure of the sensorimotor loop,” in Guided Self-Organization: Inception, ed M. Prokopenko (Berlin: Springer), 261–294.

Google Scholar

Badii, R., and Politi, A. (1999). Complexity: Hierarchical Structures and Scaling in Physics, Vol. 6. Cambridge: Cambridge University Press.

Google Scholar

Balch, T. (2000). Hierarchic social entropy: an information theoretic measure of robot group diversity. Auton. Robots 8, 209–238. doi: 10.1023/A:1008973424594

CrossRef Full Text | Google Scholar

Balduzzi, D., and Tononi, G. (2008). Integrated information in discrete dynamical systems: motivation and theoretical framework. PLoS Comput. Biol. 4:e1000091. doi: 10.1371/journal.pcbi.1000091

PubMed Abstract | CrossRef Full Text | Google Scholar

Bar-Yam, Y. (1997). Dynamics of Complex Systems. Studies in Nonlinearity. Reading, MA: Addison–Wesley.

Google Scholar

Beer, R. (1995). A dynamical systems perspective on agent-environment interaction. Artif. Intell. 72, 173–215. doi: 10.1016/0004-3702(94)00005-L

CrossRef Full Text | Google Scholar

Beer, R. (2008). “The dynamics of brain-body-environment systems: a status report,” in Handbook of Cognitive Science: An Embodied Approach, eds P. Calvo and T. Gomila (Amsterdam: Elsevier), 99–120.

Google Scholar

Beer, R., and Williams, P. (2015). Information processing and dynamics in minimally cognitive agents. Cogn. Sci. 39, 1–38. doi: 10.1111/cogs.12142

PubMed Abstract | CrossRef Full Text | Google Scholar

Bialek, W., Nemenman, I., and Tishby, N. (2001). Predictability, complexity, and learning. Neural Comput. 13, 2409–2463. doi: 10.1162/089976601753195969

PubMed Abstract | CrossRef Full Text | Google Scholar

Binney, J., Dowrick, N., Fisher, A., and Newman, M. (1992). The Theory of Critical Phenomena. Oxford: Oxford University Press.

Google Scholar

Birattari, M., Delhaisse, B., Francesca, G., and Kerdoncuff, Y. (2016). “Observing the effects of overdesign in the automatic design of control software for robot swarms,” in Swarm Intelligence – ANTS, Volume 9882 of LNCS (Cham: Springer), 45–57.

Google Scholar

Birattari, M., Ligot, A., Bozhinoski, D., Brambilla, M., Francesca, G., Garattoni, L., et al. (2019). Automatic off-line design of robot swarms: a manifesto. Front. Robot. AI 6:59. doi: 10.3389/frobt.2019.00059

CrossRef Full Text | Google Scholar

Brambilla, M., Brutschy, A., Dorigo, M., and Birattari, M. (2015). Property-driven design for robot swarms: a design method based on prescriptive modeling and model checking. ACM Trans. Autonom. Adapt. Syst. 9, 17:1–17:28. doi: 10.1145/2700318

CrossRef Full Text | Google Scholar

Brambilla, M., Ferrante, E., Birattari, M., and Dorigo, M. (2013). Swarm robotics: a review from the swarm engineering perspective. Swarm Intell. 7, 1–41. doi: 10.1007/s11721-012-0075-2

CrossRef Full Text | Google Scholar

Brambilla, M., Pinciroli, C., Birattari, M., and Dorigo, M. (2012). “Property-driven design for swarm robotics,” in Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems, AAMAS (New York, NY: ACM), 139–146.

Google Scholar

Burnham, K., and Anderson, D. (2002). Model Selection and Multi-model Inference, 2nd Edn. Berlin: Springer.

Google Scholar

Butail, S., Ladu, F., Spinello, D., and Porfiri, M. (2014). Information flow in animal-robot interactions. Entropy 16, 1315–1330. doi: 10.3390/e16031315

CrossRef Full Text | Google Scholar

Capdepuy, P., Polani, D., and Nehaniv, C. L. (2007). “Maximization of potential information flow as a universal utility for collective behaviour,” in IEEE Symposium on Artificial Life (Piscataway, NJ: IEEE), 207–213.

Google Scholar

Capi, G. (2007). Multiobjective evolution of neural controllers and task complexity. IEEE Trans. Robot. 23, 1225–1234. doi: 10.1109/TRO.2007.910773

CrossRef Full Text | Google Scholar

Cover, T., and Thomas, J. (2012). Elements of Information Theory. Hoboken, NJ: John Wiley & Sons.

Google Scholar

Crutchfield, J. (1994). The calculi of emergence: computation, dynamics, and induction. Physica D 75, 11–54. doi: 10.1016/0167-2789(94)90273-9

CrossRef Full Text | Google Scholar

Crutchfield, J., and Young, K. (1989). Inferring statistical complexity. Phys. Rev. Lett. 63, 105–108. doi: 10.1103/PhysRevLett.63.105

PubMed Abstract | CrossRef Full Text | Google Scholar

Da Rold, F. (2018). Information-theoretic decomposition of embodied and situated systems. Neural Netw. 103, 94–107. doi: 10.1016/j.neunet.2018.03.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Der, R., Güttler, F., and Ay, N. (2008). “Predictive information and emergent cooperativity in a chain of mobile robots,” in ALIFE (Cambridge, MA: MIT Press), 166–172.

Google Scholar

Der, R., and Martius, G. (2012). The Playful Machine: Theoretical Foundation and Practical Realization of Self-organizing Robots, Vol. 15. Berlin: Springer Science & Business Media.

Google Scholar

Dorigo, M., Birattari, M., and Brambilla, M. (2014). Swarm robotics. Scholarpedia 9:1463. doi: 10.4249/scholarpedia.1463

CrossRef Full Text | Google Scholar

Francesca, G., and Birattari, M. (2016). Automatic design of robot swarms: achievements and challenges. Front. Robot. AI 3:29. doi: 10.3389/frobt.2016.00029

CrossRef Full Text | Google Scholar

Francesca, G., Brambilla, M., Brutschy, A., Garattoni, L., Miletitch, R., Podevijn, G., et al. (2015). AutoMoDe-chocolate: automatic design of control software for robot swarms. Swarm Intell. 9, 125–152. doi: 10.1007/s11721-015-0107-9

CrossRef Full Text | Google Scholar

Francesca, G., Brambilla, M., Brutschy, A., and Trianni, V. (2014). AutoMoDe: a novel approach to the automatic design of control software for robot swarms. Swarm Intell. 8, 89–112. doi: 10.1007/s11721-014-0092-4

CrossRef Full Text | Google Scholar

Garattoni, L., and Birattari, M. (2018). Autonomous task sequencing in a robot swarm. Sci. Robot. 3:eaat0430. doi: 10.1126/scirobotics.aat0430

CrossRef Full Text | Google Scholar

Gauci, M., Chen, J., Li, W., Dodd, T., and Gross, R. (2014). “Clustering objects with robots that do not compute,” in Proceedings of the 2014 International Conference on Autonomous Agents and Multi-agent Systems, AAMAS (New York, NY: ACM), 421–428.

Google Scholar

Gell-Mann, M., and Lloyd, S. (1996). Information measures, effective complexity, and total information. Complexity 2, 44–52.

Google Scholar

Geman, S., Bienenstock, E., and Doursat, R. (1992). Neural networks and the bias/variance dilemma. Neural Comput. 4, 1–58. doi: 10.1162/neco.1992.4.1.1

CrossRef Full Text | Google Scholar

Grassberger, P. (1986a). How to measure self-generated complexity. Physica A 140, 319–325. doi: 10.1016/0378-4371(86)90238-4

CrossRef Full Text | Google Scholar

Grassberger, P. (1986b). Toward a quantitative theory of self-generated complexity. Int. J. Theor. Phys. 25, 907–938. doi: 10.1007/BF00668821

CrossRef Full Text | Google Scholar

Hecker, J., Letendre, K., Stolleis, K., Washington, D., and Moses, M. (2012). “Formica ex machina: ant swarm foraging from physical to virtual and back again,” in Swarm Intelligence – ANTS, Volume 7461 of LNCS (Berlin: Springer), 252–259.

Google Scholar

Islam, M., and Murase, K. (2005). Chaotic dynamics of a behavior-based miniature mobile robot: effects of environment and control structure. Neural Netw. 18, 123–144. doi: 10.1016/j.neunet.2004.09.002

CrossRef Full Text | Google Scholar

Izquierdo, E., Williams, P., and Beer, R. (2015). Information flow through a model of the C. elegans klinotaxis circuit. PLoS ONE 10:e0140397. doi: 10.1371/journal.pone.0140397

PubMed Abstract | CrossRef Full Text | Google Scholar

Joshi, N., Tononi, G., and Koch, C. (2013). The minimal complexity of adapting agents increases with fitness. PLoS Comput. Biol. 9:e1003111. doi: 10.1371/journal.pcbi.1003111

PubMed Abstract | CrossRef Full Text | Google Scholar

Kauffman, S. (1993). The Origins of Order: Self-organization and Selection in Evolution. Oxford: Oxford University Press.

Google Scholar

Khaluf, Y., Birattari, M., and Rammig, F. (2016). Analysis of long-term swarm performance based on short-term experiments. Soft Comput. 20, 37–48. doi: 10.1007/s00500-015-1958-0

CrossRef Full Text | Google Scholar

Klyubin, A., Polani, D., and Nehaniv, C. (2008). Keep your options open: an information-based driving principle for sensorimotor systems. PLoS ONE 3:e4018. doi: 10.1371/journal.pone.0004018

PubMed Abstract | CrossRef Full Text | Google Scholar

Kolmogorov, A. (1965). Three approaches to the quantitative definition of information. Probl. Informat. Transm. 1, 1–7.

Google Scholar

Lane, D. (2006). “Hierarchy, complexity, society,” in Hierarchy in Natural and Social Sciences, ed D. Pumain (Berlin: Springer), 81–119.

Google Scholar

Lehman, J., and Stanley, K. (2011). Abandoning objectives: evolution through the search for novelty alone. Evol. Comput. 19, 189–223. doi: 10.1162/EVCO_a_00025

PubMed Abstract | CrossRef Full Text | Google Scholar

Lempel, A., and Ziv, J. (1976). On the complexity of finite sequences. IEEE Trans. Informat. Theory 22, 75–81. doi: 10.1109/TIT.1976.1055501

CrossRef Full Text | Google Scholar

Li, S., Batra, R., Brown, D., Chang, H.-D., Ranganathan, N., Hoberman, C., et al. (2019). Particle robotics based on statistical mechanics of loosely coupled components. Nature 567, 361–365. doi: 10.1038/s41586-019-1022-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, W. (1991). On the relationship between complexity and entropy for markov chains and regular languages. Complex Syst. 5, 381–399.

Google Scholar

Ligot, A., and Birattari, M. (2018). “On mimicking the effects of the reality gap with simulation-only experiments,” in Swarm Intelligence – ANTS, Volume 11172 of LNCS, eds M. Dorigo, M. Birattari, C. Blum, A. L. Christensen, A. Reina, and V. Trianni (Cham: Springer), 109–122.

Google Scholar

Ligot, A., and Birattari, M. (2019). Simulation-only experiments to mimic the effects of the reality gap in the automatic design of robot swarms. Swarm Intell. 14. doi: 10.1007/s11721-019-00175-w

CrossRef Full Text | Google Scholar

Lindgren, K. (2014). Information Theory for Complex Systems – An Information Perspective on Complexity in Dynamical Systems, Physics, and Chemistry. Lecture notes. Chalmers. Available online at: http://studycas.com/c/courses/it

Google Scholar

Lindgren, K., and Nordahl, M. (1988). Complexity measures and cellular automata. Complex Syst. 2, 409–440.

Google Scholar

Lizier, J. (2013). The Local Information Dynamics of Distributed Computation in Complex Systems. Springer Theses Series. Berlin: Springer.

Google Scholar

Lloyd, S. (2001). Measures of complexity: a nonexhaustive list. IEEE Cont. Syst. Mag. 21, 7–8. doi: 10.1109/MCS.2001.939938

CrossRef Full Text | Google Scholar

Lungarella, M., Pegors, T., Bulwinkle, D., and Sporns, O. (2005). Methods for quantifying the informational structure of sensory and motor data. Neuroinformatics 3, 243–262. doi: 10.1385/NI:3:3:243

PubMed Abstract | CrossRef Full Text | Google Scholar

Lungarella, M., and Pfeifer, R. (2001). “Robots as cognitive tools: information theoretic analysis of sensory-motor data,” in Proceedings of the 2nd International IEEE–RAS Conference on Humanoid Robots (Piscataway, NJ: IEEE), 245–252.

Google Scholar

Lungarella, M., and Sporns, O. (2006). Mapping information flow in sensorimotor networks. PLoS Comput. Biol. 2:e144. doi: 10.1371/journal.pcbi.0020144

PubMed Abstract | CrossRef Full Text | Google Scholar

Martius, G., Der, R., and Ay, N. (2013). Information driven self-organization of complex robotic behaviors. PLoS ONE 8:e63400. doi: 10.1371/journal.pone.0063400

PubMed Abstract | CrossRef Full Text | Google Scholar

Martius, G., and Olbrich, E. (2015). Quantifying emergent behavior of autonomous robots. Entropy 17, 7266–7297. doi: 10.3390/e17107266

CrossRef Full Text | Google Scholar

Moore, D., Valentini, G., Walker, S. I., and Levin, M. (2018). Inform: efficient information-theoretic analysis of collective behaviors. Front. Robot. AI 5:60. doi: 10.3389/frobt.2018.00060

CrossRef Full Text | Google Scholar

Mouret, J.-B., and Doncieux, S. (2012). Encouraging behavioral diversity in evolutionary robotics: an empirical study. Evol. Comput. 20, 91–133. doi: 10.1162/EVCO_a_00048

PubMed Abstract | CrossRef Full Text | Google Scholar

Nehmzow, U. (2008). Robot Behaviour: Design, Description, Analysis and Modelling. Berlin: Springer.

Google Scholar

Nicolis, G., and Prigogine, I. (1977). Self-Organization in Nonequilibrium Systems. Hoboken, NJ: John Wiley & Sons.

Google Scholar

Nolfi, S., and Floreano, D. (2000). Evolutionary Robotics. Cambridge, MA: The MIT Press.

PubMed Abstract | Google Scholar

Odagiri, R., Yu, W., Asai, T., Yamakawat, O., and Murase, K. (1998). “Measuring the complexity of the real environment with evolutionary robot: evolution of a real mobile robot khepera to have a minimal structure,” in The 1998 IEEE International Conference on Evolutionary Computation Proceedings, 1998. IEEE World Congress on Computational Intelligence (Piscataway, NJ: IEEE), 348–353.

Google Scholar

Olsson, L., Nehaniv, C. L., and Polani, D. (2005). “Sensor adaptation and development in robots by entropy maximization of sensory data,” in IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA) (Piscataway, NJ: IEEE), 587–592.

Google Scholar

Pfeifer, R., and Scheier, C. (1997). Sensory motor coordination: the metaphor and beyond. Robot. Autono. Syst. 20, 157–178. doi: 10.1016/S0921-8890(97)80707-5

CrossRef Full Text | Google Scholar

Pfeifer, R., and Scheier, C. (2001). Understanding Intelligence. Cambridge, MA: The MIT Press.

Google Scholar

Prokopenko, M., (ed.). (2014). Guided Self-organization: Inception. Berlin: Springer Science & Business Media.

Google Scholar

Prokopenko, M., Boschetti, F., and Ryan, A. (2009). An information-theoretic primer on complexity, self-organization, and emergence. Complexity 15, 11–28. doi: 10.1002/cplx.20249

CrossRef Full Text | Google Scholar

Rissanen, J. (1986). Stochastic complexity and modeling. Ann. Stat. 14, 1080–1100. doi: 10.1214/aos/1176350051

CrossRef Full Text | Google Scholar

Roli, A., Ligot, A., and Birattari, M. (2018). “Complexity measures in automatic design of robot swarms: an exploratory study,” in Artificial Life and Evolutionary Computation (WIVACE) 2017, volume 830 of CCIS, eds M. Pelillo, I. Poli, A. Roli, R. Serra, D. Slanzi, and M. Villani (Cham: Springer), 243–256.

Google Scholar

Roli, A., Villani, M., Serra, R., Benedettini, S., Pinciroli, C., and Birattari, M. (2015). “Dynamical properties of artificially evolved boolean network robots,” in Congress of the Italian Association for Artificial Intelligence (Berlin: Springer), 45–57.

Google Scholar

Roli, A., Villani, M., Serra, R., Garattoni, L., Pinciroli, C., and Birattari, M. (2013). “Identification of dynamical structures in artificial brains: an analysis of boolean network controlled robots,” in Congress of the Italian Association for Artificial Intelligence (Berlin: Springer), 324–335.

Google Scholar

Rubenstein, M., Cornejo, A., and Nagpal, R. (2014). Programmable self-assembly in a thousand-robot swarm. Science 345, 795–799. doi: 10.1126/science.1254295

PubMed Abstract | CrossRef Full Text | Google Scholar

Salge, C., and Polani, D. (2011). “Local information maximisation creates emergent flocking behaviour,” in Advances in Artificial Life, ECAL 2011: Proceedings of the Eleventh European Conference on Synthesis and Simulation of Living Systems (Cambridge, MA: MIT Press), 688–696.

Google Scholar

Schmidt, N., Hoffmann, M., Nakajima, K., and Pfeifer, R. (2013). Bootstrapping perception using information theory: case studies in a quadruped robot running on different grounds. Adv. Complex Syst. 16:1250078. doi: 10.1142/S0219525912500786

CrossRef Full Text | Google Scholar

Schreiber, T. (2000). Measuring information transfer. Phys. Rev. Lett. 85, 461–464. doi: 10.1103/PhysRevLett.85.461

PubMed Abstract | CrossRef Full Text | Google Scholar

Shalizi, C., and Crutchfield, J. (2001). Computational mechanics: pattern and prediction, structure and simplicity. J. Stat. Phys. 104, 817–879. doi: 10.1023/A:1010388907793

CrossRef Full Text | Google Scholar

Shannon, C. (1948). A mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423, 623–656. doi: 10.1002/j.1538-7305.1948.tb00917.x

CrossRef Full Text | Google Scholar

Slavkov, I., Carrillo-Zapata, D., Carranza, N., Diego, X., Jansson, F., Kaandorp, J., et al. (2018). Morphogenesis in robot swarms. Sci. Robot. 3:eaau9178. doi: 10.1126/scirobotics.aau9178

CrossRef Full Text | Google Scholar

Smithers, T. (1995). On quantitative performance measures of robot behaviour. Robot. Auton. Syst. 15, 107–133. doi: 10.1016/0921-8890(95)00017-A

CrossRef Full Text | Google Scholar

Sperati, V., Trianni, V., and Nolfi, S. (2008). Evolving coordinated group behaviours through maximisation of mean mutual information. Swarm Intell. 2, 73–95. doi: 10.1007/s11721-008-0017-1

CrossRef Full Text | Google Scholar

Sperati, V., Trianni, V., and Nolfi, S. (2014). “Mutual information as a task-independent utility function for evolutionary robotics,” in Guided Self-Organization: Inception, ed M. Prokopenko (Berlin: Springer), 389–414.

Google Scholar

Sporns, O., and Lungarella, M. (2006). “Evolving coordinated behavior by maximizing information structure,” in Artificial Life X: Proceedings of the Tenth International Conference on the Simulation and Synthesis of Living Systems, Vol. 10 (Cambridge, MA: MIT Press), 323.

Google Scholar

Stirling, T., Wischmann, S., and Floreano, D. (2010). Energy-efficient indoor search by swarms of simulated flying robots without global information. Swarm Intell. 4, 117–143. doi: 10.1007/s11721-010-0039-3

CrossRef Full Text | Google Scholar

Tarapore, D., Lungarella, M., and Gómez, G. (2004). “Fingerprinting agent-environment interaction via information theory,” in The Eighth International Conference on Intelligent Autonomous Systems (IAS-8), (Amsterdam: IOS Press), 10, 512–520.

Google Scholar

Tarapore, D., Lungarella, M., and Gómez, G. (2006). Quantifying patterns of agent–environment interaction. Robot. Auton. Syst. 54, 150–158. doi: 10.1016/j.robot.2005.09.024

CrossRef Full Text | Google Scholar

Teixeira, A., Matos, A., Souto, A., and Antunes, L. (2011). Entropy measures vs. kolmogorov complexity. Entropy 13, 595–611. doi: 10.3390/e13030595

CrossRef Full Text | Google Scholar

Teo, J., and Abbass, H. (2005). Multiobjectivity and complexity in embodied cognition. IEEE Trans. Evol. Comput. 9, 337–360. doi: 10.1109/TEVC.2005.846902

CrossRef Full Text | Google Scholar

Thurner, S., Hanel, R., and Klimek, P. (2018). Introduction to the Theory of Complex Systems. Oxford: Oxford University Press.

Google Scholar

Valentini, G., Moore, D., Hanson, J., Pavlic, T., Pratt, S., and Walker, S. (2018). “Transfer of information in collective decisions by artificial agents,” in Artificial Life Conference Proceedings (Cambridge, MA: MIT Press), 641–648.

Google Scholar

Vapnik, V. (1995). The Nature of Statistical Learning Theory. New York, NY: Springer.

Google Scholar

Watson, R., Ficici, S., and Pollack, J. (2002). Embodied evolution: distributing an evolutionary algorithm in a population of robots. Robot. Auton. Syst. 39, 1–18. doi: 10.1016/S0921-8890(02)00170-7

CrossRef Full Text | Google Scholar

Werfel, J., Petersen, K., and Nagpal, R. (2014). Designing collective behavior in a termite-inspired robot construction team. Science 343, 754–758. doi: 10.1126/science.1245842

PubMed Abstract | CrossRef Full Text | Google Scholar

Winfield, A., Sa, J., Fernández-Gago, M.-C., Dixon, C., and Fisher, M. (2005). On formal specification of emergent behaviours in swarm robotic systems. Int. J. Adv. Robot. Syst. 2, 363–371. doi: 10.5772/5769

CrossRef Full Text | Google Scholar

Xie, H., Sun, M., Fan, X., Lin, Z., Chen, W., Wang, L., et al. (2019). Reconfigurable magnetic microrobot swarm: multimode transformation, locomotion, and manipulation. Sci. Robot. 4:eaav8006. doi: 10.1126/scirobotics.aav8006

CrossRef Full Text | Google Scholar

Yang, G., and Anderson, G. (2011). “An experimental study of environmental complexity as seen by robots,” in IEEE Transactions on Systems, Man, and Cybernetics (Piscataway, NJ: IEEE), 3102–3106.

Google Scholar

Yang, G.-Z., Bellingham, J., Dupont, E. P., Fischer, P., Floridi, L., Full, R., et al. (2018). The grand challenges of science robotics. Sci. Robot. 3:eaar7650. doi: 10.1126/scirobotics.aar7650

CrossRef Full Text | Google Scholar

Yu, J., Wang, B., Du, X., Wang, Q., and Zhang, L. (2018). Ultra-extensible ribbon-like magnetic microswarm. Nat. Commun. 9:3260. doi: 10.1038/s41467-018-05749-6

PubMed Abstract | CrossRef Full Text | Google Scholar