Automatic Off-Line Design of Robot Swarms: A Manifesto

Designing collective behaviors for robot swarms is a difficult endeavor due to their fully distributed, highly redundant, and ever-changing nature. To overcome the challenge, a few approaches have been proposed, which can be classified as manual, semi-automatic, or automatic design. This paper is intended to be the manifesto of the automatic off-line design for robot swarms. We define the off-line design problem and illustrate it via a possible practical realization, highlight the core research questions, raise a number of issues regarding the existing literature that is relevant to the automatic off-line design, and provide guidelines that we deem necessary for a healthy development of the domain and for ensuring its relevance to potential real-world applications.

Although swarm robotics is widely recognized as a promising approach to coordinating large groups of robots (Dorigo et al., 2014;Yang et al., 2018) and has already gained a prominent position in the scientific literature (e.g., see Rubenstein et al., 2014;Werfel et al., 2014;Garattoni and Birattari, 2018;Slavkov et al., 2018;Yu et al., 2018;Li et al., 2019;Xie et al., 2019), a general methodology for designing collective behaviors for robot swarms is still missing (Brambilla et al., 2013). The design problem is particularly challenging because it aims at producing a system that is autonomous, fully distributed, and highly redundant: robots do not have any predefined role and do not rely on any external infrastructure (Beni, 2004;Şahin, 2004). A robot swarm is a loosely coupled system in which the collective behavior of the system results from the local interactions between individuals, and between them and the environment. These interactions cannot be explicitly defined at design time due to the high uncertainty that characterizes the operation of a swarm. As a result, at least in the general case, it is impossible to tell what the individual robots should do so that a desired collective behavior is achieved. This rules out the application of traditional multirobot systems and software engineering techniques, which rely on formally deriving the individual behaviors of the robots from specifications expressed at the collective level (Brugali, 2007;Di Ruscio et al., 2014;Bozhinoski et al., 2015;Schlegel et al., 2015).
This paper is intended to be the manifesto of the automatic off-line design of robot swarms. In this approach, the design problem is cast into an optimization problem that is solved off-line-that is, before the swarm is deployed in the target environment. An optimization algorithm searches a space of possible designs with the goal of maximizing an appropriate mission-specific performance measure. Within the design process, the performance of candidate designs explored by the optimization algorithm is assessed via computer-based simulations. Once the optimization algorithm terminates, the selected design is uploaded to the robots and the swarm is deployed in its target environment. In the following, we focus mostly on the development of software but the discussion can be directly extended to the hardware. For example, the automatic off-line design process might optimize the number of robots in the swarm; if the swarm is heterogeneous, select the fraction of robots of type A, B, C,. . . ; fine-tune parameters of hardware or firmware; activate/deactivate or add/remove hardware modules; design chassis, shell, or attachments.
Our vision is that, in a relatively close future, automatic offline design will be a practically relevant way of realizing robot swarms. Likely, it will not be the only one: other approaches will be available, each with its specific advantages and ideal areas of application, as well as its disadvantages and limitations. Among them, we foresee that a relevant role will be played by manual design, semi-automatic design, automatic on-line design, and hybrid approaches that combine the previous ones. Nonetheless, we expect that the automatic off-line approach will play a major role, both on its own and also as a component of hybrid approaches.
In the automatic off-line approach, robot swarms are generated to perform missions that are sampled from a given class of interest and are sufficiently different from one another to possibly require (or benefit from) a tailored design. An automatic off-line method must operate on the missions of the given class without requiring either mission-specific adjustments, or per-mission human intervention. The notion of a class of missions plays here a central role. It refers to a set of missions, together with a probability measure defined on them, which determines their relative frequency of appearance. Typically, an explicit, closed-form definition of the set of missions and of the probability measure is not available-and is not even needed. Instead, what we have is a stream of missions sampled from the class of interest according to the aforementioned probability measure. The assumption that missions are sampled according to a probability measure gives a formal meaning to the notion of expected performance, as well as to any other statistics one might wish to adopt to describe the aggregate behavior of a design method across the missions of interest. To illustrate the automatic off-line design of robot swarms, it is convenient to sketch one of its possible practical applications.

Fiorella's swarm gardening
(for an artist's rendition, see Figure 1) Fiorella owns a robot-swarm gardening business and offers her individually-tailored service to her many customers in the Brussels area. She has a busy schedule: every day, she visits three or four customers with her gardening swarm. Customers book Fiorella's service via a form on her website. Through the form, they ask for one or more specific interventions-e.g., cutting grass, watering flowers. They also provide relevant information on their gardene.g., size, shape, orientation. The interventions requested and the characteristics of the garden specify the mission that Fiorella's swarm must perform for a specific customer. As the list of possible interventions and characteristics of the garden is huge, the class of possible missions is overwhelmingly large and rather diverse. To provide her customers with the best gardening experience-but also to cut costs and maximize her benefit-Fiorella relies on an automatic offline method that designs and fine-tunes the behaviors of her swarm specifically for each mission. The design process takes place while Fiorella drives her swarm to the customer's garden: her powerful computers run simulations using the information provided by the customer on the interventions and on the garden. The design process must be performed within a limited amount of time-the time of the ride to the customer's. As Fiorella arrives on the spot, the selected design is uploaded to the robots and the swarm is deployed in the garden. Fiorella cannot intervene in the design processshe drives the van in the meantime. Moreover, due to her tight schedule, Fiorella cannot either test the selected design on the robots before deployment and possibly re-run the design process: once she reaches the customer, robots must be operational. Any per-mission human intervention and any test on the robots in the target environment would be too time consuming and expensive: they would increase costs dramatically and Fiorella would be unavoidably out of business.
Missions in the class of interest can be relatively minor variations of each other-e.g., cut the grass in a small garden; in a large one; in one with a central flower bed. In this case, the behaviors to be produced will be similar, with some minor differences to increase performance or reduce execution time. Missions can also be substantially different in the nature of their goals and require major differences in the behaviors to be realized-e.g., cut the grass; gather dead leaves in a specific place; locate and map mole tunnels.
In Fiorella's example, the central role of the notion of class of missions emerges clearly. Fiorella faces a stream of missions sampled from the possible missions for which customers might demand her intervention-and which her swarm can hopefully accomplish. It is in the repetitive nature of the design problems faced by Fiorella that the significance of automatic design lies. Indeed, if Fiorella had to solve a single design problem (instead of a stream thereof) she could more profitably solve it either manually or via a semi-automatic design method 1 . It is only when one has to solve a stream of design problems that the human intervention might become uneconomical or even unfeasible. FIGURE 1 | Fiorella's swarm gardening. Fiorella's robots can perform a large class of gardening missions. Through her website, customers book Fiorella's services, specify the interventions to be performed, and provide a description of their garden. On the basis of this information, while Fiorella drives her robots to customers, her algorithms automatically design and fine-tune the behavior of the robots so as to offer a tailored service. When she arrives at a customer location, the gardening swarm is operational and immediately deployed.
Conceiving, implementing, and setting up an automatic design method is in itself an investment of time and resources, which pays off only if the design process is then repeated a sufficient number of times on multiple missions-those of the class for which the automatic design method is conceived. If one had to address a single mission, it would be more sensible to invest time and resources on that specific mission-by adopting an adhoc manual or semi-automatic approach-rather than on the development of an automatic design method that would be then used only once. For a schematic representation of the automatic off-line design process, see Figure 2.
Fiorella's example allows us to highlight a number of issues and research questions that are relevant to the automatic off-line design of robot swarms. Can effective robot swarms be designed automatically via an off-line process? Can we conceive a method to automatically design a swarm for any mission within a given class? What is the class of missions for which a given method can design an appropriate swarm? How can a given method be generalized to solve a larger class of missions?
Given a class of missions, which is the most appropriate design method? What elements or characteristics of a design method influence its ability to handle missions of a given class? Vice versa, what elements/characteristics of a class of missions might suggest that a given method is appropriate to handle them? Which features of a specific mission make it challenging for a given design method? Are these challenging features equally challenging for all possible design methods? Is it possible to match challenging features of missions with characteristics of design methods?
To what extent a given design method is robust to the so called reality gap-that is, the difference between simulation models and reality? Is it possible to predict the performance drop that a swarm designed off-line will experience when deployed in the target environment? Are different design methods equally sensitive to the reality gap? What elements/characteristics of a design method make it more or less robust to the reality gap? Can these characteristics be leveraged to engineer a design method that is inherently robust to the reality gap? How should models be devised to be effectively used within an off-line design process?

How efficient is a design method?
In other terms, how many off-line simulation runs are required to produce an effective design? What elements/characteristics of a design method increase/decrease its efficiency? How well does a given design method behave for a large/small design budget-that is, when allowed to perform few/many off-line simulation runs? Does the efficiency of a design method depend on the specific mission or class of missions considered? What elements/characteristics of a mission determine the minimum size of the design budget needed to produce an effective design? When should a design process be stopped?
This list of questions encompasses many relevant issues but it is by no means exhaustive. For example, other relevant issues would concern the off-line definition of the swarm size (or its spatial density), its impact on performance, and the robustness/scalability of behaviors that are automatically designed. A body of literature exists that is relevant to the automatic off-line design of robot swarms. The largest share of the design methods described in the relevant literature belong in the neuro-evolutionary domain (Nolfi and Floreano, 2000): robots are controlled by a neural network whose synaptic weights (and possibly the topology) are optimized by an evolutionary algorithm (Quinn et al., 2003;Christensen and Dorigo, 2006;Baldassarre et al., 2007;Trianni, 2008;Hauert et al., 2009;Trianni and Nolfi, 2009;Waibel et al., 2009;Ferrante et al., 2013Ferrante et al., , 2015Gomes et al., 2013;Trianni and López-Ibáñez, 2015). For a review of the neuro-evolutionary approach (including also single-robot studies) see Floreano and Keller (2010), Bongard (2013), Bongard and Lipson (2014), Trianni (2014), Doncieux et al. (2015), and Silva et al. (2016). Other approaches depart from neuro-evolution as (i) robots are controlled by software architectures other than FIGURE 2 | Flowchart diagram of the automatic off-line design process. A mission is sampled from a class of interest. Using computer-based simulations, an automatic design method defines a robot swarm tailored to the mission sampled. Once the automatic design terminates, the swarm is deployed in the target environment and has to cross the so-called reality gap-the possibly subtle but inevitable difference between simulation and reality (Brooks, 1992;Jakobi et al., 1995)-which is among the most challenging issues in automatic off-line design. The process is repeated ad libitum. It should be noted that an automatic design method could generate a robot swarm from scratch for every mission sampled or could refine and adapt a solution previously generated for a similar one. It could also, for example, produce a robot swarm by combining and modifying solutions (or partial solutions) contained in a catalog of template solutions that were pre-defined by a human expert. The only condition that needs to be respected for the process to be qualified as automatic off-line design is that such initial (partial) solutions must be selected without any per-mission human intervention and without recourse to tests performed in the target environment.
It is our contention that, with only few exceptions, the aforementioned methods have been studied following protocols that were not conceived to directly address the core research questions sketched above. Although these protocols allowed studies which partially addressed those questions, they were conceived to target other questions that are mostly relevant to other domains including, for example, evolutionary biology and the semi-automatic design of robot swarms 1 . In almost the totality of the studies, the focus is on a specific mission that must be performed by a swarm-or, equivalently, on a specific capability that the swarm should acquire and display. The design method is proposed only as a way to achieve the desired collective behavior and is not the protagonist of the study: the study is not structured to highlight its properties and assess its performance. The design method has so little importance that it is not customarily given an identifying namecontrary to what happens in related fields such as machine learning or heuristic optimization. Typically, the design method is tested on a single mission and it is not compared to any alternative. It is rare that a same design method is tested across multiple studies on multiple missions without undergoing any (manually-applied) mission-specific modification. In many studies, the control software produced by a design method is tested only in simulation and no assessment is provided on whether and to what extent it crosses the reality gap satisfactorily. Moreover, design methods survive only the time span of the paper in which they are introduced and their implementation is not routinely made publicly available for further studies, to be possibly performed by a third party. Often, a design method is run iteratively on a single mission. It is run once, the behavior generated is inspected by the designer who then modifies the method itself or the objective function to be optimized-e.g., by adding/removing terms. These activities are then iterated at will until a satisfactory behavior is obtained. In most cases, this iterative process is not detailed in research articles: it is often unclear how many iterations have been performed, what has been measured at each iteration, what modifications have been implemented, what ideas have been tried and then abandoned. In these cases, the research articles present only the final setting that eventually generated the behavior discussed. The iterative process is repeated only once, as it would be difficult to produce independent trials of a process that features a human in the loop. As a result, the robustness and the repeatability of the process are not assessed.
An appropriate protocol to address the aforementioned issues should reflect the following tenets of the research in automatic off-line design: (i) automatic off-line design methods should not be mission-specific and should be able to address a whole class of missions without undergoing any modification; (ii) once a mission is specified, human intervention is not provided for in any phase of the design process. Indeed, research that is intended to be relevant to the automatic off-line design of robot swarms should exclude the case in which design methods are conceived for or are manually adapted to a specific mission-for example, by manually tuning parameters of the optimization algorithm and/or of the control architecture, or by pre-filtering sensor readings on the basis of insight that only a human designer can provide. It should also exclude the case in which, on a per-mission basis, human designers are allowed to inspect (via either simulation or robot experiments) the behavior of an automatically designed swarm and, on the basis of their observations, modify elements of the automatic design process and iterate it at will, until they obtain satisfactory results.
In particular, human designers should not be allowed, on a per-mission basis, to use any insight gained through inspection to modify the design method (optimization algorithm, architecture, sensor pre-filtering, etc.); to adapt simulation models; and to amend the objective function by adding/removing terms so as to steer the design process as wished. On the other hand, to effectively contribute to the development of the domain, researchers in the automatic off-line design of robot swarms should pay particular attention to a number of methodological issues. In particular, they should: (a) provide a clear and thorough description of the design methods they propose, including a list of the value of all parameters; (b) precisely characterize the platforms for which these methods can generate control software; (c) clearly identify and name methods for future reference; (d) publish implementations; (e) test methods on multiple missions; (f) identify-at least informally-the class of missions that a method is intended to address; (g) perform comparative studies in which methods under analysis are tested under the same conditions; and (h) run robot experiments to assess robustness to the reality gap. It is our contention that this minimal set of guidelines will allow the domain to grow healthy and thriving so as to eventually prove its practical relevance in real-world applications.

AUTHOR CONTRIBUTIONS
All authors contributed to the elaboration of the ideas presented in the paper, read the text, and provided comments. In particular, MBi started the discussion and lead it, he also drafted the paper and coordinated its revision. AL selected/reviewed the relevant literature and contributed to the formulation of the research questions. DB contributed to framing the complexity of the design problem in swarm robotics. The other authors equally contributed to this paper and are listed in alphabetical order.

FUNDING
The project has received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreement No 681872). MBi, JK, and TS acknowledge support from the Belgian Fonds de la Recherche Scientifique -FNRS. DR acknowledges support from the Colombian Administrative Department of Science, Technology and Innovation -COLCIENCIAS. AR acknowledges the support of the Chaire internationale programme of Université libre de Bruxelles.