ORIGINAL RESEARCH article
Sec. Social Physics
Volume 10 - 2022 | https://doi.org/10.3389/fphy.2022.1019422
Exploring norms indispensable for both emergence and maintenance of cooperation in indirect reciprocity
- 1Faculty of Business Administration, Rissho University, Tokyo, Japan
- 2Faculty of Business Administration, Soka University, Tokyo, Japan
- 3Research Center for Ethi-Culture Studies, RINRI Institute, Tokyo, Japan
- 4Department of Community Development, Koriyama Women’s College, Koriyama, Japan
Indirect reciprocity is one of the major mechanisms of the evolution of human cooperation. In indirect reciprocity, social norms with which individuals distinguish good people from bad people play essential roles. On the one hand, previous studies have suggested the various different norms which are evolutionarily stable against the invasion of free riders. However, these approaches could not reveal what norms would be selected in the process of evolution of cooperation because they are based on the premise that a single norm is shared in a society. On the other hand, recent studies have tackled the mechanisms of the coevolution of norms and cooperation. However, the norms which are necessary for emergence or sustenance of cooperation have not been revealed. Here, we show some indispensable norms for emergence and sustenance of cooperation using a norm knockout method which has been developed to analyze a function of each norm in the environment of coexistence of many norms. The results revealed that norms known as “shunning” and “image scoring” are indispensable in the emergence of cooperation but they are not required after a cooperative society is achieved. Furthermore, “simple standing” is a unique norm which is necessary to maintain cooperation. We call the former as a pioneer norm and the latter as a keystone norm. The results indicate importance to focus on the dynamics of evolution because the role of indispensable norms has been overlooked by the static analysis of evolutionarily stable norms.
As a proverb says that one good turn deserves another, the evolution of a costly cooperative behavior can be understood as a mechanism in which a positive evaluation of the present cooperation is shared with others and the cooperating person receives cooperation from a third person at a future time. This mechanism is generally referred to as indirect reciprocity and numerous studies have addressed this mechanism [1, 5, 19, 23, 30]. In order for the aforementioned mechanism to work as the basis of cooperation, it is necessary for people to have norms that effectively distinguish good people from bad people and to have a tendency cooperating only toward the good people. In previous studies of indirect reciprocity, researchers have focused on several pro-social norms, such as “cooperation is good and defection is bad [6, 12]” (image scoring, “IS”), “cooperating toward bad person is bad and defecting toward bad person is good [3, 16]” (stern judging, “SJ”), and “cooperation even toward bad person is good and defection toward bad person is also good [2, 6, 18]” (simple standing, “SS”).
Most theoretical studies on indirect reciprocity have explored norms that are evolutionally stable. Theoretically, assuming that the same norm is adopted by all members of a population, Ohtsuki and Iwasa [11, 14] have shown that only eight out of 4,096 norms can stabilize cooperation. Here, “stable” means that the corresponding population which adopts the same norm cannot be invaded by other action rules. Although their approach allows for identifying norms that can sustain cooperation, it cannot answer what norms will evolve in a mixed situation of various norms. Additionally, in the empirical approaches, it has been explored what kind of norms people have in an indirect reciprocal situation [4, 9, 32]. The earlier approaches have pointed out that image scoring [6, 12] to judge whether people cooperate by past actions of others has been adopted. In recent years, more detailed experiments have been conducted and it has been found that the information referred to by individuals is varied [15, 16]. However, few studies have elucidated how norms are accepted by groups.
To reveal the evolution of cooperation in an environment in which various norms coexist as in a real society, competition among multiple norms must be taken into consideration. By coexisting of norms originally shared in each group or society, the conflict of norms has been causing fatal social problems, for example, pro-social behavior seen from one side is judged as anti-social behavior from the other and then they intensely conflict with each other. Especially in recent society, interaction between communities with divergent values is becoming more active and members are coming and going more intensely. Popularization of the Internet and becoming borderless among countries has accelerated this trend. The issue on how to construct a cooperative society in an environment in which diverse norms coexist and sometimes conflict is an urgent problem in modern society. Therefore, it is necessary to model the environment in which multiple norms coexist and compete in society as a norm ecosystem and clarify the coevolutionary process of norms and cooperation.
Studies focusing on competition among multiple norms have been limited to those dealing with competition among a few norms [17, 26]. On the other hand, recent other studies have also revealed the mechanism which enables to evolve cooperation in an environment where many norms coexist . Systems where many norms coexist are complex in their interactions, making it difficult to understand the characteristics of individual norms. A previous study  proposed a new method for analyzing the role of a particular norm. The method removes a particular norm from a group in which multiple norms coexist and observe the group’s behavior to estimate the norm’s role. The method is called a norm knockout method, a variation on a technique known as gene knockout, which is used in genetic engineering.
In the norm ecosystem, two perspectives are needed to analyze the process of cooperation evolving and stabilizing. One is whether indispensable norms exist to evolve cooperative behavior from a coexistence state of norms or a state where defection is dominant. A previous study  has focused on this point. In the ecosystem, there is a species, called as a pioneer species, which first takes root in a devastated environment and contributes to the development of the subsequent ecosystem . For example, plants that initially invade lands covered with lava by volcanoes are typical pioneer species. If this species does not exist, it is difficult for ecosystems such as forests to grow. Metaphorically applying pioneer species in the norm ecosystem, the previous study  can be regarded as a discovery of pioneer norms.
The other is whether indispensable norms exist to maintain that state after the cooperative society has been achieved. Little is known about norms indispensable for stabilizing cooperation in competition among various norms. In the ecosystem, there is a biological species called as a keystone species. This species greatly affects the ecosystem despite not being in a significant position in the food chain [10, 22]. A keystone species refers to a species that, when removed from a stable ecosystem, collapses ecosystems. For instance, sea otters in the kelp forest of the sea and beavers building dams in rivers are typical keystone species. In this study, similar to the relationship between the pioneer species and the pioneer norms, we focus on the indispensable norms to maintain cooperation after the cooperative society has been achieved. In other words, this study reveals a keystone norm in the norm ecosystem. In addition, by comparing the pioneer and keystone norms, we clarify what role various norms play in the evolution and stability of cooperation.
In this study, we model finite individuals who play as donors and recipients in the giving game with no spatial structures. The group size of the model is N. The overview of the giving game is as follows. 1) One donor (i) and one recipient (j) are chosen at random from the population and they engage in a giving game. 2) The donor decides whether to cooperate with the recipient at a personal cost c (>0). If the donor chooses to cooperate, the recipient receives a benefit b > c; otherwise, the recipient obtains nothing. The donor chooses cooperation or defection with a recipient using an image that the donor has to the recipient. An image is either Good or Bad. If a donor’s image to a recipient is Good, the donor cooperates with the recipient and if the image of the recipient is Bad, the donor chooses defection. 3) Subsequently, N − 1 agents other than the donor, as observers, evaluate and update images of the donor using their own norms. Each agent independently has images of other agents, and thus images from each agent to others can be represented as an image vector. These image vectors form an N × N matrix, A. Every element of A is either Good or Bad. All agents consider themselves as Good, and thus all diagonal elements of A are set to Good constantly. Observer k updates donor i’s image (Aki) depending on an action of donor i (C/D), an image of recipient j from viewpoint of observer k (Akj) and a norm observer k adopts. In the case of the recipient being an observer (j = k), the recipient updates the donor’s image using his/her norm as well as those of other observers’ cases. Note that any updated donor’s images in the eyes of different observers are not necessarily same. Figure 1 shows the overview illustration of the game.
In this study, the norm of an agent is denoted as one of four possible “assessment combinations” and there are two possible “alleles (Good/Bad)” at the “locus” for each of the four assessment combinations. This combination of Good/Bad is the norm held by the agent. There is a total of 24 = 16 possible norms. The first locus of the gene represents an assessment rule for an agent who cooperates with a good recipient. The second locus represents an assessment rule for an agent who cooperates with a bad recipient. The third locus represents an assessment rule for an agent who defects with a good recipient. The fourth locus represents an assessment rule for an agent who defects with a bad recipient. The new image to i from k’s viewpoint depends on i’s action (C/D) and depends on the image to the recipient j from the viewpoint of k (Good/Bad). Table 1 shows an example of a typical norm which have pro-social characteristics. SH evaluates a donor as Good only when the donor cooperates with a good recipient. In other words, a donor who is matched with a bad recipient will consistently be rated Bad. SJ evaluates cooperation with a bad recipient (unjustified cooperation) as Bad and defection against a bad recipient (justified defection) as Good. IS evaluates cooperation as Good and defection as Bad, regardless of whether the recipient is Good or Bad. SS evaluates only defection against good recipients as Bad and all other cases as Good. These norms have two common characteristics, cooperation with a good recipient is Good, and defection against a good recipient is Bad.
Each simulation runs throughout G generations. Each agent plays the giving game R times as donor in each generation. At the beginning of each generation, an image of each agent from all of other agents (including themselves) is set to Good. Each player evolves one’s norm Si using a pseudo-genetic algorithm  at the end of each generation. The pseudo-genetic algorithm adopted in this study follows an evolutionary-computation method that represents norms as sequence of numbers (genes) and applies genetic operations such as selection, mutation, and crossover to obtain a norm that has a high degree of fitness. In the model, fitness is represented as the accumulated payoff in a generation. In the process of evolution, each agent randomly selects two agents from N agents (including itself) to become its parents. For choosing parents, we adopt a roulette selection method. This roulette selection sets a probability distribution of all agents as
Each agent has two different types of errors: one, the probability that the agent’s updating of its evaluation of others, Good/Bad, is inverted (errors in perception), described as p, and two, the probability to perform an action different from the one prescribed by its action rule (errors in implementation), described as q. The implementation error is bilateral. State variables and initialization in the simulation are shown in Table 2. Generation G is set to 1,000 in the analyses of pioneer norms. Also, G is set to 2,000 in the analyses of keystone norms and the norm knockout method for a focal norm is executed at the 1,000th generation. The reason for executing the norm knockout at the 1,000th generation is that a cooperative regime is achieved and stabilized in all replications (50 replications) at the 1000th generation. We can observe the dynamics after the focal norm is removed from a cooperative society.
The norm knockout method is implemented as follows. When we knockout a particular norm, that norm is removed in the first round of each generation. Concretely, if the norm of an agent evolves into a norm that is knocked out as a result of the adopting process, the norm of the agent is changed to one of the other 15 norms randomly. In other words, the norm that is knocked out will never exist at all in a society.
The analysis of pioneer norms is executed as follows. The norm knockout method is used in the first generation. As a result, there are 15 norms except for the knocked-out norm in the population. This operation enables us to observe which norms are indispensable for the evolution of cooperation from the melting pot of norms. On the other hand, the analysis of keystone norms is executed as follows. In the first generation of the simulation, all the possible norms exist in the population. Cooperation can evolve in the situation which exists all 16 norms . After cooperation is achieved and kept stable, at the 1,000th generation, we adopt the norm knockout method. This operation allows us to observe whether cooperation could be sustained when the focal norm is removed from the population.
The purpose of this study is to clarify the keystone norms. For this purpose, we first discuss the character of keystone norms by conducting a pioneer norm analysis found in previous studies  and comparing it to a keystone norm analysis. Figure 2 shows the results of pioneer norms analysis and keystone norms analysis. In the results of the pioneer norms analysis, GBBB(SH), GGBB(IS), and GGBG (SS) are pioneer norms as same as in the previous study. Cooperation cannot evolve in an environment in which these norms do not exist. On the other hand, the results of the keystone norms analysis show that cooperation collapsed only when GGBG (SS) was knocked out. In other words, a keystone norm is only SS. SH and IS are pioneer norms but not keystone norms.
FIGURE 2. Results of pioneer norms analysis and keystone norms analysis. It shows the average cooperation ratio in the final generation when a representative norm was knocked out. The number of trials is 50 and error bars represent standard deviation. In pioneer norms analysis, when SH, IS, and SS are knocked out, cooperation does not evolve, whereas, in keystone norms analysis, only SS is necessary to maintain cooperation and cooperation is maintained even if SH and IS are knocked out. In addition, there is no effect even if IS is removed from a cooperative society, but cooperation also cannot evolve in the environment in which IS does not exist from the beginning.
As shown in Figure 3A, we reveal that cooperation cannot evolve when SH is knocked out in the pioneer norms analysis. In an environment in which SH does not exist, cooperation never evolves because defection is dominant at the beginning and no other norm can invade BBBB(ALLB). On the other hand, as shown in Figure 3B, cooperation is stable in the results of the keystone norms analysis. Although SH is knocked out at 1,000 generations, cooperation maintains robustly in a coexisted situation of four norms of GGGG (ALLG), GGGB, GGBG (SS), and GGBB(IS) which are cooperative norms.
FIGURE 3. Pioneer norms analysis (A) and keystone norm analysis (B) regarding SH. The x-axis shows generations and the y-axis shows the fraction of norms and cooperation ratio. The solid line represents population of each norm and the black dotted line represents cooperation ratio. In pioneer norms analysis, SH is knocked out at 0 generations and in keystone norms analysis, SH is knocked out at 1,000 generations. In pioneer norms analysis, cooperation does not evolve because SH is knocked out and also no other norm can invade ALLB. After cooperation evolves, SH cannot exist. Therefore, even if SH is knocked out in keystone norms analysis, cooperation is maintained stably.
On the other hand, as shown in Figures 4A,B, when SS is knocked out, cooperative society cannot realize in both pioneer norms analysis and keystone norms analysis. In pioneer norms analysis, when SS is knocked out from the beginning of the simulation, although SH is dominant over ALLB, cooperation cannot evolve from that state to a stable situation because SH assesses only cooperation toward good agents as Good, it does not cooperate by assessing all other things as Bad when other norms cooperate or defect toward bad agents. Furthermore, SH does not cooperate with other SH agents who have failed to cooperate because of errors. As a result, mutual defection among SH increases and cooperation never evolves among SH. Although SJ and IS exist, these norms are also vulnerable to errors and cannot maintain a stable cooperative state.
FIGURE 4. Pioneer norms analysis (A) and keystone norms analysis (B) regarding SS. The x-axis shows generations and the y-axis shows the fraction of norms and cooperation ratio. In pioneer norms analysis, SS is knocked out at 0 generations and in keystone norms analysis, SS is knocked out at 1,000 generations. When SS is knocked out in both pioneer norms analysis and keystone norms analysis, the cooperative state is not realized.
Additionally, in keystone norms analysis, even when SS is knocked out in a state in which cooperation is stable, cooperation collapses. ALLG is vulnerable to the invasion of defective norms such as ALLB because it cooperates unconditionally. GGGB is a norm that assesses defection toward good agents as Good, and it cannot contribute to maintaining cooperation. Since IS is vulnerable to errors as aforementioned, there is no other norm that can maintain cooperation in the absence of SS.
Subsequently, we analyze the cases where all 16 kinds of norms including BBBB and GGGG were knocked out. Table 3 shows the results of pioneer norms analysis and keystone norms analysis in the cases where all 16 kinds of norms were knocked out. In pioneer norms analysis, we can clarify that SH, IS, and SS are pioneer norms. In keystone norms analysis, we can clarify that only SS is a keystone norm. Although GGBB(IS), GGGB, and GGGG (ALLG) coexist with GGBG (SS) in a cooperative society, a cooperative society never collapses even if other than SS are knocked out. In other words, when we observe the cooperative society obtained as a result of evolution, the coexistence of these four kinds of norms is observed but only one norm is necessary for maintaining cooperation. Also, when we observe cooperative society, SH does not exist. However, cooperative society is not realized in an environment in which SH does not exist from the beginning.
TABLE 3. Average cooperation ratios at the final generation. Figures between parentheses indicate standard deviations. It is highlighted in red that the cooperation ratio falls below 0.2. It shows that average of cooperation ratio after 1,000 generations in pioneer norms analysis and average of cooperation ratio after 2000 generations in keystone norms analysis. In both the analyses, the number of replications is 50.
We analyzed the norms necessary for cooperation based on indirect reciprocity to evolve. In the previous research studies, because they had a premise that a single norm is shared in a society, it was not possible to analyze the mechanism which enables to evolve cooperation in environments with various norms coexisted. Therefore, we developed the model where various norms coexist in a society as a norm ecosystem and analyzed the evolutionary mechanism of cooperation by agent-based simulation.
Applying the norm knockout method , we revealed two types of norms which are indispensable to evolve or maintain cooperation. One is the norm that is indispensable for cooperation to evolve from an environment in which defection is dominant and the other is the norm that is indispensable to maintain cooperation in an environment in which cooperation is dominant. SH, IS, and SS correspond to the former and SS corresponds to the latter. It is worth noting that SS is an indispensable norm in both cases. Some previous studies have revealed that SS can generate high levels of cooperation [8, 13, 25]. These results indicate that SS might be the keystone norm. On the other hand, a recent study has suggested that SS has problems when defectors are predominant . However, no mechanism has been established to explain why SS can play an indispensable role in both defective and cooperative regimes. Understanding specific behavior and the relationship among multiple norms remain for future studies.
Although the two norms, especially SH and IS, are indispensable to emerge and evolve cooperative behaviors in a society in which defection is dominant, they are unable to survive or only a small number can exist in a society in which cooperation is stable , and these norms are called as unsung hero norms. Surprisingly, the keystone norms analysis revealed that unsung hero norms do not affect cooperation in a society in which cooperation is stable even if it is completely removed.
This is thought to give important suggestions for maintaining a cooperative social system. In other words, SH and IS are norms that do not contribute to a society when they are observed in a state in which cooperation is stable. Therefore, these norms may be excluded as not being adaptive. However, when a society becomes a defective state, it is impossible to rebuild a cooperative society without these norms. By analyzing the norm ecosystem, it was suggested that maintaining the diversity of norms in a society has an important meaning for a stable and sustainable cooperative society. Moreover, SJ was found to be neither a pioneer norm nor a keystone norm. In the study of  an environment with no errors, SJ temporarily accounted for a majority in the evolutionary process of cooperation and played an important role in a transition from a defective society to a cooperative society. However, it has been pointed out that SJ is vulnerable to errors  and cannot play an important role in the evolution and maintenance of cooperation in environments with both errors and coexistence of diverse norms because SJ assesses cooperation toward bad (pampering) as Bad and defection toward bad (punishment) as Good, it is a norm that uses secondary information most precisely. The role of SJ in the norm ecosystem is an issue to be explored in the future.
In this study, we examined the norm that combines two types of information: donors’ behaviors and recipients’ reputations. On the other hand, in the experimental studies, it has been discussed whether people actually use two types of information [4, 9, 25]. Although the information processing cost is ignored in this model when evaluating others, we need to analyze a more realistic norm ecosystem by introducing costs to the use of reputation as future development. Furthermore, models that introduce structures into the scope of interaction and learning have also been proposed; however, research studies have been limited to the studies dealing only with image scoring [7, 29] and to the models in which only three of pure defection, pure cooperation, and target norms exist in spite of dealing with multiple norms . It is also a future issue to expand a norm ecosystem on a complex network. This study deals only with donor behavior (first-order information) and recipient image (second order information). However, some literature deals with donor image (third-order information)  or higher-order information . A norm ecosystem that adopts higher-order information should be considered in the future.
Data availability statement
The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.
HY initiated and performed the project. HY, IO, SU, and TS designed the project and approved the submission. HY, IO, SU, and TS wrote the manuscript. All authors reviewed the manuscript.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors, and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
19. Uchida S, Yamamoto H, Okada I, Sasaki T. A theoretical approach to norm ecosystems : Two adaptive architectures of indirect reciprocity show different paths to the evolution of cooperation. Front Phys (2018) 6:14. doi:10.3389/fphy.2018.00014
27. Hilbe C, Schmid L, Tkadlec J, Chatterjee K, Nowak MA. Indirect reciprocity with private, noisy, and incomplete information. Proc Natl Acad Sci U S A (2018) 115:12241–6. doi:10.1073/pnas.1810565115
30. Liu A, Wang L, Zhang Y, Sun C. Coevolution of cooperation and complex networks via indirect reciprocity. In: Liu D, Xie S, Li Y, Zhao D, and El-Alfy E-SM, editors. Neural information processing. Cham: Springer International Publishing (2017). p. 919–26.
Keywords: indirect reciprocity, evolution of cooperation, game theory, social norms, pioneer and keystone species
Citation: Yamamoto H, Okada I, Uchida S and Sasaki T (2022) Exploring norms indispensable for both emergence and maintenance of cooperation in indirect reciprocity. Front. Phys. 10:1019422. doi: 10.3389/fphy.2022.1019422
Received: 15 August 2022; Accepted: 12 September 2022;
Published: 30 September 2022.
Edited by:Setsuya Kurahashi, University of Tsukuba, Japan
Reviewed by:Zhen Wang, Hangzhou Dianzi University, China
Francisco Welington Lima, Federal University of Piauí, Brazil
Copyright © 2022 Yamamoto, Okada, Uchida and Sasaki. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Hitoshi Yamamoto, email@example.com