Module Athens TP-09

 jean-louis Dessalles - Associate professor at Telecom-Paris Julien Lie-Panis - final year PhD student at Telecom-Paris and Institut Jean Nicod

# Evolution of Cooperation

This Lab work is based on the use of Evolife. If necessary, install it.

## The Green Beard effect

This small experiment was popularized by Richard Dawkins in his book The selfish gene. It shows how some interesting coupling between genes may emerge in a population.

Suppose a given gene has two effects on its carriers:
• (1) they get a conspicuous characteristic, like a green beard.
• (2) they tend to behave altruistically toward green-bearded individuals.
Load the configuration GreenBeard.evo in the Configuration Editor. The scenario offers two parameters: GB_Gift and GB_Cost that you may use to control individuals’ altruistic behaviour.

Open the file S_GreenBeard.py (in the Scenarii directory) and implement the function interaction so that green bearded individuals (those whose gene named GreenBeard has value 1) behave altruistically toward their green-bearded fellows by helping them: they give them life points (through the score function) at their own expense (they lose points, but possibly less, as controlled by the GB_Cost coefficient). To do so, you need to use functions such as:
• indiv.score(bonus) (defined in Individual.py; if score is omitted, merely reads the individual’s current score). bonus may be positive or negative.
• indiv.gene_value(GreenBeard) (defined in Genome.py; returns the gene’s value).
The values of parameters may be accessed from the scenario through:

self.Parameter('GB_Cost')

 Implement the interaction function, using only the GreenBeard gene. Copy the relevant lines of your function in the box below.

Observe that the GreenBeard gene invades the population. To understand the phenomenon, one must look at genes rather than at individuals. On average, the payoff of the GB+ allele is:

where p is the probability of GB-GB encounter. The average payoff of the GB- allele is 0.
 Try values for which the cost is slightly larger than the benefit. You should observe that the GreenBeard gene does NOT drop down to 0! (you may turn the selection method to ‘selection pressure’ for comparison). Why is it so?

Because in the ‘selection pressure’ condition, parenthood is no longer biased in favour of the GreenBeard gene
Because in the ‘selection pressure’ condition, GreenBeard carriers accumulate negative scores
Because in the ‘selectivity’ condition, some lucky GreenBeard carriers get positive scores

The GreenBeard gene happens to have two coupled effects:
• appearance (green beard) and
• altruism (toward other green-bearded individuals).

 Split the two effects by using the second gene Altruist. You read the value of that gene by calling:     indiv.gene_value('Altruist') Copy the relevant lines of your function in the box below.

If you made appropriate changes, appearance is now controlled by GreenBeard, whereas the altruistic behaviour is controlled by Altruist. So whenever an agent that has the GreenBeard gene set to 1 meets another GreenBeard carrier, it checks for the value of its own Altruist gene to decide whether it will help its fellow.
 What happens? Why?

#### Suggestions for further work

You may play with the idea.
• For instance, in the initial situation where the GreenBeard gene has the two effects, you may use a second gene (name it Nasty instead of Altruist) that prompts its carriers to attack green bearded individuals at their own expense (i.e. both lose points). It becomes good and bad to carry a green beard. The new gene may check (or not) that its carrier has a green beard or not.
• Keep the idea that ‘green beards’ help each other only if they are ‘altruists’. Now mimic a brotherhood effect by implementing a collective benefit: ‘altruists’ get a bonus divided among their kind. For certain values of the parameters, oscillation may emerge.

## Delayed cooperation

Launch Evolife from the Evolife directory with starter. Load the Cooperation.evo configuration. Let the program run (button [Run]). In the meantime, examine the S_Cooperation.py scenario (in the directory Scenarii) to see how this basic version of cooperation is implemented. Cooperative behavior is controlled by two genes: Cooperativeness and Exploration. The former controls the amount of wealth you give (at your own expense) to partners during encounters. Cooperative acts are rewarded, because if your gift was large enough for partners to remember you, they will choose you as partner when they are in position to make a gift themselves. However, it is in the interest of those partners to explore the rest of the group to see if there are some more generous individuals around. The exploitation/exploration compromise is controlled by the Exploration gene.
 Observe the evolution of both genes in the long term. The meaning of curves is given in the Legend window. Wait until cooperativeness significantly drops. How would you explain the phenomenon?

Visualize social links (shortcut ‘n’; you may alternatively uncomment the default_view line in S_Cooperation.py). In this view, individuals are displayed twice, on the lower and on the upper horizontal axis. Links connect individuals on the lower axis to individuals to the upper axis. Individuals are ranked by their cooperativeness (most cooperative to the right; see function update_positions ).

 From time to time, links are directed towards most cooperative individuals exclusively. On which occasions?

In the section Genetics of the Configuration Editor, change the GeneCoding parameter to Unweighted. Now, the bits of the genes are added up to give its value. A 10 bit gene may therefore have 11 different values, from 0 to 10 (instead of 2^10 = 1024 in the weighted condition). Run the simulation (button [Run]). Is it different? Why? You may visualize genomes (shortcut ‘g').
 In the Genetics section of the Configuration Editor, set GeneLength to 5 instead of 10. Run the program again (button [Run]) and explain what happens.

[ Go to the discussion forum about cooperation ]

## Reciprocity

Reciprocal cooperation is one of the most studied problems in social sciences, economics and theoretical behavioural ecology, as it is often seen as a foundation of many social conducts. The basic scenario is as follows:
• Agent A makes the first move by offering a fraction of an amount G (gift) to agent B. This fraction is controlled by its gene g1. This gift involves a cost for A, which is proportional to the actual gift and to a constant C1.
• Agent B may reciprocate by offering a reward that is proportional to the received gift, to one of its own gene g2 and to a constant R. This rewarding behaviour has a cost, proportional to the reward and to a constant C2.
• Agents hold in memory the best partner previously encountered and try to interact with it again.
The third condition is crucial for cooperation to emerge. With amnesic players, the best strategy would be to let others make the first step and run away with their gift, as reciprocating would be mere charity. But if players can remember who does who does not reciprocate, then reciprocating is a way of being preferentially chosen as partner in the future.
A critical condition for cooperation to emerge is that:     $$\frac{G R}{C_1 C_2} > 1 + \frac{s(1)}{s’(1)}$$ (see proof there).
s(g2/g2m) is the probability for an individual with gene g2 to be selected as partner in a group in which this gene is on average g2m. The derivative s′(1) measures at equilibrium the marginal efficiency of the ability to detect cooperative agents.

This form of cooperation is still delayed (one agent must take the risk of making the first step), but this time it is the first agent who decides whether it should become friend with the partner.

Modify scenario S_Cooperation.py and implement the interaction. You need to specify two genes to control cooperation, one that controls the amplitude of the first move (g1) and another that controls the amplitude of the response (g2). The function Indiv.best_friend() returns the partner that made the best gift so far. In the interaction section, implement the symmetrical exchange and use the function Indiv.follow(Partner, PartnerOffer) that stores an asymmetrical friendship link from Indiv to Partner.

You may choose to reset the agent’s memory periodically, in the start_game function (use Indiv.detach( )). Note that partnership is not eternal anyway, due to natural death and migration. Set parameter Rounds to the number of times each individual will play the game within a season.

To control the second step, you may need to enable two additional parameters in the Configuration Editor. To do so, exit the Configuration Editor, manually edit EvolifeConfigTree.xml with your favourite editor to delete the two parasitic occurrences of the 'XXX' string and restart the Configuration Editor (starter).
 Copy the relevant lines of your function in the box below. You should still observe oscillations between the average values of the two genes. You may try to figure out why it is so.

 Models of cooperation are fundamentally unstable. In a cooperating world, there is an incentive for cheating. And yet, the human species seems to be characterized by its prosociality and generalized cooperativeness. You may share your thoughts about this apparent inconsistency.

## Emergence of a communication code

In a famous experiment[1], Gregory Werner and Michael Dyer could show how a complex communication code could emerge in an artificial species. Their idea was that females and males should communicate in order to meet and reproduce, so that natural selection would let an efficient code emerge.

They created a population of agents with static singing females and mobile blind mute males. Females have a 5x5 vision field, in which they can perceive males (the closest if several are present) together with their orientation (there are thus 24 x 4 possibilities). From this, they emit a song chosen among 4 possible calls. The female part of the genome thus requires 96*2 = 192 bits to code for this behaviour.

Males hear the call of the closest female if any (within a 5x5 range). They decide to move or turn accordingly. The male part of the genome thus requires only 8 bits for this mapping, as there are 4 possible calls and each requires two bits to decide next action.

Experiments show that in most of the cases one can observe the emergence of a communication code that enables females to cleverly steer males towards themselves.

Load the configuration WernerDyer.evo in the Configuration Editor. You will notice two specific parameters that are available for this experiment:
• WDGridSize, which control the size of the grid.
• compass, which control the use of relative (0) or absolute (1) directions.
The compass option gives the possibility to use absolute direction for the song. The female part of the genome is reduced to 48 bit in this case, as the direction of the male is irrelevant.

 Run Evolife (button [Run]) and observe the evolution of reproduction efficiency and of majority strategies. Make several trials by changing the density of population. Observe (possibly by running the experiment several times) that the population may go extinct. Why ?

 Run the experiment (button [Run]) with a high density population. Wait until a communication code is eventually found (it may be long, restart the simulation if you are losing patience). Do males turn in both directions? Can you explain why?

The evolution of a communication code is a coordination problem. Even if two individuals happen to agree on a bit of code, they are unlikely to meet again soon and their agreement is likely to be lost. Moreover, their children will not inherit this agreement between the male part and the female part of the genome, as crossover will probably destroy it. To evaluate this effect, why not merely clone the parents? Modify the Move function in Scenarii/S_WernerDyer.py to get clones of the Male and the female when they are reproducing. Reproduction occurs when a male and a female meet. The couple is added to the list Parents. So replace:

self.Parents.append((Male, Partner))

by

self.Parents.append((Male, Male))
self.Parents.append((Partner, Partner))

 Run Evolife again (button [Run]) with this cloning strategy (set the compass parameter to 1). What can you say about the convergence of the genome ? Is that a good or a bad thing ?

#### Suggestions for further work

• The genome could code for the weights of a simple associative network, as in the original paper, instead of coding behaviour directly.
• The genome could code for probabilities instead of rigid behaviour.
• One may introduce semi-permeable barriers to see if dialects develop.

#### Notes

[1] Werner, G. M. & Dyer, M. G. (1992). Evolution of Communication in Artificial Organisms. In C. Langton, C. Taylor & J. D. Farmer (Eds.), Artificial Life II, 659-687. Addison-Wesley.

Back to the main page