Still more on prisoners’ dilemma

Tit-for-Tat was long considered to be the most effective strategy. However, a team from Southampton University in England introduced a new strategy at the 20th-anniversary Iterated Prisoner's Dilemma competition held in 2004, which proved to be more successful. This strategy relied on cooperation between programs to achieve the highest number of points for a single program. The University submitted 60 programs to the competition, which were designed to recognize each other through a series of five to ten moves at the start. Once this recognition was made, one program would always cooperate and the other would always defect, assuring the maximum number of points for the defector.

If the program realized that it was playing a non-Southampton player, it would continuously defect in an attempt to minimize the score of the competing program.

There is a key point to be remembered - It requires connivance and collusion of a significant proportion of players to beat strategies that have the characteristics of “Tit for Tat” – nice, retaliatory, forgiving and non-envious. This is possible in many cases such as oil cartel, monopolies and cartels in limited geographies, uneven power equation, etc.
However, for most individuals living in society and having a large number of interactions with many people, there needs to be no fear of a large number of them ganging up.

More on Cooperation

There is a great post by the venerable Tom Peters. Please check it out. It’s titled “Love Thine “Enemy”! It’s good Business”.

He says that if others thrive his share of market will go down for sure but the enhanced market size will more than compensate for the loss of share. Further, quality competition will only enhance quality of the originator and will force the innovator to do better and different work.

This yet another example of how cooperation is not just a tool for altruism, but also an effective means of growing and prospering. Just like Dawkin’s “Memes” and Analole Rapoport’s “Tit for tat” strategy.

Anatole’s comments on the prisoner’s dilemma are worth repeating “the cooperative solution is the only truly rational outcome in a non-constant sum game”. The Prisoner’s Dilemma is played many times by the same participants. Eventually they learn to cooperate; and cooperation is based on trust, the belief that the other will reciprocate, now or later, the actions taken by me that benefit the other. This is how social capital is built.

Trust is created by people who repeatedly interact with one another. It follows that trust will be difficult to obtain in one-off interactions or interactions limited to a few people where depending on the power equation a cartel can coerce the less powerful.

That in effect is a vote for a democratic and open economy.

The Selfish Gene

I read this Richard Dawkin’s work only recently, about 30 years after it was originally written. It’s a superb book and helps one understand behavior, among other things.

A difficulty in discussing the book, particularly with people who haven’t read the book is that people conclude that it says that human beings are selfish, or are naturally inclined to be selfish. This switches them off. However, a gene is different from a human being and a selfish gene is not a selfish human being. The gene’s selfishness is looking for its preservation, and that is what it strives for.

The book also goes on to introduce and discuss the concept of a meme, an idea stored in brains, and which can reproduce and grow like genes, and which differentially does better or worse according to its previous experience in the world. The conditions are there for evolution to occur among these abstract things. This is a far more complex issue to grasp, but has tremendous implications. It helps explain the altruistic actions of men and groups of men – why do they do this even when the genes are driving you towards preservation. Somewhere, in societies, the preservation of ideas and ideals takes over from preservation of genes.

In his book Dawkin included a chapter on the iterative prisoner's dilemma tournaments (this happened in the 1989 edition), in which cooperative and forgiving strategies thrive over cheats. Which indicates that evolution promotes a large dose of altruism.

It’s a fascinating mix. I have found the iterative prisoner’s dilemma to be very interesting. Dawkins work on the genes and the memes comes from a very different approach and yet the two approaches meet somewhere in a powerful way.

More on Prisoners Dilemma

Further to my earlier post, I would recommend the following links to readers

Robert Axelrod's Home Page

Evolutionarily Stable Strategy

Prisoner's dilemma

The classical prisoner's dilemma (PD) is described below:

Two suspects, A and B, are arrested by the police. The police have insufficient evidence for a conviction. They visit each of them to offer the same deal to both - if one testifies for the prosecution against the other and the other remains silent, the betrayer goes free and the silent accomplice receives the full sentence of 10 years. If both stay silent, the police can sentence both prisoners to only six months in jail for a minor charge. If each betrays the other, each will receive a two-year sentence. Each of the two prisoners must make the choice of whether to betray the other or to remain silent. Neither prisoner knows what choice the other prisoner will make. And hence the dilemma

The dilemma, in summary
1. Both Stay Silent - Both serve six months
2. Prisoner A Stays Silent and Prisoner B Betrays - Prisoner A serves ten years, Prisoner B goes free
3. Prisoner A Betrays and Prisoner B Stays Silent - Prisoner A goes free, Prisoner B serves ten years
4. Both Betray - Both serve two years

Each prisoner has two options: to cooperate with his accomplice and stay quiet, or to defect and betray his accomplice in return for a lighter sentence. The outcome of each choice depends on the choice of the accomplice, but the player must choose without knowing what their accomplice has chosen to do.

If my partner stays quiet, my best move is to betray as I then walk free instead of receiving the minor sentence. If my partner betrays, my best move is still to betray, as by doing it I receive a relatively lesser sentence than staying silent. At the same time, the other prisoner's thinking would also have arrived at the same conclusion and would therefore also betray. So going from the best outcome for an individual, we would both betray.

However, from the perspective of the optimal outcome for the group (of two), the correct choice would be for both to cooperate with each other, as this would reduce the total jail time served by the group to one year total. Any other decision would be worse for the two prisoners considered together. When the prisoners both betray each other, each prisoner achieves a worse outcome than if they had cooperated.

The above game interested game theorists a lot, and from the basic construct a more general form of the game emerged.
There are two players (1 and 2). Each player has two choices in an interaction - "Cooperate” or "Defect". Each player makes his choice simultaneously, ie., they know each others moves only post facto. At the end of a turn, each player receives the amount as described below, depending on what each of them did (cooperate or defect) -
1. If player 1 defects and player 2 cooperates, player 1 gets the Temptation to Defect for a payoff of 5 points while player 2 receives the Sucker's payoff of 0 points.
2. If the reverse happens, the exact reverse in terms of score happen.
3. If both cooperate they get the Reward for Mutual Cooperation payoff of 3 points each
4. If they both defect they get the Punishment for Mutual Defection payoff of 1 point.

This is basically same as the original game without the same terminologies. Here positive points are scored in every case (or zero), which gives the players an illusion that they are not losing. This simulates real life better.

From this the next step is the iterative prisoners dilemma. Here a set of two players play each other repeatedly. This means that players have memories of what the other did last time, and that influences his strategy against the particular player.

Finally the big step was a tournament of several players, each playing the above game with everyone else simultaneously and iteritively. For example if there are 100 players, each is playing the game individually against each of the other 99 repeatedly, say 20-25 rounds or more.

What strategy is best under these circumstances – cooperate or defect? Moreover, in a society where every interaction has a potential to cooperate or defect, what strategy would finally emerge in life? Would people defect from each other to maximize individual gains or would they end up cooperating? These questions had immense value.

Dr. Robert Axelrod, a game theorist got interested in the above, and back in 1984 he decided to conduct a tournament. He invited many players to give their strategies, which were fed in a computer. Each player had to play other players with a strategy for over 100 rounds. The strategies had to be programmed. He received several computer programs, some of which were very complex. Some were very simple such as always cooperate or always defect. Some had random moves, and so on. On the day of the tournament, the strategies were fed in and the computer flagged off. There were over 70 participants with individual strategies.

The end result was startling. The winner was a simple strategy developed by a person called Anatole Rapoport called Tit for Tat. This strategy was to cooperate on the first move and then with each opponent do exactly what the opponent did on the last move. If he cooperated, cooperate, if he defected, defect.

This was a surprise. This simple strategy beat all the complex strategies which tried to look at various patterns in each opponent.
Axelrod concluded that the lessons were that the strategies needed to be:

1) Nice - It will not defect before its opponent does. Almost all of the top-scoring strategies were nice. Defecting strategies faced heavy retaliation, except for the very few “always nice” strategies.
2) Punishing - The successful strategy, though nice, needed to punish dissenting behavior. A non-retaliating strategy is Always Cooperate. This is a very bad choice, as "nasty" strategies will ruthlessly exploit such softies like parasites.
3) Forgiving - Another quality of successful strategies is that they must be forgiving. Tit for tat, for example, punishes dissent, but when the other comes back with cooperation, it again cooperates. This stops long runs of revenge and counter-revenge, which reduce points.
4) Consistent – Strategies that had random elemnts, tended to elicit retaliatory behavior. Consistent strategies worked well in getting cooperation from other strategies.

Axelrod then started thinking of other strategies such as Tit for tat that starts with defect instead of cooperate, tit for two tats, and others. He published his analysis and asked for another participation in a tournament. Once again he got great response, except that now the participants had full report of the last tournamnet. Once again the tournamnet was flagged off. Axelrod himself was looking for new successful strategies.

However, once again Tit for tat came up trumps over all other strategies. It was an overwhelming evidence in favor of Tit for tat.

The whole exercise has direct relevance to life. In life and in business, we are dealing with hundreds of people and we are cooperating or defecting all the time and building a track record and history with each interaction. Hence everyone else is also reacting to our moves. The success of Tit for tat strategies and the lessons drawn by Axelrod provide a guideline for us. To succeed in life we need to elicit cooperative behavior from others, which Tit for tat does beautifully.

Kindly note that one needs to be non-envious, that is not strive to score more than the opponent. Tit for tat will not score more than its opponent. At best they will both be equal. In fact Tit for tat treats them as partners not opponents. That is the secret of its success.

I would encourage the reader to google on prisoner’s dilemma and derive more insights into it.