In human societies, if some individuals fail to cooperate in joint enterprises, there’s often an institution that imposes sanctions on them. If there isn’t, people seem to be willing to punish them directly, even at a cost to themselves. How does that behavior get started? How does a cooperative society “get off the ground”?

In 2007, researchers from Harvard University and the University of Vienna created a model in which individuals can choose either to receive a secure income or to play a risky game. Those who choose to play the game can either pay a fee or choose not to. The fee supports an investment, the proceeds of which are distributed among all the players. If enough of the players pay the fee, then everyone benefits. But if too many players opt out of paying the fee, then the remaining donors suffer.

In order to avoid that situation, the donors are allowed to impose a penalty on the freeloaders, but this incurs a cost for them, so not every donor will choose to enforce it. So overall there are four strategies: nonparticipants don’t play the game at all; freeloaders play but choose not to pay the fee; contributors pay the fee but choose not to impose penalties on the freeloaders; and enforcers play the game and punish the freeloaders.

After many rounds of computer simulation, the researchers were surprised to find that when participation in the game was mandatory, most of the players became freeloaders, and the game fell apart because no one was willing to pay the fees. Adding enforcers at this point couldn’t save the game — they could make no headway against the freeloaders.

But when opting out of the game was permitted — when players could choose to receive a secure income instead of playing the game — many of the freeloaders simply withdrew, leaving behind contributors, enforcers, and a few freeloaders. And now the resulting game was stablest if the enforcers dominated the group, since they ensured that everyone cooperated. If contributors began to outnumber enforcers — that is, if the group became unwilling to punish freeloaders — then the freeloaders took over and everything fell apart again.

“The paradoxical result is that cooperation can be enforced by penalizing freeloaders, but only if participation in the community is voluntary,” writes George Szpiro in A Mathematical Medley. “This reminds us that discipline in the dreaded foreign legion, which legionnaires join of their own free will, is legendary. In the compulsory army, on the other hand, it is often in rather short supply.”

(Christoph Hauert et al., “Via Freedom to Coercion: The Emergence of Costly Punishment,” Science 316:5833 [2007], 1905-1907, via George Szpiro, A Mathematical Medley, 2010.)