I mean, all that said I think Meditations on Moloch is pretty confused about the evolution of norms, because it implies that natural selection tends to suppress coordination “ordinarily,” and then suggests “coordination” as this sort of extra thing that could “solve” this problem

But a lot of people use evolutionary game theory to study the evolution of norms that help coordination.  The theory doesn’t separate cleanly into “ordinary evolution,” which sucks, and “coordination,” which swoops in and saves us.  Coordination can arise from optimization for points in the game, so “strategies that don’t optimize for points in the game die off” is not necessarily a problem.

The post contains a lot of images of things we don’t want sweeping to 100% fixation and wiping out every last vestige of certain things we do want.  But “goes to 100% fixation” isn’t just a thing that unpleasant strategies do.  Once Tit For Tat gets going, it too crushes every last competitor in its path, not because of some special value beyond game points, but because it gets the most game points.

I guess one answer to this may be that I’m misreading the post and replacing “things we value” with “coordination,” where the former is broader.  But I’m not sure that answer works; when we see that something we value is dying out, then we can save it so long as we can coordinate.

@slatestarscratchpad