As I said in an earlier post, I think the rationalists have a tendency to get stuck in rabbit holes as a consequence of the fact that people not believing in something won’t shut off the community spiral around that thing.

It’s kind of a weird side effect of “I take all ideas seriously,” where people file away certain ideas as worthy of taking seriously even if they don’t believe them, while not treating every idea in the same way (because that would be impossible).

In practice, there are certain ideas that really do cause you to take people less seriously.  This is what I am trying to get across (perhaps badly) with the flat earth examples – there are some things that you not only disbelieve but do not consider worth time worrying over, and in fact you judge people for believing them.  You can’t not do this, unless you have some sort of strange view where someone reliably coming to incorrect conclusions contains no evidence about the quality of their thinking.

Since you can’t really take every idea seriously, certain ideas become “protected,” so that no matter how little credence you give to them, you pretend they don’t convey any evidence about their believers.  Friendly AI was my example of such an idea in an earlier post: the practice among the rationalists is to treat it as a protected position even if they think it’s utterly absurd, while they (or most of them) would never give the same credence to various religious or political positions which have many more intelligent adherents than FAI.

As I said in the earlier post, it’s easy for me to imagine a world in which some blogger with the right Grey Tribe credentials had gotten convinced that there really was something in phrenology after all, and scores of mindless contrarians had latched onto this, and it had migrated into the rationalist world, and “phrenology” had become a protected position.  Although many people would nonetheless think there was nothing at all in phrenology, they would assure you that they did so in an informed way, tempered by the arguments of prominent internet phrenologists.  And arguments about phrenology would pop up again and again, often just for the sake of argument, since few would actually believe in it – it would just be one of those topics, like radical Bayesianism, or FAI, or Mencius Moldbug.

(A fondness for these things can be tolerated, although of course you don’t believe in them; a fondness for those without the right Grey Tribe credentials just indicates mindless conformity.  Of course.)

(And here is where everyone says “you’re saying things I like are as silly as phrenology!”  No, I’m trying to point out that it is possible to spiral around things in a way that has nothing to do with their truth value; the fact that you find phrenology silly is what makes it a good example of this.  Can you really not imagine that a few worlds over there is a rationalist community in which phrenology is a protected position even though almost everyone thinks it’s wrong?  Where tolerance of phrenologist views is used to signal one’s intellectual openness?  Where you aren’t a phrenologist, but some of your best friends are, and that’s OK, and fuck anyone who says otherwise?  But if this is OK, why isn’t it true here and now?)