—

Eliezer Yudkowsky (via abundance-mine)

—

Said the guy who literally believes he’s smarter than the rest of humanity and wants to build an AI that will take over not just the world, but the *universe*, and fill it with whatever One True Way all of humanity *really* wants (which it will discover).

And thinks there’s a good case to be made the children under two are not “sentient,” but that this is too much of a shock to our morals and values and we are not rational enough to question those unless we’re a rare scintillating jewel of an intellect.

And thinks that in a morally more-enlightened world than our own, people might think of rape as basically just a pleasant surprise. 

And is a big fan of evolutionary psychology as an explanation for why men and women are literal aliens to each other, and for racial IQ differences and “achievement gaps”, and has said outright he thinks that rich people really are smarter, more engaged and more alive than others (and that he really hopes to fix that because gosh isn’t it unfair). 

And has routinely boosted his own “Machine Intelligence Research Institute” (formerly the “Singularity Institute” until someone there realized how that made them all sound) as the single most ethical source for all charitable donations.

And…you know what, I don’t even want to go *on* here. XD But this sort of quote that sounds really profound and insightful (and clashes horribly with the rest of what he says/does) is sort of his bag, I guess…

(via amaranth-mantis)

I was unfamiliar with the source, but that sounds…special. :/

(via clatterbane)

Yeah it’s pretty horrible.  I know someone who fell in with his cult of personality, and it’s changed them beyond recognition.  They fell in with him after researching how to best invest their money… and of course coming to the Singularity Institute as the best possible way to do it.  They seriously believe they’re saving the world.  They probably will be insulted if they read this and find that I no longer believe them, not even remotely, after doing the research.

Another fun thing his group has done is attempt to convince people that real actual disasters facing the planet right now are not actually important, compared to the possibility of the Singularity.  So everyone should forget about climate change and feeding the world and all this other important stuff, and instead focus on making sure that a super-intelligent computer won’t be able to do anything bad to us.  Also?  Those problems won’t be problems once the Singularity comes.  Because the Singularity is essentially so advanced as to be magic, and will be able to manipulate people into doing whatever it wants, and even create elements out of nowhere so that we will no longer have shortages of anything, and everything will be fine.

Honestly I think part of the reason that people flock to groups like this is that the state of the environment is horrible, we’re facing catastrophic events in the next couple hundred years, even possible extinction… and it’s better to think that the real threat is a computer that will probably never exist — and to also believe that this computer will be their savior if they can get it to be on their side instead of destroying humanity or something.

And the reason this doesn’t make sense to anyone with common sense?  Is apparently because we’re not logical enough.  Human brains, you see, weren’t built for understanding a situation like the Singularity, so we dismiss it out of hand.  Instead, we need to learn to think Logically And Rationally, and then the Singularity will make sense as this horrible threat, perhaps the worst threat facing the world today, and we will want to pour all our time, money, and energy into the Singularity Institute.

The last conversation I had with this person was harrowing and felt like having my mind shredded.  So does reading their website.  There’s, for lack of a better word (and they’d laugh at me), a really fucked-up feeling, almost an ‘energy’, around this, that I don’t trust and actually fear quite a bit.  It’s intense, it’s deliberately constructed, and it’s as foul as foul can get.  I badly miss the person in question, but I can’t talk to them as long as they still have traces of that feeling attached to them.  It feels like my mind is being shredded by cheese wire and I have to stay away from things that make me feel like that. 

Please avoid getting sucked in.  It looks laughable, but it’s also quite sinister.  Be aware, also, that they have a lot of PR aimed at getting upper-middle-class and wealthy geeks to give them a lot of money, time, and energy, and that they often succeed.  Don’t be sucked in.  Please.  Even if you can’t see the kind of mind-patterns I can see, this is bad news, it’s far more than something mildly disturbing to laugh at.

(via youneedacat)

The pattern that Amanda Baggs is talking about here— heroic responsibility + the PC/NPC thing + “the most important thing to do with your life is to give money to MIRI” + FAI + Roko’s Basilisk— scares the fuck out of me. 

“It is your job to save the world. People who really matter try to save the world; people who don’t are lesser beings. You save the world by giving us money.” That is a scary and manipulative and probably abusive.

Please note that it’s a scary thing to say even if it’s true. You know how Scott keeps saying that you don’t get to dox people and you don’t get to lie with statistics and you don’t get to try to make people get fired for saying things you don’t agree with and so on, even if you are on the Side of Good? Because maybe you are not on the Side of Good, and then you will have hurt lots of people for no reason? You should also not manipulate people into giving you money, because maybe you are not on the Side of Good.

And you should not have that high a confidence about whether people giving you money will save the world, because holy shit motivated cognition much? There’s a reason we have GiveWell and don’t trust random charities claiming they’re the literally most effective thing.

(To be clear here: I am not saying MIRI shouldn’t fundraise. I am saying that there are plenty of ways you can lay out the good you’re doing without saying that people who don’t donate to you are evil, you are going to be remembered in ten thousand years, etc.) 

I am still part of this community because the meme hasn’t mutated into a more virulent form. Most LWers don’t donate to MIRI; I’m fairly open about being Singularity agnostic and no one punishes me; a lot of people say they think that meme is a load of crap; the all time most upvoted post is about how MIRI is a bad charity to donate to. 

And… I still think of myself as a rationalist, because a lot of things in this community matter a lot to me. I started reading Less Wrong my senior year of high school, and the Sequences has fundamentally shaped the way I think. And the community is important to me, because it’s a place where I am respected as a person and I can talk about things that interest me where people challenge me but don’t trigger me. 

But I’m worried that all the good things were invented to sell people on this poisonous meme, and I am worried that my participation in the community is adding credibility and helping more people to get infected with it, and I am worried that the meme will mutate to a more virulent form and I’ll be so far in I can’t get out. 

Non-rationalist friends: think about this before you get involved.

Rationalist friends: Beware the evaporative cooling of group beliefs. Remember that not everything someone who says they’re rational says is rational. Remember that you can be wrong.

I am not going to argue with people about this, but I am happy to talk with people who have doubts. 

(via ozymandias314)

(via ozymandias314-deactivated201404)