I have a question about Yudkowsky's current doomsaying, but it's not about alignment itself. Sometimes when he's talking about alignment he makes a claim like: "A near-future AI could develop nanomachines capable of spreading across the world undetected and then killing everyone simultaneously." The feasibility of such nanomachines seems far-fetched to me. Do any other experts (in AI or nanotechnology) believe super-nano-weapons are plausible, or is this something only he is convinced of?
I don’t know. I’ve wondered the same thing for many years, really.
The reason EY or anyone else believes in this kind of thing is because K. Eric Drexler’s work convinced them of it. (See the part here about Drexler’s Nanosystems.)
And I almost never see Drexler’s kind of transformative nanotech (“molecular nanotechnology”) discussed outside of futurist circles. The people I see talking about it don’t seem qualified to evaluate it. Most often these are people who work with computers, not physical systems of any kind.
Being qualified to evaluate Drexler’s work is a pretty steep bar, since the relevant knowledge spans several academic disciplines, but even having deep knowledge of a single relevant discipline would be a lot better than having deep knowledge of none of them.
Meanwhile, the world is full of physicists and physical chemists and biochemists and mechanical engineers, and I rarely see any of these people talk about Drexler, even to dismiss him. The one exception was the Drexler-Smalley debate, but that mostly felt like two guys dunking on each other. There was nothing like a serious, conclusive technical argument anywhere in there.
(As a sidenote, it’s been standard practice in academia for a long time now to brand your work as “nanotech” if it involves making anything nano-sized. Most of this has little if any relation to Drexler’s ideas. So there are lots of “nanotech experts” out there, but being a “nanotech expert” isn’t an intellectual qualification, it’s just a thing you call yourself to sell your research.)
The idea of molecular nanotechnology doesn’t seem self-evidently absurd, and when I read Drexler’s work it looks well-reasoned insofar as it touches on areas I know about. I don’t think he’s a “crackpot.” But the question of whether his ideas are viable seems to have gone mostly overlooked by domain experts, rather than accepted or rejected.
ok i just read drexler’s wikipedia article five minutes ago and don’t know anythinga bout him, but i feel like “this would be cool but i can’t syntheisze it” characterizes like SO much of chemistry that like. Idk. I think that’s how @firebendinglemur described all his materials science papers.
like I’ve seen so many papers simulating some mateiral or molecule that WOULD be cool. And hten never hear about it again. So what happened? Maybe you can’t make it. Maybe you can make it and it even has the predicted proeprties but it’s so hard nobody bothers. Maybe you can make it and it has great properties but it degrades too easily when used and nobody knows what to do about that.
Can you make tiny molecular wheels that do useful work? Yes because ATP synthase is one.
im looking at drexler’s thesis….
where “Dia” is diamondoid. So this is a structure to do a specific chemical task, ermoving a hydrogen. Can you make it? Idk
can that “support structure” actually exist?
where do those energies come from? Looks like he did molecular dynamics simulation.
What I’ve learned since entering chemistry is there’s a surprising amount of funding to sit around playing with what is essentially atomic-scale computer-aided design software. It’s what I was doing all weekend. THANKFULLY I don’t see huge debates ove whether these purely theoretical things would actually work, if nobody ends up making it it seems to get quietly forgotten.
Drexler sometimes uses the term “mechanochemistry” in this thesis, which is a cool term. Can “mechanochemistry” exist? Yes, it’s how ATP is made. Can his proposed devices be made, and if they were, would they function the way he thinks? Idk… but that’s not really the kind of thing a lot of scholarly attention gets devoted to afaik, there’s a sea of “cool thing i simulated” papers, but followup seems either of the form “i made this” or crickets… but idk im new to this field. But maybe it gives some context, Drexler would have been another water molecule in the sea if he didn’t publish books for the public
