taking a break from “ai discourse”

Over the last year or so, I’ve been writing a lot of long posts about AI timelines, AI risk, the limits of modern ML, and related issues.

People seem to enjoy these posts, and I’m proud of many of them. But the experience hasn’t been good for my mental health.

I’ve been spending way more time than I’d like thinking about these arguments in an obsessive manner – where I want to stop and think about something else, yet I somehow can’t.

—-

Part of the problem is that, this past year, I’ve gotten into the habit of reading LessWrong habitually – something I haven’t done for a very long time.

LessWrong is full of people afraid of near-term superintelligent AI, much more so than the rest of my social circle. (Even the parts of my social circle that read LW are much, much less AI-doomerist on average than the people writing most of the posts on LW.)

This created a sense that “whoa, everyone around me is suddenly way more terrified of AI!”, which felt like a scarily sudden and scarily unmotivated social shift, and made me feel obsessively driven to figure out where these fears came from and what evidence was supposed to justify them.

To some extent, I do think this shift is a real phenomenon that goes beyond LW, and was (very roughly) contemporaneous with me picking up LW again. But it’s also become clear to me that I’m driving myself crazy by reading LW, and letting “what LW posters think” seep into my assessment of “what people think, in general.”

—-

So, I’m going to stop reading LW for now. (Not just habitual reading, but reading posts on the site, period.) I’ll also avoid related sites like EA Forum, and personal blog posts containing the same kind of material.

I’m also going to try to talk about about AI risk and AI timelines as little as possible, whether online or in person.

I’ll still talk about these topics if there’s some overwhelming reason to, but not just as banter or because “someone is wrong on the internet.”

I’ll still talk about ML more generally, as long as the topic isn’t too close to big-picture prognostication about risk, timelines, and the ultimate power/limits of ML.

I don’t know how long I’ll keep this up for. With reading LW, I can probably keep it up indefinitely, since I spent years not-reading-LW in the past and feel none the worse for it. For the more general prohibition on AI discourse, I’ll keep it up at least until the end of the year, and then think about how I feel.

Writing Almost Nowhere is really exhausting.

(Just to be clear: I’m not complaining. I like doing it. If I didn’t want to do it, I’d stop.)

I write in long, continuous intervals, many hours at a time. It takes me 1-2 hours to rev up to full speed, and once I’m there I keep going for much longer, for 6 or 8 more hours.

During these trances, I rarely get distracted. I don’t take breaks, or think of taking breaks. I don’t do anything except write, steadily and slowly.

A single day of this isn’t enough to produce a chapter. Usually it takes two. (The work required can fit into a single weekend, but only if I do very little else that weekend, and only if I’m okay being kinda useless the following Monday.)

As an example, according to my git logs, the most recent chapter (#28) was written in two long stretches:

• One lasting 7 hours and 33 minutes
• Another lasting 11 hours and 17 minutes

Chapter 22, the one with Sylvie talking to the Ells, was written in a single sitting. But it was a very long sitting.

As you might expect from this description, I feel utterly drained every time I finish a chapter. Not just physically or mentally but like, spiritually.

At the time, I always think about characters in fantasy stories who can call upon magic powers, at the price of a little bit of their soul. Like the Madoka girls, or like Rose in Homestuck, the kind of stories Rose is a riff on.

I know this comparison is probably like, “extra” or “cringe” or something, but it is what always comes to mind.

I often feel a sudden apathy or even revulsion towards what I’ve written, once it’s done. I specifically remember feeling this way toward Chapter 22, that one with Sylvie and the Ells. I had to hold my nose, so to speak, and force myself to post it, against my sleep-deprived gut sense that it was wholly without any value. (It seems strange to remember this now, because when I’ve re-read that chapter I like it a lot.)

Writing this book is always an intense experience. It feels like, I dunno, taking a final exam? Or like taking the qualifying exam for one’s major, maybe, is a closer analogy.

I don’t really know why it’s like this. TNC was like this, I think, except with much shorter chapters and a much shorter book. I think Floornight was sometimes like this, but less so. Although I don’t remember Floornight as well.

Some of it is just that AN has a particular style, and that style is really effort-intensive to produce. I have to linger over every word and sentence, not even so much in the name of good writing as because it’s impossible to write in this style without lingering over every word and sentence. It’s a lingering, deliberate style.

The amount of calendar time I’ve spent on the book exacerbates the intensity, too. By now, a lot of the events I’m narrating are ones I came up with literal years ago. I’ve been playing them over and over again in my head for a very long time. And now I feel I have to produce a string of words that does justice to those years’ worth of daydreams.

I guess that’s why a qualifying exam comes to mind. You’ve spent years studying the subject, but none of that matters now, no one can see it, all they can see is what you do in the next few hours – which is suppose to sum up those years of studying. Got that? Good. Now go.

Anyway, that’s a lot of why the book’s taken so long. It is simply difficult to fit around the rest of life.

I said I would finish it in 2022, and so far I’m more or less on track. And that is no more or less than what it sounds like: spending a significant fraction of my weekend time inside of these intense, exhausting trance states. It’s a weird thing to do to oneself, and I wouldn’t choose it as a long-term lifestyle. I definitely would not have done it organically, without the target date pushing me.

I don’t know if I’ll continue writing after I’m done with this book. I probably will, sooner or later. If I do, I’ll try to write something with lighter, more flexible requirements of me. This book, though, is just what it is – that ship sailed long ago, and I do want to see its voyage through.

I’m thinking about my own perceptions of whether I’m “smart enough” to understand various things…

There’s a fine line between the tasks and ideas I find easy to understand, and those I find almost impossible to understand.

Based on the way people tend to talk about intellectual difficulty, it feels like there ought to be this whole middle band of tasks that are “intellectually challenging”: I have to stretch my mind to do them, and it’s hard work, but I can get there if I try.

This band does exist for me, but it contains very few things – fiction writing is one of them. Most things are either so easy they require little felt effort, or so hard I have trouble with what are supposed to be the basics.

If I tell myself “what I’m doing right now is an intellectually challenging task,” this thought almost always makes me feel either very smart (because the task is not challenging at all for me) or very stupid (because the task seems impossible for me).

It’s like I’m playing some highly competitive game with very well-defined skill levels, where I can trounce any player slightly below my level, and I get trounced by any player slightly above me.

For example, even though I have a lot of formal education in math and physics, I habitually think of “advanced math” as being “over my head.”

This is sort of true-by-construction, because my internal definition of “advanced math” is “the kind of math that’s over my head.” But it’s not contentless: there is a real, crisply delineated subset of math that I have a lot of trouble following, even in its basics. And most of the stuff that most math professors do is in this category.

On the other hand, I was (and am?) very “good at math” in the sense of high school and college-physics-major math. In high school and college, everyone was impressed with how good I was at the stuff. Integration, Fourier series, limits, formal proofs, probability distributions, differential equations… these things that feel still anchored to the ground in some way, they’re easy for me.

But then, if you step a little further out into abstraction, suddenly I’m lost. Once the concepts stop having exemplars in ordinary physical or pictorial intuition (an integral tells you how much stuff you have, a Fourier series breaks a pattern up into sine waves), and start being about other concepts which are themselves about concepts, suddenly things are impossible.

Or like, programming. Programming is easy. I could do it all day (and often do). Sometimes it’s tedious, but it basically never feels challenging. There’s no friction. I code as fast as I can type (or read, when I need to read code). It feels like cheating, going from grad school – where I was always in danger of reaching something impossible – to this.

But that’s about programming at “my level,” usually in python. When I start to try something like functional programming, I immediately notice the feeling of things that are supposed to be basic feeling very hard to grasp. I hit the same abstraction barrier.

I think it really is about abstraction. For some reason, I can only think about a concept if I have a concrete exemplar I can put next to it in my head, where “concrete” means something like “feels familiar.” I can understand things, and types of things, but once we get into “types of types of things” the problem begins, and “types of types of types […] of things,” i.e. most of mathematics, is impossible.

(This boundary is very clear, naturally enough, when I try to read intros to category theory. “Morphism”: fine, these are just functions or ordered pairs or that kind of thing. “Functor”: also fine, these are like analogies – two things share structure, so you can point to a relationship on one side and its analogue on the other side. “Natural transformation”: wtf is this, an “analogy between two analogies”? A “structure-preserving function between structure-preserving functions”?

It feels like a moment ago we were talking about things I use all the time in real life, and now we’ve crossed over into a void of weightless, faceless abstract terms that only refer to other terms.)

fipindustries:

nostalgebraist:

Chapter 23 of Almost Nowhere is up here

Ok i feel we may be close to the end considering how much the pace is picking up.

We are definitely getting somewhere, plot-wise, but there’s still a ways to go before the end.

The update pace has increased because, near the start of 2022, I took a hard look at how much of the plot was left and how slow I’d been writing – and the way the remainder of the story had come to feel like a burden, weighing me down year after year, on into my 30s

and I thought, “OK, either this can be a burden that haunts and teases me year after year – or this can be an exciting adventure about me facing the thing head-on and pushing through to the other side, making all my plans real now and not in some endlessly deferred future”

and I thought, “all right, me, 2022, were doing this man, were making this hapen”

I’ve calculated the average rate I’ll have to write in order to finish the book this year, and I’m not sure I’ll be able to meet it, but it’s a number that feels plausibly within reach, at least. We’ll see how it goes.

(If the pace of the plot has also picked up recently, well, that’s just the part of the story arc we happen to be in)

I blogged a fair amount about Legion the TV show when it was running but have hardly thought about it since it ended.  Since its existence randomly popped into my head this morning, I want to write down the following opinion, while I’ve had for a while but never blogged about:

That final scene of Season 2, which I wrote a long rant about when I first saw it?  That scene is really good.  I’ve re-watched it a number of times and it’s … one of the most raw, powerful things I’ve seen on a TV or film screen.  (I don’t watch many TV shows or movies, so maybe that isn’t saying much.)

It’s not like I’ve changed my mind about the qualities that my rant was about.  The scene does in fact reveal a bizarre, apparently very ignorant view of mental illness that I’m shocked could make it onto network TV in 2018.  Nothing in the final season undoes that, and in fact the writers drag their heels in further there.

But the scene is so powerful in part because the writers don’t seem in control of what they’re doing.  It’s like they’ve summoned a demon by accident, and are standing around, gawking awkwardly at the pentagram they drew and mumbling “we didn’t think it would work … ”  As a twist at the end of the story’s middle act, the scene is meant to be exciting in a standard-issue, screenwriting-textbook way, but it ends up being a very different kind of exciting.  The car has gone off the road; no one is driving it anymore; nightmare energies have been called up that no one had expected and no one can predict.

Legion is worst at portraying the perspectives that are supposed to be its moral center.  Its implicit moral worldview is naive, childlike, a “good cluster morality” in which you can read someone’s quality as a person off of incidental properties of their appearance, diction, and deportment.  Syd, the purported hero in the end, is a terribly written character with little to do but stand around delivering carefully worded didactic speeches with impeccable poise.  But the “bad” people are allowed to be flawed, impulsive, rough-edged (or just rough), traumatized, messed-up, irreverent, cynical.

The show seems to think we’ll just read all these things as proxies for “bad,” which is terrible.  Emotionally, it makes me want to “side with” these characters against the writers who created them.  I want to grab their stuffy lapels and yell “Lenny deserved better, dammit!” … for example.  And, almost paradoxically, this resistant reading of the show is interesting and emotionally involving enough to make the show worth watching.  Like Andrew Hussie, the writers are a villainous presence, but they lack Hussie’s self-awareness and metafictional bent – which makes them better villains.  You really do want to grab their labels and yell various things, because it seems plausible that they might not actually know.

Anyway, that one scene is the apex of all this, which nothing in the pretty-good-but-underwhelming final season can match, and whose power that final season cements by confirming the writers indeed had no idea what to do with the demon they had summoned.  Through their ineptitude, they become better villains in their own story than any of the textual ones.  By seeming not to recognize to the very real-looking pain and trauma they thrust in our faces, they make that pain and trauma feel all the more real – for in real life, one does sometimes feel rejected not just by other people but indeed by “the writers,” the universe, the moral law itself.

Is there a standard name for the argument, or pattern of thought, that goes 

“sometimes (or perhaps always), a single choice you make has the moral or personal weight of making that choice over and over again in similar or identical situations: unless you can supply a detail that would distinguish this one situation from its ‘copies,’ by endorsing a choice here you endorse it in all those copies”

?

I feel like I’ve seen versions of this in various places, each different from the others but still with enough of a common thread to make it a thing.  Examples include

• Eternal recurrence: willing something to happen once is like willing it to happen again and again in a universe that repeats itself identically
  
  
• The categorical imperative: an injunction to use a version of this argument whenever you propose a moral principle
  
  
• Some informal uses of game theory, specifically when games are used to think about action or morality in general rather than about a particular case, and especially when the games are one-shot.
  
  (I.e. saying that some real situation has the structure of a prisoner’s dilemma or stag hunt isn’t like this at all; taking moral inspiration from tit-for-tat or other iterated strategies is only slightly like this; viewing “cooperate in prisoner’s dilemmas” with no or few further specifications as a moral goal, as I see people do sometimes in the LW-sphere, is definitely like this)
  
  
• When I was younger, I used to think this way reflexively about some things. For example, it seemed very important whether a person treated strangers well, because a stranger is someone you know almost nothing about and hence being mean to a stranger once seemed like a disturbing endorsement of a world where everyone is mean by default.
  
  I no longer think this way – because even if the strangers are symmetrical the person being nice or mean to them will be in different situations over time – but it was compelling at the time, so “obvious” I barely thought about it
  
  
• Lately my mind has been using another version of it a lot: unlike most of the past, I’m in a life situation which I like and which has no natural end point, but this makes ill-spent time and bad habits seem worse than they used to.  Spending one day frivolously feels like deciding to spend every future day frivolously, because I can’t say “oh future days will be different eventually” with the ready confidence I used to have
  
  (I mean, they undoubtedly will be different in various ways, which is why I see this largely as a mental tic I want to reduce rather than a good argument, although there is some value to it.
  
  The thing I’m talking about involves deeming some group of situations “effectively the same,” but [except in eternal recurrence] no two situations are really identical, so it’s all a matter of where you draw the line when lumping or splitting situations.  If I’m wary of this thing, it’s because it depends on lumping more than is intuitively natural, and then puts intuition on the defensive – “go on, show me the distinguishing factor!” – rather than defending its own choice)

(Until looking over my tag just now, I had this impression I had said almost this exact thing before, but it looks like I’d only done so much more implicitly and reservedly than I remembered, and anyway not recently, so … )

I really get a lot of value out of it when other people read Almost Nowhere and say things about it.  I would be really happy if more people did this.

—————

I’m pretty nervous about feeling like I’m begging for attention and validation here, cf. the way I started off this post with the parenthetical above and even now am derailing it anew with this sentence.

In particular, I have this intuition that “when I do things that people actually like, they become self-advertising” – I didn’t have to write posts like this about TNC, or for that matter about my own nonfiction effortposts here, and if I want a comparable level of interest in AN then (the line of thinking goes) I should just keep writing and making it as good as possible, and “if I build it they will come.”

However, AN is really in a somewhat different situation than those other things.  It is a relatively long story – I can imagine it being 2x the wordcount of Floornight by the end – that I am creating very slowly over a number of years, with more care and deliberateness than I’ve applied in the past.

I feel confident that I will complete the whole thing within (to set a goofy upper bound) the next ten years, but I expect it to take at least another 1-2 years, possibly more.  I know it’s hard to get people interested in a WIP, or in very piecemeal occasional updates that don’t build an exciting sense of momentum.  I know people want to read complete things, and “read it when it’s done” might still be the best option even though it means you’ll be reading it in (could well be) 5 years when you and I and the world are five years older and god only knows what’s happened in the interim.  Just, that’s what that option looks like.

—————

And I realize that was an uninviting downer of an advertisement inasmuch as it was an advertisement at all, so here’s another one.

The reason I make posts like this is that I’m extremely proud of Almost Nowhere.  Like, distinctly prouder of it than any other creative or quasi-creative thing I’ve ever made, as far as I can tell.

I can’t say it’s strictly better than my previous novels, since they’re all doing different things and can’t be usefully compared like substitutes for one another.  But when I re-read the earlier novels, there are parts I like and parts I don’t, there are things I cringe at, places where I think “ugh, I took the easy road” or “oh, I feel bad about this chapter, should I skip it?”

Yet when I re-read AN, as I do every so often, I just feel this sense of pure glee over the whole thing, even parts I wrote 2 or 3 years ago: I like each chapter individually, I like every character and plot thread and theme and verbal motif, I like virtually every sentence.  It feels like what I imagine an actor or animator might feel watching their own demo reel, curated to string together only the peaks of their output without anything else.  I’m inordinately pleased with what I’ve done here.  (Admittedly some of this comes easier with something incomplete, as endings are uniquely hard to pull off for writers in general and me in particular, but still.)

So, if you tend to like things I like, much less things I make, you might really like this one.  FWIW.

I’ve sometimes talked about how the level of distance I feel from my past self varies as a pretty consistent function of relative time.   That is, at any given time, things I remember thinking/saying/doing X years ago tend to feel familiar or alien in a way that depends predictably on X.

It’s weird to watch past selves recede along this trajectory.  Looking back was more comfortable back when this “alienness function” made my high school self (pre-2006) much more alien than my college-era self (2006-2010), since I actually changed relatively quickly between those two periods, and since that’s a socially ordinary place to place a chapter break in one’s life.  But these days, my college self has mostly receded back into the same mists that I’ve grown accustomed to placing my high school self in.  2010 was a long time ago.

And because the high-school-to-college transition happened so rapidly, and seemed so ripe for interpretation even at the time, it feels relatively easy to think of it as a well-defined thing that might be boiled down into a set of specific, articulable lessons and changes.  Indeed, interpretations like this came pre-made for me, handed down to me by the same self they described.  The distance I face now as I look back is more continuous, less tied to any identifiable start or end points, and it comes with fewer helpful annotations.

Nonetheless, I notice some patterns when I look back at it.  The really big one is that my self from 10-12 years ago was much more “apriorist.”

He was inclined to believe as a matter of course in things like a priori knowledge and universal human nature.  He thought that there was just a way life was, that everyone’s life was basically that way with some minor person-specific modifications added on top, and that anyone who claimed otherwise was lying or (more likely) not expressing themselves clearly.  He believed, in various ways, in a natural match between the human mind and the environment, which rendered “important” things ultimately comprehensible and tractable (although it might sometimes be very hard) and, likewise, implied that anything sufficiently bizarre and un-parseable must be “irrelevant,” a distraction from the single correct way of life for human beings.  He reacted warmly to things like the Poverty of the Stimulus argument, not so much (or only) for any empirical merits he saw in them, but because they felt like the sort of thing that just is deeply true.

In retrospect it seems jarring that he had this same reaction to, say, evolutionary psychology and mathematical Platonism.  If our mathematical faculties can ultimately be traced back to material trends in our ancestral environment, one might ask, how can we know that we grasp eternal trends in some kind of eternal Forms, rather than trends that merely obtained approximately in that environment?  As far as I can remember, my old self was less focused on the possible limits of our understanding than the apparent extent of it: to him, it was a miracle that he or anyone else could understanding anything with any reliability, and (I guess) this miracle could only be explained via some deep patterned correspondence between mind, material world, fundamental reality, all of it.

We could only get anywhere in life because we came with a detailed facsimile of Universal Truth printed on the inside of our skulls, and the most our parents or society could do was add a few footnotes or maybe cross out a passage or two.  All we could do was follow the inevitable logic written in there, which said to live “the good life” as described in the in-the-end consistent if sometimes distorted stories told by great literature, religion (all of ‘em, probably), cutting-edge science, my parents, my knee-jerk emotional reactions to stuff, etc., etc., etc., or … one could do anything else, which was to do something dumb, wrong, and ultimately empty.

I’m describing this kind of parodically, of course, and it’s a better description of my high school worldview (which has long been comfortably in the past) than of my college-and-soon-after worldview.  But the latter was still … like this, just very gradually less and less so over time, with no clear turning point, which is harder to accept and to get my head around.  I also wonder how many people are out there now, in similar positions to mine with knee-jerk attitudes similar to the ones I held.  (I can’t imagine this outlook thriving in mainstream online discourse these days.  In the period I’m talking about there wasn’t really such a thing as “mainstream online discourse,” and it was much more common to have strictly on-topic conversations about video games or whatever in which these philosophical gulfs might never have a chance to come up.)

argumate:

kontextmaschine:

The more I interact with post-millennials the more convinced I am that the internet’s (and society’s) shift in the last decade hasn’t been about “social media” but about smartphones, that the internet isn’t even a place you go anymore but something omnipresent everyone takes with them, so there’s no experience you have in isolation cut off from the akasha.

“Grew up with a smartphone” starting to seem as significant a distinction as “grew up with the internet” was for us.

yes totally, I mean it’s a cliche that everyone is on their phones all the time and yet still everyone is on their phones all the fucking time and some people haven’t known any other world.

Occasionally I think about this in connection with an otherwise trivial memory from my adolescence.

I was on a few video game-related Internet forums as a teenager in the early-to-mid 2000s, and I was pretty heavily involved in them, but none of my IRL friends had ever been on anything like them. Since smartphones weren’t a thing, if I was at a friend’s house and wanted to check for replies to my latest post, I’d have to ask to use their computer.

I was self-conscious about this being a dorky thing to do — my friends were extremely dorky themselves, but if anything that intensified the teenage crab-bucket atmosphere — and I tried to defuse the awkwardness by lampshading it: turning “Rob has to do his forum checks” into a self-deprecating injoke as though it were some embarrassing addiction I suffered from. This was successful as far as it went, and for a time “forum check” became one of those meme-like phrases that automatically qualifies as a bit of mildly funny, mocking banter whenever anyone says it, even outside of anything otherwise resembling a joke.

Anyway, every once in a while I remember this, and think about how I’m surrounded by people doing “forum checks” whenever I’m in public, on buses, in stations and lobbies, in the middle of the sidewalk, everywhere. (This is about the mainstream rise of social media and not just smartphones, but without the smartphones there wouldn’t be the image — forum checks literally all around me, now simply one of the default human activities.)