This article is very, very good and I’m linking it in lieu of some longer post I was planning to write about “explainability” vs. intrinsic interpretability, since it would have mostly covered the same ground

(To say something briefly, though: we really need a distinction between machine perception i.e. automatic feature extraction, which can and should be a complicated and difficult to compress function of raw low-level inputs, and machine judgment i.e. making a classification or decision on the basis of high-level extracted features, which had damn well better be locally expressable as a pretty shallow decision tree since that’s what human explanations of our own behavior amount to, and those are both invaluable for working together and apparently good enough for that purpose.

Relatedly, there’s a lot of current research in NLP — cf the GLUE benchmark paper and those citing it — on the dumb heuristics that modern NLP models learn, not because they can’t express complex features but because all the standard datasets are easy to game. They have excellent feature-extractors plugged into really stupid judgement-makers which probably perform worse than a few handcrafted rules on top of the same features; the tendency to view both as one magical, ineffable black box hampers progress, as does the closely related assumption that an high-performing decision process must be ineffable in itself. Meanwhile, much of the explainablility literature is off in a weird scholastic rabbit hole trying to decide how much of max(5, 4, 3)=5 is “caused” by each of the inputs [I wish I were kidding but see Figure 4 here]).

squareallworthy:

squareallworthy:

My son, who is six year old, woke up last night crying. I went in to ask him what was wrong, and he said he was scared. Scared of what? Scared of “vampire squid,” of all things. With glowing red eyes, a big cloak, and spikes.

So I told him there was no such thing as a vampire squid, and he said “yes there is, I saw it on my Octonauts show.”

And uh

…they’re real.

Rick James believed in drugs. As he’d said to the crew when he’d first assembled them: “Look at my lyrics to my songs. All of the songs are about drugs. They’re about women and about drugs, and they’re one and the same. That is the persona of this band.”

Continuing my Legion liveblogging I guess (previously)

The latest (third) episode was cool, but seemed like a diversion from the main story, which is apparently going to wrap itself up in only five more episodes?

What I’m most skeptical about them pulling off in only five episodes is any kind of real perspective shift that puts someone other than David in a protagonist-like role.  It’s weird, because Noah Hawley has done a whole bunch of interviews, both at the end of S2 and now at the start of S3, about how he’s so excited to step outside of David’s (implicit) perspective, to see him from the outside.  Maybe Syd’s been the real protagonist all along, he says.  Or, this new character Switch is great because she’s a vehicle for seeing David from the outside.

It’s weird because, however much the show succeeds or fails or its own terms, it definitely seems to be failing on this thing that the showrunning is externally asserting as his intent.  The show simply is written from David’s perspective and his alone; even when it critiques him, it’s the sort of critique you get inside your own head when you’re feeling down, from imagined versions of your friends/enemies/exes/et. al.

No one else has a clear personal story thread – or in some cases even a clearly delineated personality – that could stand on its own without David existing.  Everyone else reacts to him, and sometimes their reactions are disapproving or antagonistic, but they’re still reactions to him and are interesting to the writers mostly as such.  Who is Switch when she’s not being a vehicle for “seeing David from the outside”?  We know almost literally nothing about her.  For that matter, who is Syd without David?  What are her goals and dreams, her challenges, her baggage?

I’m sure there’s someone out there who would take me to task for not reading between the lines enough and picking up on how Syd is Actually A Real Character if you pay attention.  But, I dunno, it feels to me like saying Skylar White is as developed and engaging a character as her husband, just because she’s morally in the right to be wary of him.  She sure is.  But there’s a simple and infallible test here: would you watch a show only about Skylar without Walt?  Only about Syd, without David?  What would it even be about?  (We don’t even have enough information to judge whether such shows would be any good.  Maybe they’d be great!  After all, you can draw anything on a mostly blank canvas.)

I don’t have an entirely coherent thought here, but … OK, so the show is clearly in the same territory as something like Evangelion, where many of the specific events, characters, and sci-fi concepts are metaphorically “about” common and potent psychological experiences (heartbreak, feeling rejected, childhood trauma, yearning for an absent mother, these sorts of things), although it’s (thankfully) hard to come up with a single, rigid scheme of metaphorical correspondence that handles the whole thing and removes all ambiguity or specificity.

Hawley has said things to the effect of, oh, it’s a grandiose sci-fi depiction of a relationship and its surrounding emotions.  He talks about it being all about David and Syd, and whether they’ll get back together, as if it were a real-world romantic drama.  And I think we’re supposed to be able to engage with it on that level, viewing the fact that Syd is trying to kill David based on messages from the future, and that David has started a Manson-esque cult, as literalized and externalized versions of the things that go on in real people’s heads in real and familiar situations.

S3 makes a lot of sense when seen this way, but mostly – again – if the emotions are David’s.  The entire situation feels like some guy’s self-hating, post-breakup daydream.  Your ex is coming to kill you, because she’s convinced (perhaps rightly) of your immense moral turpitude (but at least she still cares about you, if only negatively!).  She is doing this in collaboration with this cool, in-command, effortlessly masculine guy (so different from you, you who are so twitchily desperate to prove your worth) who’s actually utterly horrible and abusive, who caused you great harm in the past, but who she and everyone else has somehow decided is okay at the same moment they decided you weren’t.  You’re now desperate for love, looking for it maybe in some bad ways, and also ruminating heavily on the past, especially on the other major female presence in your life … your mom.  Right?  It’s all very emotionally authentic, but the emotions are all David’s.  The other characters might as well be daydream phantoms, playing out their assigned daydream roles.

(In the real world, Syd would just be done with David at this point; appropriately enough, her only reason for continued involvement in his life is a message from the future, a writerly device that is both purely fantastical and purely arbitrary.  The dream must keep her in the frame somehow.)

And then there’s Lenny, the fourth part of this romantic/psychological square.  Lenny and Syd are two sides of a coin in the way that David and the Shadow King are two sides of a coin – implicit romantic rivals, opposed archetypes of the same gender.  The secondary members of each pair, Lenny and SK, aren’t judged by the same standards as the primary members: Syd et. al. make an alliance with the (present, existing) monstrous SK solely to oppose a (prophesied) monstrous David, and the same show that has Syd intoning Margaret Atwood quotes continues to treat Lenny (an addict serially abused by patriarch types) as wacky comic relief.  Really I think Lenny, much more than anyone else, has the potential to be the loose end that unravels the whole narcissistic, self-contained, David-centric tangle.  Either she’ll remain the wacky bad girl whose story doesn’t “matter,” which would dissolve the feminist pretensions of the David/Syd storyline whether the writers realize or not, or her storyline will eventually be treated with a gravity proportionate to the extent of her screentime, which will establish that there’s more in heaven and earth than David and his crush.  I expect the former but hope for the latter.

I don’t know what’s going on with the mental illness theme at all these days.  At most it’s come up occasionally as non-functional wallpaper – David’s not just a cult leader, he’s a cult leader who hallucinates occasionally! – which calls the central claim at the end of S2 (”you’re both”) even more into question, one would think.  But is that deliberate?  Is anything deliberate?  Who even knows.

dagny-hashtaggart:

I dreamt that the only work of printed narrative that survived in my post-apocalyptic tribe was a porn parody comic of Wagner called The Ring of the Nippleung. It was an extensive and monolithic piece, nearly as much so as the operas that inspired it. Over time it came to be revered as a sacred artifact, and elements of the story became woven into the cosmology and rituals of the tribe, such that our coming of age, funeral, and marriage rites all incorporated elements of niche Teutonic mythological opera porn.

(via dagny-hashtaggart)

nostalgebraist:

hello this is definitely me and not my wife, just here to tell you that i got my hair cut today and look SO FUCKING HOT it’s insane

evidence, which i clearly took myself and was not taken for me by my wife at all

hello this is definitely me and not my wife, just here to tell you that i got my hair cut today and look SO FUCKING HOT it’s insane

There’s this funny thing about Goodhart’s Law, where it’s easy to say “being affected by Goodhart’s Law is bad” and “it’s better to behave in ways that aren’t as subject to Goodhart’s Law,” but it can be very hard to explain why these things are true to someone who doesn’t already agree.

Why?  Because any such explanation is going to involve some step where you say, “see, if you do that, the results are worse.”  But this requires some standard by which we can judge results … and any such standard, when examined closely enough, has Goodhart problems of its own.

There are times when you can’t convince someone without a formal example or something that amounts to one, something where you can say “see, Alice’s cautious heuristic strategy wins her $X while Bob’s strategy of computing the global optimum under his world model only wins him the smaller $Y, which is objectively worse!”

But if you’ve gotten to this point, you’ve conceded that there’s some function whose global optimum is the one true target.  It’s hard to talk about Goodhart at all without something like this in the background – how can “the metric fails to capture the true target” be the problem unless there is some true target?

But attempts to define a “true target” for real-world behavior have a famous tendency to fail in a Goodhart-like way: they work decently in the collection of cases that inspired them, but when you try to derive their optima, you look at them and say “no, that’s not the target at all.”  (Parfit does this a number of times in the last section of Reasons and Persons, eventually admitting defeat.)  They can work when used as passive measuring instruments; they fail when used as optimization targets.

So there’s one sort of argument that goes, “overly aggressive optimization of the target you have in your head is bad, because it will tend to fail on the ‘actual’ target.”

But there’s another that goes further, saying “overly aggressive optimization of any target is bad, because it will tend to fail on whatever it is that you actually want; whatever it is that ‘you actually want,’ it cannot be a stated as a target, because the global optimum of any stated target will fail to coincide with ‘what you actually want.’”

Sometimes I’ll encounter someone in the online rationality world who’s like, I dunno, 19 years old and very good at math and has decided that clearly they should devote their life to the single goal of studying AI risk, because a misaligned AI could turn the whole world into paperclips or some such single-minded thing, and the expected value of averting that possibility dwarfs the expected value of anything else.  And I want to advise this person to sort of “chill out” in a way closely analogous to the way the hypothetical AI should “chill out” – a little more formally, I want to advise them to use a strategy that works better under model mis-specification, a strategy that doesn’t try to “jump to the end” and do the best thing without a continuous traversal of the intervening territory.

I said “a little more formally,” there, because that’s as formal as I know how to get without destroying the point.  Once you start talking shop about what the right strategy might look like, how my advice could cash out into actual “regularization terms” or “discontinuity penalties” or something, we’re back to defining an optimization target.  Which will be Goodhartable, like all the others.  There will be some kind of paperclips at the end of this road, and then we’ll have to decide whether we “bite the bullet,” and by then we’ve already lost.  What I would like is a general theory of why we are more satisfied with the results when we don’t go all the way down the road, including the road of defining our own satisfaction, and instead leave room for the black-box query: “does this seem like what I actually want?”

This all feels much less mind-bending, more mundane and obviously true, if I think about it in the context of practical delegation.  That is, how would I want people to behave if I – as in actual me, not a toy character like Alice or Bob – were managing a team of people on some project?  I wouldn’t want them to be ruthless global optimizers; I wouldn’t want them to formalize the project goals, derive their paperclip-analogue, and go off and do that.  I would want them to take local iterative steps, check in with me and with each other a lot, stay mostly relatively close to things already known to work but with some fraction of time devoted to far-out exploration, etc.

It’s totally intuitive to me that these behaviors are good for achieving what I actually want, which even I myself don’t know how to formalize – largely because I don’t know how to formalize it, and would prefer people (including myself) to just ask me things rather than consulting a function that’s supposed to approximate what I would say.  And this is the case we face, always, in the real world.  But I don’t know the right terms in which to make a formal argument about this situation, without destroying the aspects of the situation that depend on the pitfalls inherent in formalizing it.