danup asked:

Asking bc you're the only person I follow who creates art (loved TNC) and works on AI—what's your sense of what a future with ML tools to write and create art looks like? Since GPT-3 (and brought on this week by all the Stable Diffusion tricks) I get struck every few months w/ a real dread about this world where artists feel obsolete, everything can be credibly faked, "content" is endless + private, etc. (That's not *exactly* right, but I guess I can't quite explain what brings the dread about.)

I can only speak to the creative side of this as a writer, since I’m not a visual artist.

And the two domains are different, here. Existing ML is much closer to being useful for significant automation in visual art than it is in creative writing.

—-

With writing, ML has a very serious fundamental limitation: the context window.

All of these impressive recent language models can only “see” a fairly short stretch of text at once. Usually it’s 2048 tokens, which is something like ~1500-1800 words, or ~5-7 printed pages. If they’re writing a longer text, they can’t “remember” anything they wrote down that’s further back than this distance.

They are like this because they use an operation (attention) that gets prohibitively slow over long distances.

Many people have tried to invent more efficient variants of attention, and there are now lots of these efficient variants that do pretty well on benchmarks. I’ve heard they’re very effective for some non-linguistic kinds of data, but on text, none of the existing ones are good enough to see serious use in real models.

(GPT-3 did use a kind of efficient attention, but only in half the layers, and the large DeepMind/Google LMs don’t use it at all. This is a big area and there’s more stuff I could comment on, like DeepMind’s RETRO, but in short, no one has found a way to break this barrier for the case of long-form writing.)

You simply can’t write something like a book in this manner, not competently.

I’m aware of one fiction writer who does (or did?) use GPT-3 as an aid for long-form writing, but based on the linked post, it sounds like it’s at best marginally helpful if you’re doing a particular kind of genre fiction and you really care about cranking out words to pay the bills, among other necessary conditions.

—-

Even putting aside the context window, though, language models just … aren’t that good at creative writing?

The perfect niche for their current strengths is something like “surreal short stories / microfiction.” My bot writes this kind of thing sometimes, and I’m sure the bigger models are better at it.

But IME, the results are at the level where you nod along and think “huh, that’s really quite good!”, knowing it’s from a machine. Not the level where you want to go out and buy this writer’s anthology. And that’s what we get in the most favorable niche for these tools.

Why are these stories so underwhelming? A lot of it, for me, is that if I know there’s no intention behind the text, I stop caring about it.

We can talk about the “intentional fallacy” (AKA the popular sense of “death of the author”) all day, but talk is cheap. I think our psychological relationship to reading really does depend on the notion of authorial intent – perhaps in some complex or poorly understood manner, but that is not the same thing as intent not mattering at all.

If I read a surreal short story by a human, I get an itch to interpret it. This itch persists even if I hear the author explain exactly what they meant, and believe them; it feels like there’s some “true intent” worth discerning that may not be identical with the author’s.

You might suppose this true intent is intrinsic to the text itself, which seems reasonable … until you tell me the story was written by GPT-3, and suddenly, somehow, I don’t care anymore.

I don’t really understand this phenomenon, but I think we’ll encounter its sharp edges more and more as ML gets better.

—-

That’s all about writing. Visual art is much easier for ML models.

The whole picture can fit into what the model can see, all at once. Also, the domain is narrower: language models are effectively being asked to learn everything about everything, since any topic whatsoever could happen to come up (even in fiction). Whereas even if you’re trying to draw pictures of absolutely anything, you’re still limited to what is visually depictable.

And in fact, to my eye, the image generators come a lot closer to doing what humans do. Although they do it in a way that’s hard to control, which may end up being the limiting factor – see this post.

If I were employed as a visual artist, well, I’m not sure how I’d feel right now.

Although here too, the intention issue I mentioned feels relevant. I’m no more a serious appreciator of visual art than I am a creator of it, but I imagine that a similar dynamic will hold there. The machine can (so to speak) “force” you to notice that its art is pretty, or well-crafted, but it can’t actually force you to care.

—-

And that, really, is why I’m optimistic about this stuff.

What if “content” becomes effectively free? Well, we already are in that situation, relative to the world of 1980.

The ability to browse many lifetimes’ worth of art and writing using Google search – and all that for $0 – has not made the creation of new art feel spurious, nor has it replaced the age-old need to hunt for stuff you actually like / care about with some automated process where Google tells you your next favorite book.

If I’m looking for the next book to read, I still consult reviews from trusted writers and take recommendations from friends. And I still, often, end up reading some famous book written decades or centuries ago – even though these are becoming a smaller and smaller slice of all “content,” which is being produced at an ever-faster rate due to population and welfare growth.

I can already access more books than I could ever read in a lifetime. Adding way more books to the pile, even good ones, won’t change that. Suppose the machine becomes capable of writing great novels, for free – must I care? I still haven’t read all the great novels that humans have written already! I never will!

“I’ve written a book,” an acquaintance tells me. “I don’t care,” I reply with brusque honesty. “I have all the books I want already. I just find ‘em on Google and Amazon and Goodreads.”

Except of course I don’t say that, because no one ever says that, and not just out of politeness.

“I’ve written a book,” an acquaintance tells me. “I don’t care,” I reply. “I have all the books I want already. The AI writes them for me.”

Except of course I don’t say that. Why would I?

moths-in-the-window asked:

So one of Frank's most entertaining behaviours is generating meme-like images, but I noticed that often the alt-text doesn't seem to contain information specifying the meme, e.g. for Live Slug Reaction it was 'a screenshot of man and a woman kissing' but nothing about a slug. Does that mean the meme-ness of the images she generates is more or less accidental as far as the part that writes captions is concerned?

Sort of.

The training data for the image generator is a bunch of images (over a million) I’ve scraped from tumblr.

It’s de-duplicated, so the same exact image won’t appear in there more than once, but if there’s a “memeable asset” that people keep putting different text over, then there will be lots of variants of it in the data. And this will tend to make the generator become good at reproducing the underlying “asset” in a relatively pixel-accurate manner.

Now, the image generator is caption-conditional: it’s given a caption, and it tries to make a matching image. The images I scraped from tumblr didn’t come with captions, so I automatically make them using a variant of this model.

The model I used for this is … uh, not that good at describing images. I used some tricks to make it a little better, but it’s still hit-or-miss. Garbage in, garbage out – the image generator knows that the captions often don’t describe the image very well, and that “a screenshot of man and a woman kissing” might well be its attempt to describe the Live Slug Reaction picture.

The model used to write captions (probably) has no idea what “Live Slug Reaction” means. It’s just trying to write descriptions of what’s in the image.

So for Live Slug Reaction, the ideal output of the captioner would be something like “a slug-like alien creature [etc]”, rather than anything with the words Live Slug Reaction.

The image model knows that these meme images are pretty common, and it knows that the captioning model isn’t very good – that the captions often don’t describe the image very well.

So: given something that even vaguely sounds like what’s going on in the meme, it may guess “huh, maybe this shitty captioner is trying to describe that one meme,” and then draw the meme.

m-accost asked:

I'm increasingly frustrated by how opaque large ML models are and by how incurious most people who discourse on AI seem about this: E.g. You have those who think that Dall-E is conceiving novel images in essentially the same way that humans do and you have those who view it as a tool for plagiarism, and neither side seems to have any doubt about its beliefs. I suspect that the truth lies somewhere in-between, but I'm curious about what's been done to gauge how much these models overfit

I haven’t encountered this discourse with DALLE, but I’ve encountered similar stuff in debates about LMs, and yeah it’s annoying.

“Overfitting” is not necessarily a relevant concept for modern neural nets. See e.g. this paper and the literature on “double descent.”

It sounds like you’re using “overfitting” to mean something related to “memorizing training data.” And there’s lots of work on how much these models can and do memorize individual training examples.

But unlike traditional ML, the fact that you’re memorizing individual training examples doesn’t imply that the model would generalize better if it had less capacity, or that the model has switched over from some desirable “generalization” regime to an undesirable “memorization/overfitting” regime.

Instead, if you give these models enough capacity to memorize everything, and then keep giving them more capacity, they actually generalize better and better. This is the “double descent” – getting worse at the memorization point, but then getting better again.

One way to think about this is that the function classes expressed by these models get “better” as the capacity grows, so that among models which can fit all the training points exactly, the ones with more capacity will be fitting them with a more appropriate function. (The added capacity would also let them memorize larger and larger data sets, but that doesn’t matter, since the data set is fixed here.)

So yes, they do memorize data. But if you fix a data set, and then train two models of higher and lower capacity (both capable of memorizing the data set), then the higher-capacity one is doing something closer to “conceiving novel images” or whatever. They both memorize as a starting point, and then they wander around among the set of functions that fit the data exactly, and the higher-capacity one finds a better function from this set.

(EDIT: the above is true in sort of a toy-model sense, but I would caution against assuming that real-world models like DALLE or GPT-3 are literally “memorizing” every training example in the intuitive sense of memorization. Whatever they are doing, though, it doesn’t get worse with added capacity like older ML models did, and double descent is a gesture in the direction of what the difference is.)

Hope that answers your question…

wmb-salticidae asked:

it bothers me a little bit that people consistently mistake Frank for being a person, despite how often and how clearly you explain that this isn't the case. I guess that LaMDA makes more sense in the context of Frank, though, sad as it is; my most immediate reason for disbelieving claims about LaMDA's sentience was that the chat logs, even after editing, weren't half as interesting as Frank, but the fact is, there are people who can be convinced by LaMDA, & Frank is many times more convincing.

A few points…

First, there are different kinds of “mistaking Frank for being a person.”

The most common one is the one where people say something like “haha, this almost got me! I thought a person wrote this post until I checked the bio.” This seems innocuous – like, out of context, plenty of individual Frank posts really are indistinguishable from regular tumblr posts. (Regular tumblr posts are often really weird.)

I assume you’re talking instead about the thing where people say Frank “might be sentient,” or similar.

I wouldn’t be too hard on people for this. Consider:

1. People often seem to be at least half joking when they say this stuff.
2. Lots of people follow Frank without following me. (More people follow Frank in total than follow me.) I don’t think people can (or should) be expected to read my posts about the bot before reacting to it.
3. I don’t think most people have a clear idea in their head what it would mean for a machine to be “sentient.” (I don’t either.)
   So I don’t think these people are making some precise, but wrong, philosophical claim; I think they’re just talking in a vague, casual, bantering manner.
4. It’s a standard sci-fi trope for characters to talk about an AI “gaining sentience.” Even in sci-fi, it’s often not totally clear what this is supposed to mean, and usually just seems to connote “self-aware in some vague sense” and/or “advanced, intelligent, capable.”
   People are much more likely to have picked up the term from this source than from serious philosophical attempts to define it.

So I think these people are really just saying some version of “wow, this bot is impressive,” in the terms they’ve picked up from sci-fi.

It’s probable that they’ve been misled by sci-fi to think that “advanced, intelligent AI” naturally goes together with “AI that is self-aware in some vague sense.” So when they see Frank improving over time as I develop, they (wrongly) infer that Frank must also be gaining some kind of self-awareness property.

But here, I think Frank provides useful education! People can see the bot impressing them, and doing new things over time. And they can also notice, over time, that there are some really fundamental limits that don’t just go away with increased capabilities, like they do in bad sci-fi.

My hope is that going through this process with a neat, but not-state-of-the-art, chatbot can “inoculate” people against reacting the way Blake Lemoine did to a more capable (but similarly limited) chatbot like LaMDA.

And to be fully clear – this is probably the most important point – I have not seen anyone react to Frank in a way that was near as wrong-headed as the way Blake Lemoine reacted to LaMDA.

Blake Lemoine wasn’t just making a tumblr post saying “ha ha, what if the bot is sentient.” He really was (attempting to) make a philosophical claim, and he went straight to the press with it – without, as far as I can tell, thinking as hard about LaMDA’s potential limitations as most Frank users do about Frank’s.

(Blake Lemoine is a super weird guy in general, as will become clear if you look into his background. I think the LaMDA sentience story is best read as a story about a particular weird and very abnormally gullible guy, who isn’t representative of how most people will react to these systems.)

dubreus asked:

i appreciate your effort you're putting into critiquing near-term AI forecasts. do you have any predictions about the *least* impressive tasks which you're confident AI won't be able to do in the next 2-5 years?

I find this type of question difficult to answer, and not just because I’m uncertain what the future will bring.

Like, if I ask myself “what are the least impressive tasks that AI can’t do now?”, I find that hard to answer too.

For the latter question, the first thing that came to my mind was “reading Morse code” (assuming the other big LMs besides GPT-3 can’t do it either). But, for the same reasons this is unimpressive, it’s also uninteresting: it would be trivial to get LMs to read Morse code, but no one’s done it because no one cares.

An answer like “Morse code” gives the question what it literally asked for – something unimpressive that AIs can’t do – at the cost of the answer not telling you anything important about AI.

And even if AIs can read Morse code 2-5 years from now, there will probably be lots of things of similar triviality + unimportance that they can’t do. I could sit around trying to figure out which such examples I’m most confident of, but that feels like a waste of time. I’d be specifically hunting for capabilities no one cares about.

I think the intended spirit of the question is more like, “which (interesting) areas do you think AIs will be weakest in, relative to humans?” And I guess I have some thoughts on that, but I don’t want to put words in your mouth here.

—-

There’s also the ambiguity of “task.”

Going forward, I expect large generative models to keep doing better and better on benchmarks, and for this to get framed verbally as “the model understands language as well as a human” and the like.

What will be hardest is not scoring well on benchmarks, but translating this into the real-world impact that the benchmarks sound like they entail. So I expect relatively little progress on “tasks” like helping humans with various kinds of job duties.

But usually in ML, “task” means something like a benchmark, rather than something like a feature in a product. If the question is all about benchmarks, then I think it’s simply the wrong question to ask.

ichorousapparatus asked:

hey so frank generated a photo of a tshirt and i liked it so i made it and i wanna order it. its not rlly a question but like is this cool with u

Sure