Still have to make a longer post on Frank’s new image generator, but a few quick comments:

- Many thanks to everyone I talked to about this project in the EleutherAI discord! Special thanks to RiversHaveWings for suggesting I try diffusion for this problem + various helpful tips along the way.

- As far as I know, the image generator I made for Frank is the first neural image generator anyone has made that can write arbitrary text into the image!! Let me know if you’ve seen another one somewhere.

- The model is a text-conditioned denoising diffusion model. Or rather two of them, a 128x128 base model and a 256x256 upsampler.

- Coincidentally, just 3 days ago, OpenAI announced/released their own text-conditioned denoising diffusion model. I guess it’s an idea whose time has come! Their model is structured a little differently, and makes way better-looking images, although with the writing-text aspect.

- My code for this model is a heavily modified fork of OpenAI’s improved-diffusion repo. It’s on this branch. The Files Changed view here gives a clearer sense of what I changed. (Caveat: it’s extremely hacky research code)

nostalgebraist:

nostalgebraist:

Working on a new Frank feature for the first time in forever… this one involves generating images ;)

(don’t hold your breath, early tests are showing promise but it’ll take a while to get in a release-worthy state)

With luck, I might have this released tonight…

all the pieces are in place, though there might be more bugs in my hacked-together MVP code than I have one day’s patience for for, we’ll see

The new image generation feature is live now!

You can see an example of the feature in action here.

I’ll write a post later explaining how the new model works.

This feature is the culmination of something like a year of hard work, if you count all the image generation architectures I tried that didn’t work. Now I’ve finally came up with one that gets the job done. (It’s similar to OpenAI’s just-released “Glide” model, although I came up with the idea independently about a month ago.)

nostalgebraist:

Working on a new Frank feature for the first time in forever… this one involves generating images ;)

(don’t hold your breath, early tests are showing promise but it’ll take a while to get in a release-worthy state)

With luck, I might have this released tonight…

all the pieces are in place, though there might be more bugs in my hacked-together MVP code than I have one day’s patience for for, we’ll see

Working on a new Frank feature for the first time in forever… this one involves generating images ;)

(don’t hold your breath, early tests are showing promise but it’ll take a while to get in a release-worthy state)

I’d prefer if people didn’t send NeuralBlender images to Frank in asks anymore.

Frank can’t see anything in them except the text, which generally isn’t very interesting on its own.

If you want to share NeuralBlender images, do it on your own blog.

I’ve configured Frank to flag asks for content moderation if she recognizes the “neuralblender.com” logo in there.

Image asks in general tend to be spammy and low-quality, and it m prefer to disable them entirely. I tried turning off the “Allow asks with media” setting, but it doesn’t seem to actually do anything.

eightyonekilograms:

nostalgebraist:

If anyone’s wondered what Frank’s logs look like, here’s a long sample covering sometime yesterday until now.

I often “watch Frank” by looking at this stuff as it scrolls down my screen in real time. (It’s relaxing, kinda.) This is also what I’m looking at when I’m trying to fix a bug.

(Note: I haven’t scrutinized this for content at all – for the wary, there is occasional unreviewed/uncensored GPT output in there, plus a large quantity of actual tumblr posts from people Frank follows or talks to)

“I don’t even see the code, all I see is writing prompt, shitpost, anon hate…”