nostalgebraist:

larger language models may disappoint you [or, an eternally unfinished draft] - LessWrong

WHAT THIS POST IS The following is an incomplete draft, which I'm publishing now because I am unlikely to ever finish writing it. I no long

lesswrong.com

In July I wrote a long, unfinished rant about language model scaling and evaluation on LW.

It sat in my drafts folder there for a long time, and it became clear to me that I was never going to go back and properly finish it.

So I went ahead and published it as-is, with a brief note at the start explaining the situation.

If you liked my earlier posts about GPT models – the ones that weren’t about Frank, I mean – this one may also interest you.

Some updates:

Janus (AKA moire), the author of generative.ink, left a long and interesting comment.

I replied with a long comment of my own, long enough it could be a post in itself.

—-

Surprisingly, to me anyway, this post has rapidly become my highest-karma post on LW, eclipsing even the Gary Marcus post.

So my worries from this thread with @the-moti were clearly unfounded. The new post takes a “boo AI!” tone, reiterates my original critiques of the GPT-3 paper, etc., but none of that caused it get a negative reception on LW.

In July I wrote a long, unfinished rant about language model scaling and evaluation on LW.

It sat in my drafts folder there for a long time, and it became clear to me that I was never going to go back and properly finish it.

So I went ahead and published it as-is, with a brief note at the start explaining the situation.

If you liked my earlier posts about GPT models – the ones that weren’t about Frank, I mean – this one may also interest you.

Answer key for the neuralblender vs. Colab post below the cut

Keep reading

nostalgebraist:

@iridescentsprout said, in replies to this post:

subjectively to me it seems like neuralblender gets better results than katherine crowson’s colab notebook

maybe i should get someone to test my guess out with a blinded comparison test

I ran a few prompts through the notebook and neuralblender, if you want to see if you can guess…

For each prompt, two of the images were generated by the notebook, and two were generated by neuralblender.

(I used default settings and a manual random seed in the notebook, and “Cronos” + “low quality” on neuralblender. I don’t know what the “quality” setting does, but only “low” is available for free.

I cropped all images to obscure the neuralblender.com logo.

This is a pretty small sample, but I didn’t want to spend hours and hours waiting for output)

—-

Prompt 1: “a painting of a green horse in profile”

Prompt 1, Image 1

Prompt 1, Image 2

Prompt 1, Image 3

Prompt 1, Image 4

Prompt 2: “a closeup of a horse’s head in profile. the horse has green hair.”

Prompt 2, Image 1

Prompt 2, Image 2

Prompt 2, Image 3

Prompt 2, Image 4

[last batch in a reblog due to tumblr image limit]

Prompt 3: “friendly artificial intelligence”

Prompt 3, Image 1

Prompt 3, Image 2

Prompt 3, Image 3

Prompt 3, Image 4

reachartwork:

nostalgebraist:

reachartwork:

i know nobody here cares but i’m gonna bitch here about it anyway since this is my AI art blog: it *really* bites my ass that neuralblender, the thing that has become astoundingly popular seemingly overnight for AI art, very transparently uses several pre-made AI Art code assets without any sort of credit towards the creators who spent months of hard work on that code.

I’m genuinely a little offended that clicking on “credits” brings you to a page where they ask you to spend microtransaction money on generating stuff from other people’s code notebooks (THAT YOU CAN ACCESS FOR COMPLETELY FREE, WITH MORE OPTIONS, THAT RUN FASTER, HERE’S NEURALBLENDER HYPERION AND HERE’S NEURALBLENDER CRONOS, BOTH FOR FREE THAT YOU CAN RUN AS MUCH AS YOU WANT, FROM THE ORIGINAL CREATORS), and not, like, a page crediting the original sources of their code.

just as to continue the gripe chain the website is also just lazy as hell, they didn’t even change the favicon from the default react icon, so the fact that neuralblender is exploding and the original creators of the work (and the people whose shoulder’s they are standing on; Advadnoun, RiversHaveWings, and DanielRussRuss for starters) don’t receive a lick of credit or acknowledgement really just bothers the shit out of me, that they can exploit the hard work of the developers in the AI art scene without crediting them.

Anyway, I would appreciate it if you felt like spreading this around and reblogging it. Here’s a whole list of all the dozens of variations of CLIP+VQGAN and other generative art resources that you can be using for free instead of shelling out for Neuralblender’s grift. you do not need to be good at programming for any of these.

I’m not necessarily cosigning the outrage on behalf of the original creators here, since I don’t know how they actually feel about it.

Also, they put their work under the MIT License, which technically permits stuff like neuralblender.

(Details: neuralblender.com is a react frontend for a closed-source [?] black box service hosted on AWS. If there is a copy of the licensed code, it would be in the service. Since no one can see the code of the service, it might well have the MIT License blurb in there for all we know.

Since the service is a black box, I don’t think we actually know it’s doing the exact same things as these Colabs. But, it is clearly using the same general methods: the frontend source refers to Hyperion as “vqganclip” and Cronos as “diffclip.”)

——

That said, it is pretty damn annoying IMO that neuralblender took publicly documented techniques like VQGAN+CLIP, slapped its own opaque brand names like “Hyperion” onto them, and made them available without providing any resources about what is actually happening when you use it.

Apart from any failure to credit specific people for their work, this is an educational failure. If you want to learn more about how neuralblender makes those cool pictures, neuralblender won’t tell you.

It took something that other people invented and explained, re-packaged it, and stripped away the explanation.

EDIT: here’s a very readable blog post from Max Woolf explaining how VQGAN+CLIP works.

(VQGAN+CLIP is what neuralblender calls “Hyperion” and is a widely-used technique. You can find plenty of other resources if you Google it.)

Oh, hi, Mx. Nostalgebraist! Love your work. Your autoresponder was one of the things that got me into making AI-generated content in the first place so its funny that this has now worked its way back in your direction.

The only thing I have to add is that I am one of the people that created a somewhat widely used CLIP+VQGAN notebook so at the bare minimum I’m a little peeved on behalf of myself, but I can’t really say for sure whether they’re using my notebook (Zoetrope) or someone else’s as a base because, as you mentioned, it’s entirely black-boxed.

reachartwork:

i know nobody here cares but i’m gonna bitch here about it anyway since this is my AI art blog: it *really* bites my ass that neuralblender, the thing that has become astoundingly popular seemingly overnight for AI art, very transparently uses several pre-made AI Art code assets without any sort of credit towards the creators who spent months of hard work on that code.

I’m genuinely a little offended that clicking on “credits” brings you to a page where they ask you to spend microtransaction money on generating stuff from other people’s code notebooks (THAT YOU CAN ACCESS FOR COMPLETELY FREE, WITH MORE OPTIONS, THAT RUN FASTER, HERE’S NEURALBLENDER HYPERION AND HERE’S NEURALBLENDER CRONOS, BOTH FOR FREE THAT YOU CAN RUN AS MUCH AS YOU WANT, FROM THE ORIGINAL CREATORS), and not, like, a page crediting the original sources of their code.

just as to continue the gripe chain the website is also just lazy as hell, they didn’t even change the favicon from the default react icon, so the fact that neuralblender is exploding and the original creators of the work (and the people whose shoulder’s they are standing on; Advadnoun, RiversHaveWings, and DanielRussRuss for starters) don’t receive a lick of credit or acknowledgement really just bothers the shit out of me, that they can exploit the hard work of the developers in the AI art scene without crediting them.

Anyway, I would appreciate it if you felt like spreading this around and reblogging it. Here’s a whole list of all the dozens of variations of CLIP+VQGAN and other generative art resources that you can be using for free instead of shelling out for Neuralblender’s grift. you do not need to be good at programming for any of these.

I’m not necessarily cosigning the outrage on behalf of the original creators here, since I don’t know how they actually feel about it.

Also, they put their work under the MIT License, which technically permits stuff like neuralblender.

(Details: neuralblender.com is a react frontend for a closed-source [?] black box service hosted on AWS. If there is a copy of the licensed code, it would be in the service. Since no one can see the code of the service, it might well have the MIT License blurb in there for all we know.

Since the service is a black box, I don’t think we actually know it’s doing the exact same things as these Colabs. But, it is clearly using the same general methods: the frontend source refers to Hyperion as “vqganclip” and Cronos as “diffclip.”)

——

That said, it is pretty damn annoying IMO that neuralblender took publicly documented techniques like VQGAN+CLIP, slapped its own opaque brand names like “Hyperion” onto them, and made them available without providing any resources about what is actually happening when you use it.

Apart from any failure to credit specific people for their work, this is an educational failure. If you want to learn more about how neuralblender makes those cool pictures, neuralblender won’t tell you.

It took something that other people invented and explained, re-packaged it, and stripped away the explanation.

EDIT: here’s a very readable blog post from Max Woolf explaining how VQGAN+CLIP works.

(VQGAN+CLIP is what neuralblender calls “Hyperion” and is a widely-used technique. You can find plenty of other resources if you Google it.)

platonic forms…..