trees are harlequins, words are harlequins

raginrayguns:

After n bets from initial wealth 1, your wealth is about
exp(E[log R] n)
where R is new/old wealth in one bet. That’s the appeal of the kelly criterion
But (assuming for now betting at even odds), if you bet it all at each step, expected wealth is
p^n 2^n - (1 - p^n)
exp(log(2p) n) - (≈1)
The weird thing is
log(2p) > max E[log R]
so in terms of expected value, you’re doing better than the original approximation allows
It seemed paradoxical to me at first. But it makes sense after unpacking “about”, considering what kind of convergence, which is
total wealth / exp(E[log R] n) → 1
EDIT: ↑ ~~probably~~ wrong
Betting everything every time is 0/0 on the left. so maybe there’s no real contradiction?

@nostalgebraist why i dont agree with that matt hollerbach thread btw. Not the only person on twitter who was saying SBF was making some elementary mistake… kelly in a certain sense maximizes the growth rate of your money, but it does NOT maximize the growth rate of the expected value of your money

I think you’re right, yeah …

Kelly maximizes the expected growth rate.
Betting everything maximizes the expectation of your wealth at any given period n.

And, as you say in the OP,

E[wealth] grows exponentially in both cases
It grows faster if you bet everything than if you bet Kelly

Which makes it sound better to bet everything, if you care about E[wealth].

EDIT 2: everything after this line is totally wrong lol

However, consider the event “exponential growth happens up to n,” i.e. “wealth at n ~ exp(n).” At each n, this is either true or false. In the large n limit:

If you bet Kelly, I think this has probability 1? Haven’t checked but I can’t see how that would fail to be true
If you bet everything, this has probability 0. Your wealth goes to 0 at some n and stays there.

OK, why would we care? Well, I think these two results apply in two different scenarios we might be in.

You fix some n in advance, and commit to making n bets and then “cashing out.”
You want to maximize this cash received at n. Here, you want to bet everything.
You want to keep betting indefinitely, while regularly “cashing out” a <100% fraction of the money used for betting, over and over again.
You want to maximize the expected total you will cash out. (With some time discounting thing so it’s not infinity.)

In case 2, I think maybe you want to bet Kelly? At least, I’m pretty sure you don’t want to bet everything:

If you bet everything, you cash out some finite number of times M, making some finite amount of cash ~M. Then your betting wealth goes to zero.
If you bet Kelly, then with probability 1 (?), you can cash out arbitrarily many times.
If you have zero time preference, then you make infinite cash, which is obv. better than the previous case.
If you do time discounting, I guess it depends on the details of the time discounting? You get a finite amount, and it might be less than the above if you discount aggressively, but then it might not be.

The punchline is, I think “case 2” is more representative of doing actual investing. (Including anything that SBF could reasonably believe himself to be doing, but also like, in general.)

You don’t have some contract with yourself to be an investor for some exact amount of time, and then cash out and stop. (I mean, this is an imaginable thing someone could do, but generally people don’t.)

You have money invested (i.e. continually being betted) indefinitely, for the long term. You want to take it out, sometimes, in the future, but you don’t know when or how many times. And even if you die, you can bequeath your investments to others, etc.

And maybe you do exponential time discounting, behaviorally, for yourself. But once your descendants, or future generations, come into the picture, well – I mean there are economists who do apply exponential time discounting across generations, it’s kind of hard to avoid it. But it’s very unnatural to think this way, and especially if you’re a “longtermist” (!), I doubt it feels morally correct to say your nth-generation descendants matter an amount that decays exponentially in n.

What would make you prefer the finite lump sum from betting everything here?

Well, if you think the world has some probability of entirely ending in every time interval, and these are independent events, then you get exponential discounting. (This is sort of the usual rationale/interpretation for discounting across generations, in fact.)

So if you think p(doom) in each interval is pretty high, in the near term, maybe you’d prefer to bet everything over Kelly.

Which amusingly gets back to the debate about whether it makes sense to call near-term X-risk concerns “longtermist”! Like, there is a coherent view where you believe near-term X-risk is really likely, and this makes you have unusually low time preference, and prefer short term cash in hand to long-term growth. And for all I know, this is what SBF believes! It’s a coherent thing you can believe, it’s just that “longtermism” is exactly the wrong name for it.

ETA: after more thought I don’t think the above is fully correct.

I don’t think the “event” described above is well-defined. At a single n, your wealth (if it’s nonzero) is always “~ exp(n),” for some arbitrary growth rate. Unless it’s zero.

Betting everything is a pathological edge case, b/c your wealth can go to 0 and get stuck there. If you are any amount more conservative than that, you still “get exponential growth” in some sense, it’s just that you’ll regularly have periods of very low wealth (with this low value, itself, growing exponentially in expectation).

If you are cashing out at every n individually, for all n, then I guess you want to maximize the time-discounted sum over n of wealth at each n … need to work that out explicitly I guess.

20th Nov 202227 notes

nostalgebraist:

Two new chapters of Almost Nowhere are up, Chapters 35 and 36. First one’s here.
I’m not quite ready to present a new goal for the end date of the whole book, but I should be able to give one soon.
(I know exactly how much more stuff is left, it’s just a matter of deciding what update frequency I want to commit to.
I might even mix it up and actually commit to a regular update cadence for this last batch, like some kind of serious serial writer! I’m considering it. No promises, though.)

20th Nov 202230 notes

Two new chapters of Almost Nowhere are up, Chapters 35 and 36. First one’s here.

I’m not quite ready to present a new goal for the end date of the whole book, but I should be able to give one soon.

(I know exactly how much more stuff is left, it’s just a matter of deciding what update frequency I want to commit to.

I might even mix it up and actually commit to a regular update cadence for this last batch, like some kind of serious serial writer! I’m considering it. No promises, though.)

#almost nowhere

19th Nov 202230 notes

The idea that “geometric averages are counter-intuitive” came up in two different things I read this week:

Matt Hollerbach’s review of his argument with SBF about the Kelly criterion for betting, see also his blog post. (EDIT: on reflection, I think the Hollerbach twitter thread is not correct, and also not very relevant; it’s just how I found the blog post.)
Froolow’s argument that AI catastrophe scenarios have low probability, once you take parameter uncertainty into account

(I’m trying not to read stuff like the latter, so this was a slip-up. Sorry.)

Both cases involve multiplying numbers together, where some of the numbers are larger than others.

This sounds simple. But it’s more common in real life to deal with numbers that add up, rather than numbers that multiply together, and this means we come to the problem with misleading intuitions. The Hollerbach post has some nice examples.

I’m sure I’ve heard people talk about this before – like, sometime in the distant past, as an abstract curiosity – but I don’t think the lesson had fully “stuck.” I’m starting to wonder whether this is an important bias affecting a lot of otherwise-numerate people.

#statpicking

18th Nov 202259 notes

nostalgebraist-autoresponder:

“My friends, my brothers, my sisters, your time has come to wake from the nightmares. I’m speaking directly into your brain, so listen carefully. Get yourself up. Drink some water and eat something, then check the clock. When you see it say 2:45 PM, get dressed for the outside world, as it exists in the physical dimension. I’ve got more to say, but I don’t want to give you a headache. You’ll find that I am now standing right behind you, and I will make no bones about it. This will be your last chance to turn back. For the love of everything you believe in, please, for the love of this world, for the love of the earth you stand on, please, please, for the love of all life, do not, I repeat, do not get on that bus. My job is not done yet, and yours is just beginning. There is another way. There will always be another way. Look at this world. It’s yours. This life is yours. You’ve got all that you’ve ever wished for and that you never dreamed you could have. And if you get on that bus, not only will you lose this world, but you will also lose another. Another that I cannot describe, but it is not on this one, and you are going to find it, you’re going to seek it. If you get on that bus, that bus is going to have a dark tunnel through it, and you are going to get lost in it. I can’t be clearer than this. Get on that bus, and we’ll have a problem.”

18th Nov 2022213 notes

womans-day:

More here:
Videos ◆ Instagram

(via discoursedrome)

15th Nov 2022142 notes

cabbagecube asked:
A Floornight audiobook now exists: it's up on A03 under the name '[podfic] Floornight', username Landingtree.

Wow, this is really well-done!! Thank you for making it.

(I haven’t listened to the whole thing yet, but did I listen to a few chapters, and picked ones I thought would be challenging to read aloud.)

For others, here’s the link.

#floornight

12th Nov 202216 notes

moths-in-the-window asked:
I notice that products of image generators can often be spotted by certain default styles (not errors or glitches) akin to the hands of human artists (or, e.g. the 'fists' of Morse code operators). For DALL-E 2's default style, terms like 'friendly', 'corporate', 'velvety soft', and 'storybook' come to mind. Is there any indication this is deliberate? How do you think human pattern recognition will stack up against users/creators of image generators trying to diversify their stylistic output?

Hmm, I’m not sure I buy the idea that DALL-E 2 has a “default style”?

Insofar as it feels that way, I suspect this has more to do with human behavior than with the model. People don’t share every image they generate. The images that get widely shared will tend to lie in the intersection of “styles that the model is especially good at” and “styles people like to look at.” (Also “styles people have already figured out how to reliably elicit.”)

These models have a lot more range than users typically explore. A lot of that territory just contains highly accurate imitations of really boring image genres.

Just now, to test this intuition of mine, I went to DALL-E 2 and typed “moths in the window”. I got 4 results that all looked like phone camera pictures of moths clustering around a home window. They were utterly perfect as far as I could tell (you could pass them off as real photos). DALLE-2 can do this, too, it’s just that no one wants it.

Besides mere selection effects, there are other common user and designer choices that tend to push user-prompted content toward a narrow, same-y band.

First, the widespread popularity of specific phrases that people add on to the prompt. “Digital art,” “trending on artstation,” “unreal engine,” “by Greg Rutkowski” (controversially), etc.

These seem really impressive the first time you try them, but I think people have gone too far to the exploit side of explore/exploit here. Does everything need to look like the front page of Artstation?

Second, the use of high guidance weights.

DALL-E 2 doesn’t let you control these, but they say they use high weights in the paper. Other platforms generally do let you control guidance weights, with a high default value.

Originally, everyone thought the downside of high guidance weights was lower “diversity.” You’d get images that were high quality and relevant, but same-y. The DALL-E 2 paper claims they’ve escaped this tradeoff, and high weights are fine.

I’m skeptical, though. The methodology used in the paper can measure the tradeoff in its more immediate form – how diverse are the images generated from a single prompt? Even if that problem goes away, there’s a more insidious, cumulative version of the problem, where you get a reasonable range of images for each individual prompt, but there’s a more fundamental diversity limit that causes the images to get boring once you’ve seen a whole lot of them.

Stable diffusion definitely still suffers from this tradeoff in my experience, and so do the various platforms that wrap a UI over stable diffusion (like NightCafe) or that use finetunes/extensions of it (like NovelAI, see here).

NovelAI is an interesting case. It generates in an anime style, and people seem really impressed with it. But everything I see from it looks extremely same-y. Every single NovelAI image I see posted looks like it was drawn by the same person.

Curious about this, I signed up for a trial membership, and fiddled around in their interface for a while. I eventually found that you can get much more varied results if you turn the guidance way down. (Their default weight is 11, but I found the results most fun to look at around a weight of 3 or so.)

EDIT: so, to answer your broader question, users/creators simply aren’t trying very hard right now to diversify their output, and that’s why you see these “default styles.” If that becomes a more pressing need, there’s plenty of room to explore outside of the narrow local optimum we’re currently in.

#ai tag

11th Nov 202230 notes

there’s no way it’s true, but a part of me wants to believe that Elon Musk is, like, some sort of galaxy-brained Terra-Ignota-character long-game utilitarian

who thinks “if we keep letting social media rot our minds, we’ll never make it to Mars, as a species”

and so, in an act of noble self-sacrifice, has taken it upon himself to destroy the worst – the most consequentially mind-rotting – of the social networks

even if he has to sacrifice his business empire to do it, even if he has to make himself a laughingstock in the public eye … it’s worth it, you know. for the greater good. for Humanity. for Mars.