mind viruses about body viruses

@slatestarscratchpad (thread clipped for length, responding to this)

First of all, thank you for the the thoughtful and charitable response.

Re: my overall message

Second of all, yeah, my post is not too clear on a lot of things and went through some message drift as I was writing.  The message I had in mind when I started was 100% about being more careful in curation, not about doing independent work.

Then I ended up spinning this big theory of why curation was not being done carefully.   Roughly, I hypothesized that – although there is a large volume of material being produced – very little of it would qualify for curation under normal circumstances.  Either because the quality is too low (e.g. obviously bad amateur pet theories) or because the format is too indigestible (e.g. convoluted high-context twitter threads that are hard to even permalink clearly).  Hence, some of us are lowering our usual curation bars just to let anything through.

Since “maybe don’t curate anything at all” felt underwhelming as a recommendation, I added a suggestion that we could try improving the supply side.  I didn’t really mean that more independent work of any sort is good, since as you say we are glutted with independent work.  I meant more independent work good enough to pass even “peacetime” thresholds for curation, stuff that very clearly shows its work, collects scattered expert observations into an easily digestible whole without oversimplifying, doesn’t rely on misleading inflammatory phrases to get your attention, etc.

(I do think your masks post falls in this category, and thank your for writing it.)

Maybe the supply-side point is wrong – maybe, as you say in your final para, there are enough good takes out there and the limiting factor is finding and spreading them.  I don’t have a strong opinion either way there.  What I do see is the signal-boosting of stuff which I personally find “iffy” but would maybe provisionally endorse in the absence of anything better.  If better work is being done, we really need to start curating that instead.  If not, then whoever is capable of produce better work needs to produce it, and then we need to curate it.

Re: my objections to recent SSC posts (big picture)

Like I said, I got carried away with grand theorizing as I wrote.  But the original impetus for me writing the post was very simple and concrete: I read the “Hammer and dance” section in your latest post and was frustrated by it.

Taken together with my frustration about your previous discussion of Bach, it felt like there was a pattern where you were both sharing and endorsing some things without clearly understanding them or being able to summarize them adequately.

I worried that these endorsements would aid an information cascade.  But also, “an information cascade is happening” seemed like a relatively charitable option among potential explanations for the pattern.  That is, conditional on “Scott is endorsing this thing he doesn’t really understand,” your action is more defensible if it’s supported by an impression that many independent observers are converging on the same endorsement, rather than if it’s completely based on your (by hypothesis, insufficient) personal assessment.

But this “more defensible” reading still isn’t defensible enough.  When these decisions are being made on intellectual trust, and some of that trust is not well founded (e.g. the trust I suspect many people place in SSC on this topic), we are likely to see quick formation of consensus far beyond what is epistemically licensed.

Okay, you might say, but what’s the alternative – just sharing nothing?  I agree with what you wrote here:

If I stay inside and don’t spread the actual coronavirus, I’ve trivially made everyone’s lives better. If I shut up and don’t spread any intellectual memes, then that just means that people’s thoughts are being shaped by the set of everyone except me. This is good if I’m worse than average, bad if I’m better than average. Or to put it another way, I’m making a net contribution if I signal-boost true/important things disproportionately often compared to their base rate […].

This is true if we model you as a “pure transmitter” who propagates ideas without modifying them in the process.  What I’m worried about, though, is ideas acquiring an ever-growing halo of credibility/consensus as they’re endorsed by individually credible people who cite all the other credible people who believe them, etc.

As I’m writing this, I realize this is a key thing I didn’t adequately emphasize in OP: the concern isn’t about mere passing on of information, it’s about the side effects that can occur as it’s passed on.  This means my metaphor of an “information epidemic” just like a disease was, although entertainingly meta, not actually accurate or helpful. 

I would be happy with a bare link to Pueyo’s or even Bach’s pieces, without explicit endorsement, perhaps just with a note like “seems interesting but I can’t evaluate it.”  (You have said roughly that about many other things, and I approve of that.)  I would also be happy with a detailed “more than you want to know” type analysis of any of these pieces.

What I am not happy with is a link with a rider saying you endorse it, that the smart people you’re reading endorse it, that it’s the new consensus, etc., without an accompanying deep dive or evidence of good individual vetting.  When iterated, this is a cascade.

Re: my objections to recent SSC posts (specifics)

Here’s are the concrete cases I object to, which made me think I was seeing a bad pattern.

First, here is how you originally glossed Bach’s article in the 3/19 links post:

An article called Flattening The Curve Is A Deadly Delusion has been going around this part of the Internet, saying that it’s implausible to say R0 will ever be exactly 1, so you’re either eradicating the disease (good) or suffering continued exponential growth (bad) without a “flat curve” being much of a possibility.

I won’t explain here why this is not accurate, since I already wrote an SSC comment to that effect.  Shortly after I posted my comment, you modified what’s in the post to say something more accurate which also sounded much like the gloss I wrote in my comment.  (I guessed that this was a reaction to my comment, although I could be wrong.)

Although I appreciate that you made the correction, the damage was done: I was convinced that you had shared the Bach article without understanding it.  If you later came to understand it and still thought it was share-worthy, that’s fine in itself, but understanding was apparently not necessary for sharing.  Further, this called the other Coronalinks into question a la Gell-Mann amnesia: if there’s an error in the one case I happen to have already scrutinized for my own reasons, there are likely some errors in those I haven’t.

Then, in the 3/27 links post, you wrote:

I relayed some criticism of a previous Medium post, Flattening The Curve Is A Deadly Delusion, last links post. In retrospect, I was wrong, it was right (except for the minor math errors it admitted to), and it was trying to say something similar to this. There is no practical way to “flatten the curve” except by making it so flat that the virus is all-but-gone, like it is in South Korea right now. I think this was also the conclusion of the Imperial College London report that everyone has been talking about.

This appears to be an explicit endorsement of the entire article, except the “minor math errors.”  That is, “it was right (except for the minor math errors it admitted to)” implies “everything that was not one of the minor math errors was right.”

I don’t know how to square this with your comments on Bach in the post I’m responding to (I broadly agree with those comments, FWIW).  You describe being initially confused by Bach’s article, then only understanding it after reading other things that made the same point better.  If Bach’s article is confusing, and there are better substitutes, why continue to tout Bach’s article as something “right” and worth reading?

Perhaps a more useful way to say that is: it sounds like you are doing two separate things.  You’re reading articles, and you’re forming a mental model of the situation.  The model can update even when re-reading the same article, if it happens you come to understand it better.  If Bach’s article confused you, but it and things like it eventually caused a useful update to your mental model, then the valuable piece of information you have to transmit is the content of that model update, not the confusing and misleading texts from which you eventually, with effort, distilled that update.  Sharing the texts with endorsement will force others through the same confusion at best, and permanently confuse them at worst.

Remember, there is a lot of stuff in the Bach article beyond the one fact about how low the line is.  I too did not know how low the line was until I read Bach, and in that sense Bach’s meme – including its inflammatory, thus viral, title – was a kind of success.  But it’s a success at transmitting one fact which we didn’t know but every epidemiologist did.

We can take this fact on board and proceed, without – for instance – co-signing an article that explicitly advocates lockdown to stop geographic spread (i.e. creating effectively disease-free zones) as the only solution that will work, something not recommended in any of the ICL or Harvard papers, insofar as I’ve read and understood them.

Closing comments

I realize this is likely to sound like I’m picking nits with phrasing, or perhaps like fixating on a case where you said I was wrong and bloviating until you concede I was “right.”

If I’m kind of unduly fixated on Bach’s article, well … I guess I just think Bach’s article was really bad, although it happened to teach many of us a 101-level fact for the first time.  I may be more confident in this judgment than you, but it doesn’t sound like you were incredibly impressed either – Bach was the first person you saw saying a true thing you didn’t understand until people said it less badly.  

If the best sources for basic information are this polluted with badness, then the supply-side is really messed up and someone less inadequate needs to step up and fix it.  Meanwhile, we should acknowledge the badness and accord no points for merely showing up, because that will mislead people and redistribute a maxed-out attention budget towards the consumption of misleading material.

Or, if there are better sources out there, they really need to be boosted and actively suggested as substitutes for their worse counterparts.  Until Carl Bergstrom gets a Medium account, the best distiller/synthesizer available who writes in a digestible format might well be Pueyo, and his confidence + lack of domain background make me wary.  And he’s the best – there are worse ones.  In relative terms these people may be the best we have, but absolute terms are the ones that matter, and the ones we should apply and communicate.

You are already forming your own model, distinct from these writers’, and in my opinion almost certainly better.  That model could be valuable.  Promoting worse models as stand-ins for it is not valuable.  If your defense of Bach is that he caused you to update a piece of your model, then you are not saying Bach is right – you’re saying, like it or not, that you are.

xhxhxhx:

Thinking about Ross Douthat’s “The Age of Decadence” this morning:

“Do people on your coast think all this is real?”

The tech executive sounded curious, proud, a little insecure. We were talking in the San Francisco office of a venture capital firm, a vaulted space washed in Californian sun. He was referring to the whole gilded world around the Bay, the entire internet economy.

That was in 2015. Here are three stories from the five years since.

A young man comes to New York City. He’s a striver, a hustler, working the borderlands between entrepreneurship and con artistry. His first effort, a credit card for affluent millennials, yanks him into the celebrity economy, where he meets an ambitious rapper-businessman. Together they plan a kind of internet brokerage where celebrities can sell their mere presence to the highest bidder. As a brand-enhancing advertisement for the company, they decide to host a major music festival — an exclusive affair on a Caribbean island for influencers, festival obsessives and the youthful rich.

The festival’s online rollout is a great success. There is a viral video of supermodels and Instagram celebrities frolicking on a deserted beach, a sleek website for customers and the curious, and in the end, more than 5,000 people buy tickets, at an average cost of $2,500 to $4,000 — the superfluity of a rich society, yours for the right sales pitch.

But the festival as pitched does not exist. Instead, our entrepreneur’s plans collapse one by one. The private island’s owners back out of the deal. The local government doesn’t cooperate. Even after all the ticket sales, the money isn’t there, and he has to keep selling new amenities to ticket buyers to pay for the ones they’ve already purchased. He does have a team working around the clock to ready … something for the paying customers, but what they offer in the end is a sea of FEMA tents vaguely near a beach, a catering concern that supplies slimy sandwiches, and a lot of cheap tequila.

Amazingly, the people actually come — bright young things whose Instagram streams become a hilarious chronicle of dashed expectations, while the failed entrepreneur tries to keep order with a bullhorn before absconding to New York, where he finds disgrace, arrest and the inevitable Netflix documentary.

That’s the story of Billy McFarland and the Fyre Festival. It’s a small-time story; the next one is bigger.

A girl grows up in Texas, she gets accepted to Stanford, she wants to be Steve Jobs. She has an idea that will change an industry that hasn’t changed in years: the boring but essential world of blood testing. She envisions a machine, dubbed the Edison, that will test for diseases using just a single drop of blood. And like Jobs she quits college to figure out how to build it.

Ten years later, she is the internet era’s leading female billionaire, with a stream of venture capital, a sprawling campus, a $10 billion valuation for her company, and a lucrative deal with Walgreens to use her machines in every store. Her story is a counterpoint to every criticism you hear about Silicon Valley — that it’s a callow boys’ club, that its virtual realities don’t make the world of flesh and blood a better place, that it solves problems of convenience but doesn’t cure the sick. And she is the toast of an elite, in tech and politics alike, that wants to believe the Edisonian spirit lives on.

But the Edison box — despite endless effort and the best tech team that all that venture capital can buy — doesn’t work. And over time, as the company keeps expanding, it ceases even trying to innovate and becomes instead a fraud, using all its money and big-time backers to discredit whistle-blowers. Which succeeds until it doesn’t, at which point the company and all its billions evaporate — leaving behind a fraud prosecution, a best-selling exposé and the inevitable podcast and HBO documentary to sustain its founder’s fame.

That’s the story of Elizabeth Holmes and Theranos. It’s a big story. But our third story is bigger still, and it isn’t finished yet.

An internet company decides to revolutionize an industry — the taxi and limousine market — that defines old-school business-government cooperation, with all the attendant bureaucracy and unsatisfying service. It promises investors that it can buy its way to market dominance and use cutting-edge tech to find unglimpsed efficiencies. On the basis of that promise, it raises billions of dollars across its 10-year rise, during which time it becomes a byword for internet-era success, the model for how to disrupt an industry. By the time it goes public in 2019, it has over $11 billion in annual revenue — real money, exchanged for real services, nothing fraudulent about it.

Yet this amazing success story isn’t actually making any profit, even at such scale; instead, it’s losing billions, including $5 billion in one particularly costly quarter. After 10 years of growth, it has smashed the old business model of its industry, weakened legacy competitors and created value for consumers — but it has done all this using the awesome power of free money, building a company that would collapse into bankruptcy if that money were withdrawn. And it has solved none of the problems keeping it from profitability: The technology it uses isn’t proprietary or complex; its rival in disruption controls 30 percent of the market; the legacy players are still very much alive; and all of its paths to reduce its losses — charging higher prices, paying its workers less — would destroy the advantages that it has built.

So it sits there, a unicorn unlike any other, with a plan to become profitable that involves vague promises to somehow monetize all its user data and a specific promise that its investment in a different new technology — the self-driving car, much ballyhooed but as yet not exactly real — will make the math add up.

That’s the story of Uber — so far. It isn’t an Instagram fantasy or a naked fraud; it managed to go public and maintain its outsize valuation, unlike its fellow unicorn WeWork, whose recent attempt at an I.P.O. hurled it into crisis. But it is, for now, an example of a major 21st-century company invented entirely out of surplus, and floated by the hope that with enough money and market share, you can will a profitable company into existence. Which makes it another case study in what happens when an extraordinarily rich society can’t find enough new ideas that justify investing all its stockpiled wealth. We inflate bubbles and then pop them, invest in Theranos and then repent, and the supposed cutting edge of capitalism is increasingly defined by technologies that have almost arrived, business models that are on their way to profitability, by runways that go on and on without the plane achieving takeoff.

Do people on your coast think all this is real? When the tech executive asked me that, I told him that we did — that the promise of Silicon Valley was as much an article of faith for those of us watching from the outside as for its insiders; that we both envied the world of digital and believed in it, as the one place where American innovation was clearly still alive. And I would probably say the same thing now because, despite the stories I’ve just told, the internet economy is still as real as 21st-century growth and innovation gets.

But what this tells us, unfortunately, is that 21st-century growth and innovation are not at all what we were promised they would be.

I wonder if we’ll think about Silicon Valley the way we think about Texas wildcatters and Florida real estate hustlers. 

It’s not that there isn’t innovation in California. There is innovation in California. There was oil in Texas and real estate in Florida too. But their dilemmas are the same: the market has matured. The frontier is a confidence game, but the core is stagnant.

Facebook and Google monopolized social and search. They have the profits that newspapers used to have and broadcasters still do. Apple sells luxury products to the affluent, like the LVMH of Cupertino. They are not growing and innovating. They are insulating.

They acquire smaller firms that might threaten their business: Instagram, WhatsApp, Waze, DoubleClick. They do just enough to insulate themselves. They do no more. A little while ago, they challenged one another: Apple tried search, Google tried social. They don’t do that anymore.

It would be nice if they tried something new.

I know there were a few reblog chains about this post already, so I apologize if this is retreading those discussions.

That said, I think this perspective is wrong in a few related ways:

(1)

The “core” in Silicon Valley is investing in all sorts of experimental and untried avenues, and has been for a while.  Consider Google/Alphabet’s various “moonshots,” a number of which have subsidiary companies dedicated to them – stuff like autonomous driving, balloon-based internet, storing renewable energy in molten salt.  Or Facebook’s massive investments in VR.

Or – to focus on an area I’m more familiar with – consider all the machine learning research these companies are doing.  The architecture underlying GPT-2 was developed at Google, and both Google and Facebook have made important strides in applying that technology (BERT at Google, Roberta at Facebook).  AlphaGo, AlphaStar, and … well, everything else DeepMind does?  Google again.  If someone in the last few years is doing any kind of work with neural nets, they’re probably making use of foundational code written at Google (Tensorflow) or Facebook (Pytorch).

(2)

Each of the “core” companies has some secure, profitable activity they’ve monopolized.  That is what makes them the core.  Each one also pumps a lot of that revenue into startup-like research – that is, research aimed at creating new technologies, expanding into untried markets, creating markets that don’t exist.  Research that isn’t about their area they’ve already monopolized.  It’s like these companies contain miniature internal versions of Silicon Valley, in which they play both the investor and startup roles.

If they appear more conservative, less interested in innovation, that the startups around them, it’s not because they are not doing the same “wild” and experimental things that the startups are doing.  They are.  The difference is a social and psychological one.

Because the people funding Alphabet’s research are formally part of the same organization as the people doing that research, it looks different when this research fails to pan out.  It looks like “Google wasting their own money,” and Google has plenty of money which is theirs to waste if they like.  It doesn’t look good, but it doesn’t rouse moral ire and the instinct to blame specific mendacious individuals.  And that’s when we hear about it at all; in many cases, startup-like projects may be internally created, tried and shelved with little to no press.

Uber is spending a lot on autonomous driving without any proof that it can make a profit on it.  So is Waymo, an Alphabet subsidiary.  But people only care about one of these stories.

(3)

Most attempts at big-thinking innovation fail.

When the attempts happen within a “core” company, the failures are quiet.  We hear little about them, and a lot about the company’s work in their original market, which continues to be profitable.  So the company looks “stagnant.”

When the attempts happen at a startup, the failure tanks the company, or triggers a “pivot” to something completely different.  These failures are often heavily discussed in the press.  Retrospectively, they often look like (and sometimes really are) egregious misuses of someone else’s money.  So the frontier looks full of hustlers.

This is not two different phenomena, it’s a single one.

There are probably some slick operators inside of Alphabet and Facebook, using spin or outright lies to secure money from people within the same organizations.  We generally don’t hear about them.  Why would we?

(4)

Natural resource rushes don’t seem like good points of comparison for Silicon Valley.  Better to compare it to historical cases where money was spent on innovation qua innovation – on inventors, on efforts to make products out of lab discoveries, that sort of thing.

What’s the difference?  With natural resources, the rush is to grab access to something already valued on a mature market.  Success, when it happens, is immediate and easy to demonstrate.  If I find some gold, I don’t have to create a market for gold ex nihilo before getting rich in my currency of choice.

But investment in “innovation” is generally investment in a market that doesn’t yet exist.  “Social” and “search” were not already-proven sources of value that Facebook and Google discovered in a quarry somewhere.  These companies created the categories of profitable activity they now derive their profits from.  If they had waited until their ideas were proven, they would have waited too long to monopolize.

They built the bridges on which they are now collecting tolls, and at the outset there was scant evidence anyone would even cross the bridges at all.  This is inherently risky; in a sense it is inherently a bluff, a bet, or a con.  Attempts to do it generally fail.  But this is why the attempts that succeed are so massively profitable.  By the time “social” and “search” had been proven viable, it was too late for any other players to capitalize on them.  That’s the whole game: making a source of value appear from scratch, and thus acquiring 100% control over it.

(This is a context where trying to compete on each others’ turf is distinctively unpromising.  What distinguishes these players is their utter control of their own turf, and their counterparts’ utter control of theirs.  Trying to compete in these areas is scraping the bottom of the barrel, and an absence of such activity is the opposite of stagnation.  You only do this if you have no better ideas.)

(5)

This is a strange business to be in, to be sure.  Ordinary standards of prudence are inapplicable, and it’s not clear what should take their place.  If someone credibly informs you they can definitely make a profit from some market, right now, they are ipso facto too late to that market to be a Google or a Facebook.

So you have to fund unproven things.  And yet you want to avoid spending money on con artists, on ideas that couldn’t possibly work.  You must have some way of assessing the quality of ideas, and yet you can’t require the level of hard evidence you’d want when assessing almost anything else.

I don’t know the right way to do this, or if there is a right way at all.  And I’m certainly sympathetic to the idea that too much money and thought is being thrown into software innovation as opposed to more incremental, marginal, competitive (rather than monopolistic) software work.

But I do know that Theranos and Uber are children of the same system that produced Facebook and Google.  That system takes in money, throws it at unproven ventures, and emits two things: a steady stream of amusing failures and follies, and the occasional monopolistic behemoth with rights to a vast frontier that didn’t even exist before.  The failures are not what remains when the behemoths have seized all the “real” value, the actual gold or oil.  The failures are a waste product of the same process that produces the behemoths, and they will be tolerated until those who inject money into the process decide it has stopped working.

(6)

You can ask for research to happen faster, more efficiently, more reliably, but you can’t ask it to happen without frequent, repeated, extensive failure.

The old joke goes: “A mathematician only needs pen, paper, and a wastebasket to do his work. A philosopher only needs pen and paper.”  The joke’s conclusion may be controversial, but everyone can agree with the unstated premise – that if you are not throwing anything into the wastebasket, you are not doing real research.

I feel on a gut level like there’s something very weird and wrong with Silicon Valley, as a system for investing in research.  I wish I knew the right diagnosis.  But we cannot count an intrinsic quality of research itself among the defining symptoms of the disease, or else all research would be equally diseased.

the-real-numbers:

necarion:

nostalgebraist:

jadagul:

This looks cool and I need to read it later.

the-real-numbers:

Just, uh, gonna leave this here for… reasons

https://arxiv.org/pdf/1610.08401.pdf

(Tagging @stumpyjoepete​ since he tagged me on this post)

This is definitely a cool result.

It’s an extension of previous adversarial example work, showing that you can find a single adversarial perturbation  – i.e. a very faint, nearly imperceptible pattern you can layer on top of an image that will cause neural net classifiers to mis-classify it – that works generically for any image in the standard ImageNet challenge dataset.  These even generalize across different classifiers, to some extent.

My strong hunch is that this is a “feature, not a bug,” and reflects the inherent mismatch between the ImageNet challenge and real vision, rather than reflecting a flaw in neural net image classifiers.

The paper doesn’t draw this conclusion, but it contains various pieces of evidence pointing in that direction, IMO.  Namely:

• As mentioned, if you design one of these “universal perturbations” to target one classifier, it will also tend to fool other classifiers, even those with very different architectures.
  
  This increases the burden of proof for someone arguing that this reflects a flaw in how these models classify images: this person would not be arguing just that some architecture has a blind spot, they’d be arguing that many apparently distinct architectures somehow have the exact same blind spot.
  
  On the other hand, the different architectures have this in common: they’re all good at the ImageNet challenge.  So if “susceptibility to universal perturbations” is actually a natural result of being good at ImageNet, it’s no surprise that all the architectures have that property.  (Humans find the ImageNet challenge difficult without special training, so it’s not a problem for this hypothesis that humans aren’t thus susceptible.)
  
  
• The authors do a finetuning experiment that tried to teach the VGG-F architecture not to misclassify the perturbed images.  This helped a little, but cannot get the model below a “fooling rate” of 76.2%, which is still high.
  
  To explain this as a defect in the architecture, one would have to imagine that the universal perturbations are somehow “invisible” to it in a way that prevents them from learning a signal correlated with them; this seems implausible.  [ETA: of course the perturbations aren’t invisible to the models, otherwise they wouldn’t work.]  But if “don’t misclassify the perturbed images” actually competes with “do well at ImageNet,” then of course you won’t get very far on the former while still trying to preserve the latter.  (In this connection, note also the following: “This fine-tuning procedure moreover led to a minor increase in the error rate on the validation set […]”)
  
  
• The incorrect class labels given to perturbed images tend to come from some very small set of “dominant” labels, as visualized in the directed graph.
  
  This made me think of a hypothesis like “there are a few classes in the ImageNet challenge that have certain distinctive visual patterns not shared by any other classes, and so the optimal way to identify these classes (in the context of the challenge) is just to check for these patterns.”
  
  This seems a priori plausible.  The ImageNet challenge asks for classification at a very fine-grained level, without partial credit for getting the right general sort of thing.  Many of the 1000 ImageNet challenge classes are specific species (or other low-level taxonomic group) of animal.  The images themselves, largely scraped from Flickr, are photographs of the animals (or other things) from numerous angles, in numerous contexts, sometimes partially obscured, etc.  In this context, developing a high-level concept like “bird” is actually quite difficult, and of limited value (no partial credit for knowing it’s a bird unless you can tell exactly what kind of bird).  But identifying the distinctive markings that are the hallmark of one exact kind of bird will work.
  
  When you get points for saying “African grey” but not for another kind of parrot, and you have to do this across diverse pictures of African greys, and you’re a neural net that doesn’t know anything at the outset, of course you’re going to develop a detector for some exact textural feature that only African greys have and use that as your African grey detector, and skip over the much harder task of developing detectors for “parrot” or “bird.”
  
  (African grey is in fact one of the dominant labels.  Macaw is another.)
  
  

The authors do this other thing where they look at singular values of a matrix of vectors from images to the nearest decision boundaries, and show that these vectors have some orientations much more often than others.  I’m not sure I understand this part – isn’t it just a restatement of the result, not an explanation of it?  (If this were false, wouldn’t the result be impossible?)

Anyway, this way of describing the situation – “the nearest decision boundary is frequently in a specific direction” – needs to be interpreted in light of the dominant labels things.  It would be different, and arguably more interesting, if there weren’t dominant labels, or if they weren’t quite so dominant; in that case the result would mean that the models identify certain textural differences as inherently “salient for distinctions.”

Instead, it just means that the models make some distinctions differently than others.  Some distinctions are made in a more “realistic” way, on the basis of higher-level features that correspond to different pixel-level variations depending on what base image you’re varying.  And then, some are just simple pattern detectors that always look about the same on the pixel level.  And again, that’s not really surprising.  Distinguishing bird from non-bird is a high-level judgment, but distinguishing one species within birds really is a matter of looking for one telltale pattern that’s relatively stable across orientations.

Now, if you’re a human who has to track objects over time, understand salient categories like “is this animate?”, and so on, you will tend to make the “YES-bird” and “YES-African-grey” judgments simultaneously.  Thus it sounds bizarre for something to say “YES-African-grey” when it’s looking at a bathtub that happens to have a bit of the African grey texture sprinkled on top.  But if you’re an ImageNet challenge machine, the “YES-bird” judgment doesn’t even exist in your universe.  In the toy ImageNet universe, in fact, it is arguably not even wrong to classify that bathtub as an African grey – for in that universe, there are no birds as such, and there is no such thing as a bird for a bathtub to be distinctively not.

Are there CNN training sets that include these hierarchies? So something could be an African Grey and a Parrot and Bird? Or modifying the network to go through some sort of word embedding, so results that are particularly closely clustered might be “partly” acceptable to the training?

There are CNN data sets that have hierarchical classes in the DSP/ML space. I’m not sure how available they are to laypeople. Sometimes you can handle the subclass superclass problem by classifying on the subclasses and have an additional loss factor for superclasses/categories, although I imagine you could try having one head CNN for superclasses that passes off the processed images to various trunks for subclassing.

But say for example if it’s hard to tell the difference between a titmouse and a pug. The traditional superclass may send titmice to the wrong subclass net and you’re guaranteed to get a wrong answer.

Although, you may find that you might want to superclass based on the most confused subclasses, which could mean training a subclassifier and determining superclasses with a mutual information approach or eyeballing a confusion matrix, then trying again.

A relevant, fairly new area of research that I find exciting is hyperbolic embeddings.  Some key papers are

1. The original paper introducing them (or the one everyone cites, anyway)
2. This paper which provided an important conceptual advance over #1
3. This one which builds up some of the necessary building blocks for neural nets over these spaces

The idea behind hyperbolic embeddings is… hmm, let me describe it this way.  Suppose you have some hierarchically nested categories, and you’re trying to model them in Euclidean space in some way.

There are two (?) ways to do this (this distinction is mine, not from the above papers):

• “Map” model: each category is a region of R^n, and the hierarchy’s nesting relation is represented by the R^n subset relation.  Like, “human” might be some blob of R^n, and “doctor” is a proper subset of that blob, and then “oncologist” is a proper subset of “doctor,” and so forth.
  
  This is like a map, where “doctor” is inside “human” the way “Colorado“ is inside “U.S.”
  
  
• “Tree” model: each category is a point in R^n, and the points are arranged like a literal picture of a branching tree structure.   If the tree(s) start at the origin, the nesting relation is represented by R^n vector magnitude, with more specific categories further from the origin.
  

Now, a downside of the “map” model is that finer-grained category distinctions are encoded as smaller distances in R^n.  This might sound natural (aren’t they “smaller” distinctions?), but the practical importance of a distinction doesn’t necessarily scale down with its specificity.  (Sometimes it’s very important whether a doctor is an oncologist or not, even though that’s a “fine-grained” distinction if your perspective also captures doctor vs. non-doctor and human vs. non-human.)

One might hope that the “tree” model could solve this problem: you can have each level “fan out” from the previous level in space, making its nodes just as far apart from one another as the nodes in the previous level.

But, in Euclidean space, there isn’t enough room to do this.  Deeper levels in the tree have exponentially more nodes, so you need exponentially more volume to put them in, but going further from the origin in R^n only gives you polynomially more volume.

However, hyperbolic space gives you just what you want: exponentially more volume as you go out.  Like in the famous Escher illustrations (visualizing the Poincare disk model of 2D hyperbolic space):

In the actual hyperbolic metric, the bats are all the same size.  A tree embedded in the Poincare disk model might look like (figure from the Poincare Embeddings paper):

where again, things don’t actually get closer together near the rim, they’re just visualized like that.

OK, so what does that have to do with the original topic?

Well, almost any classifier you encounter these days is going to do two things: map its inputs onto a (Euclidean) latent space in some complicated non-linear fashion, and then divide up that latent space into regions for the different labels.  (Usually the latter step is done with hyperplanes.)

We’re discussing ways of letting the classifier “know” that the labels have a hierarchical structure, with some of them “going together” as part of a larger group, which might then be part of an even bigger group etc.

If we do this by allowing “partial credit” for labels in the same coarse class (as in @necarion​‘s word embedding proposal), this will encourage the network to put these labels close together in the latent space.  Which is like the “map” model: all the types of bird will get assigned to adjacent regions, and you could draw a big shape around them and say “this is ‘bird’.”  So at best we end up with the “map” model, with its “oncologist problem” as described above.

Alternately, you can actually change the model to explicitly encode the hierarchy – like what the @the-real-numbers​ describes, where you have different classifiers for different levels.  This can let you get around the downsides of the Euclidean “map” model, because the different classifiers can operate only on their own scales: the coarse classifier that just has to output “bird” is free to squash lots of bird types close together in its latent space, while the intra-bird classifier gets a whole latent space just for birds, so it can make them far apart.

Suppose – as the hyperbolic embedding work suggests – that the discriminations we want out of the model cannot be mapped well onto distances in Euclidean space.

Then:

• The partial-credit approach says “let’s just do the best we can in Euclidean space, with the nesting relation of an arbitrary hierarchy modeled by the subset relation on a Euclidean space learned from data with that hierarchy.”
  
  This provides an intrinsic model for “nesting” as a generic concept, but distances inside the same model don’t behave in all the ways we’d like (oncologist problem).
  
  
• The multiple-classifier approach says “let’s give up on modeling the nesting relation of an arbitrary hierarchy; instead let’s tie ourselves down to one specific hierarchy, and design N copies of Euclidean space tailored for it.”
  
  This does not provide an intrinsic model for "nesting” as a concept – you’re tied to one particular case of nesting, expressed by the output code that maps the various latent spaces to parts of your specific hierarchy.
  

With hyperbolic latent space, hopefully you can model the nesting relation as a relation in the space (intrinsic) and still have the distinctions you want to make map naturally onto distances in the space (no oncologist problem).

human psycholinguists: a critical appraisal

(The title of this post is a joking homage to one of Gary Marcus’ papers.)

I’ve discussed GPT-2 and BERT and other instances of the Transformer architecture a lot on this blog.  As you can probably tell, I find them very interesting and exciting.  But not everyone has the reaction I do, including some people who I think ought to have that reaction.

Whatever else GPT-2 and friends may or may not be, I think they are clearly a source of fascinating and novel scientific evidence about language and the mind.  That much, I think, should be uncontroversial.  But it isn’t.

Keep reading

internet explorers (not exploiters)

I was reading this post from 2013, about addictive games and game-like websites (such as Facebook), and the possibilities for “addictive” educational resources that could compete with them.  It was interesting, but something felt off about it.

I found myself fixating on this passage [emphasis mine]:

Somebody who is tackling a truly novel problem often feels at a complete loss, having no idea of whether they are even on the right track. Someone who is facing a tough problem that is known to be solvable, but which nobody has yet turned into an addictive one, might feel similarly. If we motivate people to work by giving them frequent external rewards, does that train them to become even more impatient and eager to quit in cases where no such rewards are forthcoming?

Apparently, Western cultures are already doing badly with this. According to this NPR piece, American first-graders who were given an impossible math problem to work on tended to give up within less than 30 seconds. Japanese students, on the other hand, spent a whole hour trying to solve it, stopping only when the researchers told them to. We Westerners have already been trained to desire instant gratification, and it might not be a good idea to turn society even more in that direction.

I am not at all sure that we have a choice, however. It is all well and good to say that we should stop being so focused on instant gratification and train ourselves to work on problems for longer before giving up. But how are we going to do it in a society that keeps becoming ever more addictive? Most people have big projects and noble goals that they say they intend to accomplish, one day – and then they never do, because there are easier rewards available. “We should train ourselves to have a longer tolerance for uncertainty and delayed rewards” is exactly the kind of goal that provides an uncertain reward sometime late in the future… and is thus likely to be put aside in favor of easier goals.

I’m not sure the blogger is saying that the behavior of the Japanese students was strictly better than that of the American students.  I think the argument is just that “Western cultures” have swung too far in one direction, that in the short term it would be helpful to take local steps toward the mindset of the Japanese students.  It’s not a Goofus vs. Gallant thing.

Still, I think “delay gratification more” is difficult advice to act on, in a way the blogger is not accounting for.  This is because “how much should I tolerate delayed gratification?” is an enduring, intrinsically difficult question, one that will always trouble us as long as we are still alive and making decisions.  It’s a question that has been academically formalized as the “explore/exploit tradeoff,” which comes up for instance in the design of computer programs, which have no will of their own.  (That is, we can make the programs delay gratification as much as we like without them “complaining,” but this doesn’t actually lead to the best results.)  And I think the explore/exploit tradeoff is a very important concept if we want to understand why the internet is so addictive.

If you aren’t familiar with it, here’s the basic gist.  (There are probably better explainers somewhere if you feel like Googling for them.)

Say you are designing a machine that makes choices.  (I’m using a machine example here, rather than a person, so it’s clear that the tradeoff arises even when you don’t have a desire for instant gratification built in at the start.)

Your machine doesn’t know everything about the world.  So there are two reasons it might choose an action:

(1) it knows what will happen as a result, and likes it

(2) it doesn’t know what will happen as a result, so it will get new information from taking the action, even if the result is bad

Type 1 is called “exploitation” and type 2 is called “exploration.”

As a concrete example, imagine that the machine gets “points” every time it enters a room.  There are ten million rooms.  It knows that it will get 2 points for entering Room One, 1 point for entering Room Two, and 3 points for entering Room Three, but it doesn’t know how many points the other rooms deliver.

The best room it knows about is Room Three.  The “pure exploitation” strategy would just be to enter Room Three over and over again, forever.  But clearly this is a pretty silly strategy.  It only knows about 3 out of 1,000,000 rooms, and what if one of the others give more than 3 points?  Hence it may want to do some “exploration,” not just “exploitation.”

To learn the answer, though, it will have to take the risk of entering some rooms that might give less than 3 points, so it might lose out in the short term.  And there are so many rooms that it would make sense to settle down and exploit at some point, even if there are still rooms it doesn’t know about.

(The classic formal version of this problem is called the “multi-armed bandit,” BTW.  If you like math/CS there’s a lot of interesting stuff out there on it.)

This framework makes sense for thinking about instant vs. delayed gratification, although the connection can go in two opposing directions.  On the one hand, we need a tolerance for delayed gratification in order to explore, because exploration involves trying many things that aren’t as good as the best thing we already know about.  (Are you going to try a new recipe that might suck, or just make that easy dish you know you like?)  On the other hand, if we always stick with the same thing in the hopes that it’ll pay us delayed rewards sometimes, we are exploiting when we should be exploring.  (If you’ve been “working on” that unsatisfying job/relationship/whatever for 10 years, hoping that maybe next year it’ll finally improve, you might be too tolerant of delayed gratification.)*

When I read the anecdote about the American and Japanese students, I immediately thought back to my time in grad school.  Every day of work in grad school was a vivid explore/exploit dilemma: I was working on a project, and I had various ideas I was pursuing, many of which would probably not work out in the end.  Each day, I could plug away at my latest idea (exploitation), or I could spend time exploring – thinking about other possibilities, trying to critique my existing ideas, trawling Google Scholar to see if someone’s done it already.

This was never a Goofus vs. Gallant, good responsibility vs. bad hedonism sort of decision.  Exploration was scary: I could spend a whole day thinking or reading without anything to show for it, except even stronger misgivings about everything I’d done.  But exploitation was also scary: what if I was wasting time on something clearly misguided?  (I’ll never forget the time I spent weeks working on a beloved idea while my adviser was out of town, wrote up 10+ pages of TeXed notes on it, presented it proudly on the whiteboard to my adviser and a colleague, and watched them sit in impressed silence for about 30 seconds, after which my adviser walked up to the board and wrote a two-line proof that my idea could never possibly work.)

Indeed, one of the biggest psychological pitfalls was that exploitation felt so responsible.  If I was writing equations and doing math, I was “getting things done,” not just screwing around.  I was delaying gratification, putting a big project together step by step, carefully and dutifully.  This responsible feeling, though, got me into plenty of situations like the one I just described.  Delayed gratification can be dangerous!

*(Boring technicality paragraph: this is all a bit circular.  If we really didn’t care whether a reward happens now or in a million years, there wouldn’t be as much of a reason to prefer exploitation over exploration.  There still may be a reason if there’s an infinite choice set, but that’s a mathematical curiosity; more relevantly, any machine or human project has a finite time horizon, and besides, time is money.  If you spend 10 years agonizing over finding the perfect job posting to apply to, you are losing money by exploring too much.  No messy value judgments there, just concrete $$.)

Back to addictive games and the internet.

It’s tempting to lump together addictive video games and addictive internet behavior, since they’re both recent phenomena involving digital technology.  But I think they’re really very different.

Addictive games are all about exploitation.  They may have internal explore-exploit dilemmas: maybe you are trying to get the most points, and you have choices like the “rooms” above.  But when you’re entranced by the game, it makes you forget that you value things other than its points.  You end up spending three straight hours trying to get more Tacky Phonegame Points, not thinking “do I really have nothing better to do with the next three hours than mining Tacky Phonegame Points”?  This is why they’re draining and ultimately un-fun, like chemical addictions: you end up pursuing the game’s internal objectives even beyond the point where you stop truly wanting them.

Addictive internet behavior, though, is all about exploration.  That blog post mentions checking Facebook notifications, which do have a certain resemblance to Tacky Phonegame Points.  But the vast majority of the time I spend on sites like Facebook and tumblr isn’t spent following up on notification, it’s spent scrolling and refreshing the feed.  I am not pursuing any explicit score here; the website presumably records how much time I spend reading the feed, but it doesn’t give me any kind of feedback about it.  You have Follower counts and Friend counts, but you don’t have “Number of Posts Read” counts.  Yet I read a whole lot of posts.

Another distinctive feature of these sites, which is totally different from addictive games, is their heterogenous content.  When I scroll down my feed, I have no idea what I’ll see next – a joke? a serious political statement? a personal lament or celebration?  There is an “instant gratification” element here: I’m curious what the next post will be.  But there is also a strong exploration appeal.

The internet is better than anything else in the world at making you aware, at all times, there there may be better Rooms out there.  You tend to end up reading one tab of 20, acutely aware that there are 19 other possibly-better reading choices a mere click away.  Everyone knows those social media posts that make the plaintive appeal, “why isn’t anyone talking about this?”  But even when this is not explicitly said, it’s always the subtext of the internet.  There are always so many things you aren’t (yet) talking about, thinking about, reading about.

A typical breeze through my feeds in the morning might expose me to 2 important-sounding political issues I’ve never heard of, and 2 noteworthy developments in the lives of friends or acquaintances, and 2 things that trigger thoughts I want to write down in my own posts, and 5 or 6 articles or blog posts that I will open in new tabs because they give me an “I ought to know about this” feeling.  Which of these opportunities should I spend my day pursuing?  I’ve only just woken up, and already I have so many responsibilities – and that’s if I want to “waste the day on the internet,” instead of “getting real work done”!

The internet connects us to many people on ambiguous terms, in relations that are somewhere between friendship, acquaintance-ship, fan-ship, etc.  Unless we spend all day reading the feeds, we are constantly – by the standards of real-world friendships – “falling out of touch” with people.  So many personal situations I could pay more attention to, but at the cost of not reading up on the latest political thing I ought to know about, or reading that oh-god-so-interesting blog post.

And on the internet, we no longer have any excuses for not “educating ourselves.”  That important issue that more people should be talking about?  You can’t say you don’t have time to go to the library.  You probably haven’t even read the Wikipedia page, or all 10 of the must-read long-form journalism pieces, much less all the acerbic must-read blog critiques of those pieces (what kind of sucker are you, not being up on the acerbic critiques?).  Meanwhile, of course, you’ve been neglecting that tab about the different local variants of Polynesian mythology, which maybe isn’t as “important,” but which is super interesting, and which was giving you ideas for your fiction project earlier on, and wouldn’t it be a shame if you forgot about it and your fiction suffered as a result?

So much exploration to do.

Why isn’t anyone talking about Room #130,327?

BRYCE
What about the massacres in Sri Lanka, honey? Doesn’t that affect us, too? I mean don’t you know anything about Sri Lanka? About how the Sikhs are killing like tons of Israelis there? Doesn’t that affect us?

BATEMAN
Oh come on, Bryce. There are a lot more important problems than Sri Lanka to worry about. Sure our foreign policy is important, but there are more pressing problems at hand.

BRYCE
Like what?

BATEMAN
Well, we have to end apartheid for one. And slow down the nuclear arms race, stop terrorism and world hunger. But we can’t ignore our social needs, either. We have to stop people from abusing the welfare system. We have to provide food and shelter for the homeless and oppose racial discrimination and promote civil rights while also promoting equal rights for women but change the abortion laws to protect the right to life yet still somehow maintain women’s freedom of choice.

I think the internet has made it a lot harder for me to read books.  Really, it’s made it harder to focus on a lot of things, but reading books stands out, since it’s such a close substitute for what I do on the internet (mostly reading text of some sort), and because it’s something I want to do more of.  I don’t mean that in an aspirational, self-improvement way; the books are actually more interesting, in an immediate-fun way, than the internet.  Yet I choose the internet.

The problem here is not weak-willed hedonism.  Weak-willed hedonism is when, instead of reading a book or browsing the internet, I choose to play Starcraft.  When I make that choice, I am plunged into an almost non-conscious exploitation void, I get a lot of Points, and I come out on the other end 30 minutes later as though nothing had happened, without no learning or thinking in between.

No, this is something else.  You know what’s weird?  The reason I have so much trouble reading a book – any book – is that it feels complacent.  It feels like writing equations all day that my adviser is going to demolish on the board at our next meeting.  It feels like exploitation in a world full of visibly under-explored possibilities.  Even “important” reading, reading on “things I should know about,” feels palpably excessive, frivolous, an overblown luxury.

350 pages about Sri Lanka alone?  Come on, Bryce.  There are a lot more important problems than Sri Lanka to worry about.

The internet connects us to many many people, which makes us aware of all the possibilities for exploration.  To compound the problem, it also makes us aware of the worst pitfalls of exploitation.  It routes us directly to all the world’s crackpots and single-issue obsessives, who we all put in tabs so we can guiltily skim their work and think about how much more worldly we are than them, how much more aware.

The most superficially charismatic people on the internet (to me) are the people who are the opposite of these obsessives, who seem to know (a little) about everything, who always have the latest must-read links, who expose you to new things every day.  The Batemans, not the Bryces.  These people presumably have relatively shallow knowledge of any one thing, but since we’re all hyper-explorers on the internet, we don’t have enough time to exhaust their knowledge of any one thing.  The internet doesn’t show me “professor spends 10 years working on topic, contributes steadily to human knowledge”; it shows me “professor fields ignorant opinion on twitter; should have known better.”  Good thing I know better, I think, clutching my 20 tabs to myself like armor.

I don’t think traditional notions of addictiveness are sufficient to describe this particular problem.  It isn’t a matter of just choosing the responsible thing.  The problem is that Bateman is (figuratively) right.  There are a lot of things to worry about, and now we’re hyper-aware of this fact in a historically unusual (unique?) way.

And our traditional notions of responsibility, of instant and deferred gratification, were shaped in a world where there were much stronger barriers to exploration.  Where you could still say, truthfully, that you hadn’t educated yourself on an issue because you just didn’t have time to go to the library.

Now there really are millions of Rooms a few clicks away, and any one of them might be a very important Room.  And the question is what we should do in that situation.  And there is no good, traditional answer.  This is a new problem and we need new answers.

tadrinth:

nostalgebraist:

nostalgebraist:

Returning to my graduate thesis (for what is hopefully a brief round of polishing before the whole thing’s over) has put into stark relief how unpleasant grad school was.

Suddenly I’m back to the old (half-forgotten!) daily pattern of accomplishing a few minor things in the first few hours of the day, then spending the rest of the day in an anxiety loop where I plan to do a more substantial task and then worry about how hard it’ll be given how anxious I already am, which makes me more anxious, etc.  Fiddling with code and running endless tests becomes a way to avoid thinking about the bigger picture, because thinking about the bigger picture feels like watching some absurdist play about how there are no longer any clear standards of value in the world.  My desire for alcohol in the evening has increased, just as it immediately decreased when I stopped working on this project earlier in the year.

I hope this isn’t just “what confronting something serious and difficult” is like for me.  But I’ve had other challenges that don’t make me feel like this; the first time I remember this distinct thing was when I was a research assistant after college, which – hmmm – was the first time I did “real” academic research (as opposed to my undergrad thesis, where I just did a project I thought was cool and didn’t expect to publish any results or otherwise interact with academia).  So probably(/hopefully) I just hate doing academic research.

There are probably a number of distinct reasons why I hate doing academic research, and I’ve obsessed over one possibility or another from time to time, but here is one that crystalized for me after writing the OP:

In academia, it’s not just that the value of what you are doing is uncertain.  It is uncertain, because you’re doing research and by definition no one knows for sure what the outcome will be.  But that in itself I might be able to live with: negative results are still results, and there’s nothing shameful about saying “this seemed like it might work; we tried it; it didn’t, for these reasons; now you know.”

It’s not just that the value is uncertain, but that you’re working alongside many people – many of them incredibly smart and successful – doing things of likewise uncertain valueand that, in merely doing your research, you are implicitly claiming that your inherently-uncertain project is a relatively good bet among all conceivable such projects, or at least all the ones currently in play.  You are implicitly saying that the direction you have chosen is a shrewd direction, not just from your personal perspective, but from the perspective of the entire vast many-tentacled apparatus of Science*.

The boundaries of “what your field knows” are wide, twisting coastlines you can’t honestly claim to know in full, and at every point on this coast, just offshore, there are innumerable new experiments that could be done, new directions that could be tried.  Only a finite number of them will be tried (in the next year, say), and in many cases – when applying for grants, say, or competing for publication in a journal – you are fighting in a zero-sum competition to make your direction one of those few.  Even when the finite pie isn’t as visible – when giving talks, say – you’re still presenting your little in-progress expedition not just as one interesting on its own terms, but one (as it were) advantageous to the whole nation.

In actuality, you only have a vague mental map of the island nation and its coast, with most of the detail concentrated on the little area you call home, and the real reason you’re setting out from the local port is because it’s local and you know and like the waters.  But you have no option except to play ball with the whole national community of explorers.  You may not especially care about Admiral Bigwig’s exploits off the other end of the coast, but like it or not, Admiral Bigwig’s contributions to the national interest will be ranked against yours.  You will be asked why Her Majesty should send resources to your backwater, specifically, of all the ports on the island, and suddenly every port on the island is your business.

Supposedly it’s possible to avoid this by specializing in a small enough subfield, but I’ve never actually had that experience.  After all, people in different subfields can still effectively do the same things, and often they do.  If you’re in a small enough pond, you then have to worry that the size of that pond might shrink to zero as it is revealed to be just a shallow murky subset of something bigger and cooler.

Research is inherently combative, “political,” involved with everyone else’s business.

---

This kind of thing isn’t unique to academia, and indeed it may not be possible to avoid it entirely.  Job applications, say, are a bit like this (although “contributing to a company” is easier to get a mental handle on than “contributing to a scientific field”).

I get the impression that the startup world is a lot like this, which means I should be very careful about getting involved in it.  In the broadest terms, I don’t want to be in an organization where everyone’s working their asses off but no one quite knows what “the product” is or why it’s any good.

*(focusing on science here just because that is the kind of academic work I have experience in)

Competition for venture capital is a bit like this, but more from the perspective of the VCs; I’ve worked at two software-as-a-service startups.  At both, all of the employees knew exactly what the product was and why it was any good: the software did roughly the same thing as a competitor’s product, except that theirs was built twenty years ago and ours was built today.  There’s been so much improvement in software infrastructure that you can blow big, slow-moving companies away by rebuilding their product, because the modern infrastructure lets you either charge less or add features faster.   

That’s not so much saying that every startup is correct about what the product is or why it’s any good, but having a very good narrative for those two things is a big factor in attracting venture capital, so most startups will have a good story even if it isn’t true.  

That issue does affect founders, so I wouldn’t start your own company, but it doesn’t sound like something that would be an issue when joining a start up.

I pursued a PhD but instead left with a Masters; switching careers to programming has done wonders for my emotional health and income.  

This is interesting and helpful, thanks.

I guess I’m confused about the point at which “the issue affects founders, but not new hires” kicks in.  The comparison to academia comes to mind when I think of the stories I’ve heard about startups in their chaotic early days (with very few employees, etc).  It seems to me (is this wrong?) that if your company is five people in one room frantically talking about how to make money, potential pivots, what narrative to spin for VCs in the next round, and so forth, it doesn’t really matter whether only four of the people were there at the beginning (”founders”) and the other one came aboard later.

Of course not all startups are in this phase, but I’m not sure what a safe proxy for the transition is, besides “no longer being thought of as a startup,” which is what I had in mind when writing the previous post.  When I think about “applying to work at startups” I think about a lot of places I’ve looked at that aren’t selling anything yet and/or have a team of ~5 people, where I wouldn’t feeling comfortable saying “oh, none of those anxieties would reach me, since I’m not a founder.”  Any thoughts?

(via tadrinth)

wrangletangle:

lizziegoneastray:

prokopetz:

modularnra40:

prokopetz:

becausedragonage:

prokopetz:

I’ve mentioned “romantic fantasy” in a few recent posts, and some of the responses have made it apparent that a lot of folks have no idea what that actually means - they’re reading it as “romance novels in fantasy settings”, and while some romantic fantasy stories are that, there’s a bit more to it.

In a nutshell, romantic fantasy is a particular genre of Western fantasy literature that got started in the 1970s, reaching its peak in the late 1980s and early 1990s. Its popularity sharply declined shortly thereafter, for reasons that are far too complicated to go into here; suffice it to say that you won’t find many pure examples of the type published after 1998 or so.

It’s tough to pin down exactly what romantic fantasy is in a few words, but you’ll definitely know it when you see it - there’s a very particular complex of tropes that defines it. I’ll try to hit the highlights below; not every romantic fantasy story will exhibit all of these traits, but most will exhibit most of them.

Romantic fantasy settings are typically “grown up” versions of settings that traditionally appeal to young girls: telepathic horses, wise queens, enchanted forests, all that stuff. Note that by “grown up”, I don’t mean “dark” or deconstructionist; romantic fantasy is usually on board with the optimistic tone of its source material, and any grime and uncertainty is the result of being a place that adult human beings actually live in. Protagonists are natives of the setting, rather than visitors from Earth (as is customary in similar stories targeted at younger audiences), though exceptions do exist.

In terms of stories and themes, romance is certainly a big presence, but an even stronger one is politics. Where traditional fantasy is deeply concerned with the geography of its settings, romantic fantasy focuses on the political landscape. Overwrought battle scenes are replaced by long and complicated discussions of political alliances and manoeuverings, brought down to the personal level through the use of heavily stylised supporting characters who function as avatars of the factions and philosophies they represent. Many romantic fantasy stories employ frequent “head-hopping” to give the reader insight into these philosophies, often to the point of narrating brief scenes from the villain’s perspective.

The “good” societies of romantic fantasy settings tend to be egalitarian or matriarchal. Patriarchal attitudes are exhibited only by evil men - or very occasionally by sympathetic male characters who are too young and sheltered to know better (and are about to learn!) - and often serve as cultural markers of the obligatory Evil Empire Over Yonder. Romantic fantasy’s heydey very slightly predates third-wave feminism, so expect to see a lot of the second wave’s unexamined gender essentialism in play; in particular, expect any evil or antagonistic woman to be framed as a traitor to her gender.

Usually these societies are explicitly gay-friendly. There’s often a special made-up word - always printed in italics - for same-gender relationships. If homophobia exists, it’s a trait that only evil people possess, and - like patriarchy - may function as a cultural marker of the Evil Empire. (Note, however, that most romantic fantasy authors were straight women, so the handling of this element tends to be… uneven at best.)

Magical abilities are very common. This may involve a unique talent for each individual, or a set of defined “spheres” of magic that practically everyone is aligned with. An adolescent lacking magical abilities is usually a metaphor for being a late bloomer; an adult lacking magical abilities is usually a metaphor for being physically disabled. (And yes, that last one can get very cringey at times, in all the ways you’d expect - it was the 1980s, after all.)

In keeping with their narrative focus, romantic fantasy stories almost always have an explicitly political character with a strongly progressive bent. However, most romantic fantasy settings share mainstream fantasy’s inexplicable boner for monarchies, so there’s often a fair bit of cognitive dissonance in play - many romantic fantasy settings go through elaborate gymnastics to explain why our hereditary nobility is okay even though everybody else’s is icky and bad. This explanation may literally boil down to “a wizard did it” (i.e., some magical force exists to prevent the good guys’ nobles from abusing their power).

I think that about covers it, though I’m sure I’ve overlooked something - anybody who knows the subject better than I do should feel free to yell at me about it.

(As an aside, if some of this is sounding awful familiar, yes - My Little Pony: Friendship is Magic draws a lot of inspiration from romantic fantasy, particularly the early 90s strand. It’s not a straight example of the type - there are very few of those around today - but it’s not at all subtle about its roots.)

Oh, I read so much of this as a teen and young adult. It might have started a touch earlier than the 70′s with Anne MacCaffrey and Dragonriders of Pern? The most obvious example I can think of is Mercedes Lackey’s Valdemar books and over in the comic book medium, I think Wendy Pini’s Elfquest just squeezes in. 

One thing about this genre, when I reread something from it that I loved 20 or 25 years ago, I go from extreme and affectionate nostalgia to quite literally blushing in embarrassment over some of those cringe-worthy bits you mentioned.

Yeah, Lackey’s Valdemar books are basically the platonic ideal of romantic fantasy for a lot of folks - though in spite of being arguably the most influential romantic fantasy author of her generation, Lackey herself was a relative latecomer to the genre.

As for McCaffrey, I’d hesitate to classify her Dragonriders of Pern series as romantic fantasy. I’ll grant that later entries in the series certainly develop in that direction, but especially early on it hews a lot closer to traditional heroic fantasy. Her Talent universe, however, is a dead-perfect example of the type, in spite of having an extremely variant setting.

(For those who haven’t read them, McCaffrey’s Talent books take place in a gonzo far-future space opera setting, revolving around the personal dramas of a pseudo-noble caste of godlike telepaths who enjoy their privileges as a consequence of being the setting’s only economical source of faster-than-light communication and transport. Weird stuff.)

I read so much Mercedes Lacky and Anne McCaffrey as a kid. I’d love to hear about the decline of the genre - I’m guessing that modern feminism and the lgbt movement had a lot to do with it? That is - the growth out of a lot of the more cringey tropes morphing the genre into something distinctly different?

Yeah, there were a number of different factors involved. Losing the LGBT audience was certainly part of it - not because of the inept handling of the subject matter per se, but because a lot of LGBT readers were reading romantic fantasy simply because they couldn’t get that kind of representation anywhere else, and when more LGBT authors started getting published in the mid 1990s, they had better options.

The Internet itself was another big culprit. Commercial Internet service went mainstream circa 1995, and suddenly, a lot of content that had formerly been the province of a hard core of dedicated hobbyists was accessible to everyone - most critically, online fanfic. Many folks, particularly among younger readers, found that online fanfic scratched the same itch as romantic fantasy; I recall a great deal of mid-to-late-1990s fanfic that basically applied the tropes and forms of romantic fantasy to video game settings, for example. (Chrono Trigger was an oddly popular choice - anyone old enough to remember that?)

This was compounded by mishandling by both authors and publishers. Though the new communication channels afforded by the Internet could have been a great boon to them, most romantic fantasy authors (correctly) perceived online fanfic as competing for their audience, and responded with extreme hostility. We’ve talked a bit about Mercedes Lackey; her stance on online fanfic was legendarily draconian, and often backed with litigation, to the extent that her nascent Internet fandom was basically smothered in its crib. By the time she mellowed out on the medium, it was too late. A lot of other romantic fantasy authors and publishers followed the same trajectory.

Lastly, the final nail in romantic fantasy’s coffin was basically J K Rowling’s fault, believe it or not. During the period in which romantic fantasy literature enjoyed its peak popularity, YA fantasy literature was in a low ebb; there wasn’t much of it coming out, and most of it wasn’t very good, so a lot of kids were reading romantic fantasy (in spite of its subject matter often being wildly inappropriate; I’ve mentioned in the past how many books about teenage girls having sex with dragons I ended up reading!). That youth demographic ended up being the last bastion of romantic fantasy’s mainstream readership - then the YA fantasy renaissance of the late 1990s stole that audience wholesale.

There were probably half-a-dozen other significant factors that contributed to romantic fantasy’s commercial decline, but those are the highlights.

I knew it was Rowling’s fault I couldn’t find “my” type of fantasy anymore! All of a sudden, everyone seemed to be trying to write the next Harry Potter. It was quite upsetting, as I had rather liked the fantasy genre the way it was before, back when it was generally agreed upon that magic ought to have actual rules :P I had no idea there was an actual name for this type of fantasy. I miss it dreadfully, though :( though, yes, certain scenes in the Mage Winds trilogy were pretty horrifying when I was ten… 

Another element in the decline was related to the development of the internet, but only tangentially.

In the late 80s and early 90s, anime and manga began to be licensed more and more in the Americas and Europe. At first, most offerings were male-focused and had a narrow audience, but with the shift from bbs and rec.alt. to free personal webpages (thank you Netscape!), information about series from Japan spread much faster. At this point, the fansub community boomed (no really, boomed to the point where there were distributors in countries all over the world, not just in college clubs), due to the ability to publish their catalogs and contact information more easily. This brought a variety of shoujo and josei series to the attention of a wider audience, specifically of women, and suddenly female geeks who formerly had been following Romantic Fantasy found out that entire swaths of television and comics were already dedicated to them in Japan. (You can thank Sailor Moon for the explosion of shoujo that decade. No, really. I’m serious.)

1995 was a big turning point. In a single year, while Sailor Moon was finishing up season S and moving on to Super S, the following powerhouse anime were released: Fushigi Yuugi, Magic Knight Rayearth, Wedding Peach, Gundam Wing, Evangelion, and Slayers. Of these, the first 3 were shoujo; Fushighi Yuugi was an ancient China-themed portal anime that pretty much nailed the Romantic Fantasy genre right down the middle, Magic Knight Rayearth was a mecha portal magical girl series, and Wedding Peach was a real world magical girl series. As for the others, Gundam Wing was intended as a shounen SF war story to reboot the Gundam franchise, but it ended up with basically a yaoi fanbase dominated by women (fandom-wise, it was the Supernatural of its day, but with more lead characters and less incest). Evangelion was a groundbreaking grimdark apocalyptic disaster as notorious as it still is famous, and its audience was pretty well split in every way imaginable, including on whether they hated it or not. The only unmitigated success of the year not to draw most of its fanbase from among women was Slayers.

The impact of that year and the following (1996 was the year of Escaflowne and Hana Yori Dango) was immediately obvious if you went to SF&F cons in the US. The cosplay shifted, the panels shifted, there was a lot of sudden interest from women in what had been presented as a mostly male genre often erroneously equated with porn. Many women I had formerly discussed Bradley, Lackey, McCaffrey, and Rawn with were now discussing CLAMP and Takeuchi-sensei and the best places to get reasonably-priced import manga.

So yeah: internet fanfiction, Rowling/Duane/the YA crowd in general, books by queer authors who didn’t encourage us to think of ways to die heroically, anime & manga, and of course Supernatural Romance. Romantic Fantasy was a genre so tenacious that it took that many blows for it to mostly fall (and I would argue that it still informs fantasy television today). Or, conversely, you can think of the need that women have to see fantastical stories that reflect us as so powerful that for over 2 decades it drove an incredibly diverse group of women to all converge on a genre that didn’t entirely satisfy most of them but on which they were totally willing to spend money, because it was a genre women were actually producing for ourselves, and nobody else was listening.

There’s a reason women dominate fic.

(via wirehead-wannabe)

wirehead-wannabe:

wirehead-wannabe:

argumate:

notraptor:

argumate:

real furries would treat the opposite sex with indifference or even aggression until the breeding season, while animals pretending to be humans (humies?) would prance around affectedly going oh look at me, a human, in estrus 24/7, concealing my current reproductive status from potential mates, ohhh,

Man, everyone’s always trying to deconstruct the furries. “Why are the wolves blue?” “Why do the sharks have horse ears?” It never ends.

That said, I propose that furries play fast and loose with zoology because the appeal of furry characters is not actually the animalistic features. It’s the triangular faces.

Compare the above designs with humans and their unattractive oval faces.

The attractiveness of triangular faces would explain why certain species are more popular than others among furries.

The appeal of triangular faces is not specific to furry designs either. Anime may also enjoy popularity due to its triangular faces.

good lord.

That Argonian one is a stretch imo

OKAY BUT

DID WE JUST COME UP WITH A UNIFIED THEORY OF WAIFUS

ALSO WHY HAS IS SPREAD TO DISNEY

WHAT IS THIS

(via academicianzex)

Happy Labor Day →