An introduction to this blog

birdblogwhichisforbirds:

storyskylings:

For a while now, I’ve been planning to write a story about a world without humans, but where a species of crow has evolved human-like levels of abstraction and technology. This started out as a coping mechanism for fears about human extinction - I imagine a world in which humans have ruined everything for ourselves, but maybe crows get to succeed where we failed. But once I realized how morbid that was, I switched to merely imagining it as some kind of alternate timeline of our world - just something that is interesting to think about, as a piece of fiction.

I had all sorts of extremely vague plots I tried out in this world. A murder mystery. War, or almost-war prevented at the last minute by Virtuous Protagonist. Romance. Various quests for various Maguffins. None of the stories really set me alight. The questions the plot ideas would force me to answer were interesting: e.g. a murder mystery raises the question of what legal system (if any) crows have, what violence looks like in crow world, what their morality says about justice and revenge, etc. But I didn’t actually care about writing a crow murder mystery.

It occurred to me that I don’t want to write a story about crows. At least not yet. What I want to do is build a world. I want to figure out what it is like to be one of these crows - their biology, their culture, their history, their morality, their food, their spirituality. This sounds fascinating and fun. It will also mean researching a lot of real-world things as inspiration for the imaginary world, which sounds like a cool way to motivate myself to learn.

My goal here, which I may modify, is to write at least one post per week on some aspect of the Crows’ world (I am planning on a minimum length of 1k words per post) every week in 2020.

At some point, I may want to start writing fiction set in the world I’ve built. I may also just want to keep on building the world indefinitely. Or I may just quit. I hope I don’t though.

A little note on the name: I’m never going to conlang a whole language for them (they do have a language, but I’m not fucking Tolkein and even if I was conlanging for species with a goddamn syrinx is not something I relish) but I do want to think about their word for themselves. Humans’ Linnaean name for ourselves is homo sapiens, wise man (obviously humans have many languages and many names for ourselves, but I don’t have the whole damn day). The homo part comes Proto-Indo-European word that means something like “earthling” (the same root that gives the latin humus, for soil.) So I’m going to steal from my own species and give this idea to the birds: the crows’ word for themselves is something best translated as “Skylings.” And, if these crows ever develop something like Linnaean classification (I don’t know if they will; I haven’t decided how their world goes) the word they will add is not wise, exactly. The thing they see as different about themselves is that, while most social species communicate, only crows tell stories. They are the Story Skylings.

This is a fiction/world-building project I’m starting. Please follow if you’re interested!

(via birdblogwhichisforbirds)

The compositionality of neural networks: integrating symbolism and connectionism →

In reaction to my big post from yesterday, LW user steve2152 linked to this super-interesting recent paper, which evaluates several neural sequence models (including the Transformer) on five different kinds hallmarks of “compositionality” after training them on a context-free grammar designed specifically for this purpose.

Not sure (yet, if ever) what high-level conclusions to draw from it, but it usefully divides the ill-defined question of whether neural nets “can do compositionality” into several well-defined questions whose answers don’t all look the same, which seems more illuminating than the sweeping claims of that Wired article by Marcus I quoted.

Also relevant: it explicitly tested for “overgeneralization” during early learning before special cases have been memorized, i.e. exactly the phenomenon Marcus and the connectionists were studying with verb inflection.  Among the various fruits of this particular investigation is this cool figure:

(Note that the pattern along the vertical axis does not match the pattern reported in humans with verbs, where children will over-generalize a rule even when it applies to <10% of the vocabulary.  OTOH, the gradual transition away from overgeneralization and the relative paucity of “blends” [grey areas] observed with the Conv and Transformer models resembles the humans-and-verbs case.)

fyi, as quote formatting is somehow *still* completely broken on mobile, all of your multi-paragraph quote formatted passages only show the first paragraph indented, making it very difficult to read this on mobile

Aw man, I’m very aware of this issue as a frequent user of the mobile app, but it’s always newly annoying when I notice how it affects any given post I’m reading or writing :(

FWIW, I cross-posted this one to LessWrong (…wait, what have I become???), so if you want a mobile-friendly version of this specific post, you can read it over there.

(There is at least one bug in the LW crosspost setup, a harmless but very weird one: it tries to parse my bold section headings and make them into a table of contents, but in both posts I’ve tried it misses some of them.  In this case, the table of contents lacks sections #2 and #6, of 7.  Your guess is as good as mine.)

human psycholinguists: a critical appraisal

(The title of this post is a joking homage to one of Gary Marcus’ papers.)

I’ve discussed GPT-2 and BERT and other instances of the Transformer architecture a lot on this blog.  As you can probably tell, I find them very interesting and exciting.  But not everyone has the reaction I do, including some people who I think ought to have that reaction.

Whatever else GPT-2 and friends may or may not be, I think they are clearly a source of fascinating and novel scientific evidence about language and the mind.  That much, I think, should be uncontroversial.  But it isn’t.

Keep reading

twocubes:

Why do people keep complaining about my plan to destroy the Earth? It’s largely uninhabited! If I shot a random point on its surface, do you know what the probability is that I’d hit any human? It’s less than one tenth of one percent of one percent!

(via transgenderer)

music-genre-robot:

Microblues

(via dharma-initiative-official)