Philosophical problems typically arise from the clash between the inevitably idiosyncratic features of special-purpose concepts —true, good, object, person, now, necessary — and the scientistically driven insistence upon uniformity.
Paul Horvich (source)
People who follow MIRI are probably aware of the extensive, somewhat negative evaluation of MIRI that OpenPhil did in fall 2016
But one thing about this evaluation that took me a while to notice – and which I suspect a lot of people still aren’t aware of? – is that OpenPhil released a supplement (PDF) containing the text of the anonymous external reviews, as well as MIRI’s internal reviews of the same papers, and MIRI’s responses (PDF) to the external reviews
(It’s linked in the evaluation post, but it’s one little link in a long post, and is easy to miss)
The external comments are very interesting, both from the general “what do academics think of MIRI’s work?” perspective and for many of the specific things the reviewers say.
For instance, the one external review for a paper on reflection was quite positive, while the external reviews for the two papers related to logical induction were much more negative. These reviews were done before the LI paper was published; in the evaluation post, the material in the LI paper is referred to as the mystery-shrouded “Result 5,” as in this noteworthy sentence:
MIRI feels its most recent result (the unpublished Result 5) is more impressive than its older work, but we will have substantial uncertainty on this point until the work has been written up and we have had a chance to review it more thoroughly.
Below, I quote some noteworthy comments by the external reviews.
On the reflection paper:
As far as I know, this paper gives the first demonstration of a system that can support an arbitrarily deep tower of reflective reasoning within a general-purpose mechanized logic. […] It’s a new and intuitively appealing capability that is nontrivial to implement.
[…]
I see these results as an important milestone toward formal analysis of systems with some level of self-understanding.
On the “Logical Coherence” paper, one of two predecessors to the LI paper:
As I said, I don’t find the results of this paper particularly interesting. The first paragraph suggests that this problem is motivated by the concern of assigning probabilities to computations. This can be viewed as an instance of the more general problems of (a) modeling a resource-bounded decision maker computing probabilities and (b) finding techniques to help a resource-bounded decision maker compute probabilities. I find both of these problems very interesting. But I think that the model here is not that useful for either of these problems. Here are some reasons why […]
I see no obvious modification of uniformly coherent schemes that would address these concerns. Even worse, despite the initial motivation, the authors do not seem to be thinking about these motivational issues. […]
Additional comments: I’ve looked (not very carefully) at 2-3 other MIRI papers, and I had much the same reaction in terms of motivation. These are smart guys, but they have no real computer science sensibilities (although their steering committee certainly has terrific folks with great CS sensibility!).
More on the “Logical Coherence” paper, from a second external reviewer:
Whether these assurances, and the related algorithm, have important significance is a matter for debate. It is to a large extent a subjective question. This reviewer is not extremely impressed but others might feel differently.
What I would have liked to see are concrete natural examples where their algorithm assigns some natural probabilities and prior constructions do not.
Also, there is an inherent issue with algorithms that work by enumerating over all proofs. They run in exponential time and even practically it seems that this enumeration will quickly explode before we see any reasonable probabilities.
On the “online learning with unbounded delays” paper, the other predecessor to the LI paper:
The observation that under unbounded delays no algorithm can compete with the best expert in hindsight, albeit a trivial one, has not appeared in the literature before. The same is true for the particular positive result proven.
[…]
There are several problems in the proofs and the validity of the results cannot be established (though the reviewer suspects that the proof can be fixed with considerable effort). In particular, there are important problems in the proof of Lemma 7, which is a main ingredient of Theorem 5, the main result of the paper. For example: [reviewer goes on to describe 3 perceived errors they found]
[…]
The results are vaguely relevant for this problem [i.e. logical induction].
First, the requirement of asymptotic consistency (in the sense of matching the loss of a Bayes-optimal expert) is on one hand very weak (asymptotics is “easy”), while at the same time, the particular assumptions, being able to access a Bayes-optimal expert and the strong convexity of the “loss”, are very strong. In any remotely practical setting, we don’t expect these to hold and there is not much novelty in achieving asymptotic consistency by comparing losses. […]
Finally, even if this interpretation is correct, it is only made possible by a computationally expensive algorithm under the strong assumption of having access to a Bayes-optimal forecaster, and only for strongly convex losses. Note that while the authors are trying to provide an answer to this question “in principle”, computational concerns do matter for this specific question, as discussed next.
As usual, online learning of course offers a powerful set of tools for addressing prediction problems, but online learning can only do so much. In particular, one can push online learning towards philosophy (lacking any practical relevance), e.g., by using infinite expert sets and unbounded computational power, as done here. But it is rather intriguing if not self-defeating if this is done in the context of trying to answer a question that is derived from the lack of sufficient computational resources!
More on the “online learning with unbounded delays” paper, from a second external reviewer:
This paper’s conclusions add very little to what is already known. The problem is that the setup considered in this paper (of unbounded delays) is far too general to be analyzable theoretically or useful practically. […]
I do not think these results any significant light on standards for good reasoning under deductive limitations. The primary reason is that the setup is far too general and too unrealistic, and the algorithm completely impractical. It would be far better to model the problem of delays more realistically and come up with efficient algorithms to deal with delay.
First, we build a machine learning model that predicts a post’s quality (q) by training a binomial regression model using only textual features extracted from the post’s content, i.e. q is the predicted proportion of a post’s up-votes. This way we are able to model the relationship between the content of a post and the post’s quality. This model was trained on half the posts in the community, and used to predict q for the other half (mean R = 0.22).
We validate this model using human-labeled text quality
scores obtained for a sample of posts (n = 171). Using
Crowdflower, a crowdsourcing platform, workers were
asked to label posts as either “good” (defined as something
that a user would want to read, or that contributes to the
discussion), or “bad” (the opposite).
Basham, he noted, was smart, knew model tricks about posing and makeup, and used social media hacks such as SEO and A/B testing. (“For example, although her Instagram photos are G-rated, any hint of side-boob adds at least 10% to her engagement.”)
n.b. this is probably one of the *less* surreal passages from that article
Yeah, the whole article is pretty fun, at least if you’re in the mood to read about Scott Adams being very Scott Adams
forget about sex, the important thing in life is to know that benjamin franklin wrote a book about maize in which he described popcorn like this “each grain bursts and throws out a white substance twice its bigness.” that’s what really matters chums.
Bad WSJ article I will not link here, talking about this good Vox article:
Could we be approaching the end of the Bernie Sanders era in liberal political thought? The pendulum has been swinging so fast and so far in the direction of radical leftism that even mainstream publications have become comfortable dismissing endorsements of western civilization itself as “alt-right” nativism. But a new book from a chaired history professor at Duke University could represent the end of the cycle.
[…]
But it’s not just libertarians who cite problems with the MacLean rendering of history. At Vox of all places, two academics who describe themselves as “on the center left” have written a piece entitled, “Even the intellectual left is drawn to conspiracy theories about the right. Resist them.” […]
Could it be that progressives have been moving left so quickly that the young adults at Vox have already become – at least in relative terms – the moderate elder statesmen of the movement?
The Vox article was by Henry Farrell and Steven Teles, both seasoned academics. Farrell is 47; I can’t find Teles’ age online but he got his BA in 1989, so he’s probably around 50. Farrell has been a prominent left-wing academic blogger for many years (he’s one of the main contributors to Crooked Timber). How the hell are these guys “the young adults at Vox”?
Perhaps the author means the young adults in charge of Vox, like Ezra Klein, who gave the article a venue. But like, Ezra Klein? Am I supposed to be surprised that he is less far left than people who write for Jacobin? Outgroup homogeneity bias is a hell of a drug.
Basham, he noted, was smart, knew model tricks about posing and makeup, and used social media hacks such as SEO and A/B testing. (“For example, although her Instagram photos are G-rated, any hint of side-boob adds at least 10% to her engagement.”)
I had a dream last night that I was at a party and Jeff Bezos was there. He had become disconnected from reality, like a mad king. Amazon had released some sort of word processor / text editor, which was entirely tailored to Bezos’ apparently very strange preferences in such things, and which almost everyone else considered unusable. At the party, he simply ranted on and on in defense of this program. I ended up talking to him for several hours out of some weird kind of sympathy, since everyone else there quickly got tired of him and started avoiding him.