The following seemed mildly interesting when it occurred to me, YMMV:
The idea of President Donald Trump would be shocking to many/most people 10 or 15 years ago, and part of their shocked reaction would stem from the sense that Trump is not a “serious person.” We expect reality TV stars (even if they are also real-life businessmen) to be the sort of people satisfied with the insulated pseudo-importance of reality TV stardom. Not the sort of people who would reach further, for the importance that comes from holding the levers of real power, from controlling life outside the TV.
(Trump did run for president in 2000, but I didn’t know it at the time, and I don’t think many did.)
That much is, I think, obvious. What is not as obvious is that the idea of Russian meddling in the election would ping the same absurdity sensor. Because people are worried about Russia having swung the election by doing things like spamming a web forum (Facebook), and 10 or 15 years ago, web forums were also part of the “unserious world.”
(Indeed everything about Facebook would have had this sense of disorienting unseriousness about it, from its sophomoric origins as a tipsy college nerd’s Hot or Not website to the way that nerd, now one of our foremost business magnates, seems perpetually fixed in a state of looking about 15 years old.)
One thing I’m curious about that was not addressed in the post is the role, in all of this, of computerized tests – specifically, “continuous performance tests.”
I had to take one of these – the TOVA (Test of Variables of Attention) – when I went in to get tested for ADHD in 2014. (I was in grad school at the time, and wanted to get tested for the same reasons as the “Senior Regional Manipulators Of Tiny Numbers” Scott talks about.) The tester said I didn’t have ADHD, and at the time I assumed my normal TOVA results weighed heavily in her decision, and (also) that this was normally how such things were decided.
But Scott’s post makes it sound like the usual procedure is a lot more of a human judgment call. He mentions a variety of things that prescribers do to make themselves feel better about their decisions, but none of them are “administer a computerized test with no human oversight and always follow what it says (or always do so unless you can think of a really good reason not to).” If nothing else, this would certainly reduce worries about human biases.
I say “if nothing else” there because the same thing would be true of any such test, even if it had no diagnostic value at all. (Then your decisions would suck – but even then, not because of your biases!) However, tests like the TOVA may indeed have a lot of diagnostic value. That is, they may have good sensitivity and specificity in discriminating controls from people with ADHD diagnoses***.
(There are even some studies showing it can discriminate these groups from people who are “faking bad,” i.e. malingering. This makes some sense if the distribution is light-tailed, e.g. normal, so that that if you overdo your faking by just a little bit you’ll stray from a region where 5% of the population lives to a region where 0.01% of it does.)
For one thing, if this is true, it means that we could just automate the whole process and get roughly the same results we were getting before, but without worries about human factors getting in the way.
Additionally, if true this is scientifically interesting, in part because of what it says about existing (non-computerized) diagnostic techniques. Scott’s post describes a very fuzzy, human process with a lot of variation between clinicians. But apparently this process has enough reliability to agree with a computerized test a lot of time, which would not be a priori obvious.
Moreover, if (as Scott says) ADHD is one extreme of a continuous/unimodal distribution, then we could use the TOVA to figure out where clinicians are already implicitly setting the cutoff. Scott writes:
We could still have a principled definition of ADHD. It would be something like “People below the 5th percentile in ability to concentrate, as measured by this test.”
We aren’t doing this, but what we are doing may be accidentally similar to it. The Schatz et. al. 2001 study, discussed further below, includes an ROC curve showing us how many false and true positives we get for various thresholds. The thresholds are for “T scores,” which are apparently like z-scores except the mean is set to 50 and the SD to 15, so that e.g. a threshold of 65 (the recommended one) means you say everyone who’s 1.5 SDs or more above the mean of the reference population has ADHD.
If everything were normally distributed, you could get quantiles out of this, and translate clinical behavior into cutoffs separating X% of the population from (100-X)% of it. (Well, sort of – the “reference population” here is neither the full population nor the non-ADHD population, it’s sort of a mixture determined by the selection criteria used to make the normative stats.) Of course, as usual, the people who made the reference stats don’t say anything about whether the distribution was normal. But this kind of analysis could be done by someone, in principle, anyway.
(***Caveat: the most widely cited study I could find on this was is Forbes 1988, which – astonishingly – was not blinded. That is, the TOVA was administered in the process of making the diagnostic decisions against which it was later compared, and were [Forbes’ words] “usually known before the final diagnosis was made.” Forbes goes on to claim that different TOVA results would not have flipped any of the diagnoses, to which my reaction is “okay, great, so if that was true, why did you show them to the clinicians at all?”
However, there are also studies like Schatz et. al. 2001 that give the TOVA to people who have already had a formal diagnosis done before the study started, and also to controls. There are still worries like “are we sure the original diagnoses didn’t use the TOVA or a similar test?” and “given our screening procedures for controls, what base rate of undiagnosed ADHD should we expect in our control population, i.e. how sure are we that some of our control ‘false positives’ weren’t true positives?”, so I still am not impressed with the evidence quality I’ve seen. That said, if you grant for the sake of argument that Schatz et. al. did things right, they get good sensitivity/specificity results too. Oddly, they interpret their results as bad news for the TOVA, on the basis that it does worse than a test based on parent ratings, but since the original diagnoses themselves involved parent ratings, this doesn’t seem like a fair/useful basis for comparison.)
People in 18th century France were waaaaaaaay into how sad this painting was:
On seeing the second version of the painter’s Girl Weeping over Her Dead Canary in the Salon of 1765, Charles Mathon de La Cour placed the girl’s age (around eleven) as exactly at the stage where “Nature begins to soften the heart to receive the sweetest impressions,” with the result that her tears were both childish and pre-adult. He then went on to examine in great detail the painterly treatment of this damp sorrow:
One sees that she has been crying for a long time and that she has finally given herself over to the prostration of a profound grief. Her eyelashes are wet, her eyelids red, her mouth still in the contraction that brings on tears; looking at her chest one can also feel the shudder of her sobs.
“Connoisseurs, women, fops, pedants, wits, the ignorant and the foolish,” he claimed, were “all of one mind about this painting,” for in it “one sees nature, one shares the grief of the girl and one wishes above all to console her. Several times I have passed whole hours in attentive contemplation so that I became drunk with a sweet and tender sadness.”
(Simon Schama, Citizens)
And here’s just a small excerpt of Diderot’s effusion on the subject:
Was randomly thinking about this post while reading the new installment of Terra Ignota – a series whose characters do a lot of that specifically 18th century kind of weeping
So what on the surface looks like a matter of bad acting vs. good acting reveals itself as a conflict between subjectivism and post-modernism on the one hand and dialectics and materialism on the other.
