internet explorers (not exploiters)

I was reading this post from 2013, about addictive games and game-like websites (such as Facebook), and the possibilities for “addictive” educational resources that could compete with them.  It was interesting, but something felt off about it.

I found myself fixating on this passage [emphasis mine]:

Somebody who is tackling a truly novel problem often feels at a complete loss, having no idea of whether they are even on the right track. Someone who is facing a tough problem that is known to be solvable, but which nobody has yet turned into an addictive one, might feel similarly. If we motivate people to work by giving them frequent external rewards, does that train them to become even more impatient and eager to quit in cases where no such rewards are forthcoming?

Apparently, Western cultures are already doing badly with this. According to this NPR piece, American first-graders who were given an impossible math problem to work on tended to give up within less than 30 seconds. Japanese students, on the other hand, spent a whole hour trying to solve it, stopping only when the researchers told them to. We Westerners have already been trained to desire instant gratification, and it might not be a good idea to turn society even more in that direction.

I am not at all sure that we have a choice, however. It is all well and good to say that we should stop being so focused on instant gratification and train ourselves to work on problems for longer before giving up. But how are we going to do it in a society that keeps becoming ever more addictive? Most people have big projects and noble goals that they say they intend to accomplish, one day – and then they never do, because there are easier rewards available. “We should train ourselves to have a longer tolerance for uncertainty and delayed rewards” is exactly the kind of goal that provides an uncertain reward sometime late in the future… and is thus likely to be put aside in favor of easier goals.

I’m not sure the blogger is saying that the behavior of the Japanese students was strictly better than that of the American students.  I think the argument is just that “Western cultures” have swung too far in one direction, that in the short term it would be helpful to take local steps toward the mindset of the Japanese students.  It’s not a Goofus vs. Gallant thing.

Still, I think “delay gratification more” is difficult advice to act on, in a way the blogger is not accounting for.  This is because “how much should I tolerate delayed gratification?” is an enduring, intrinsically difficult question, one that will always trouble us as long as we are still alive and making decisions.  It’s a question that has been academically formalized as the “explore/exploit tradeoff,” which comes up for instance in the design of computer programs, which have no will of their own.  (That is, we can make the programs delay gratification as much as we like without them “complaining,” but this doesn’t actually lead to the best results.)  And I think the explore/exploit tradeoff is a very important concept if we want to understand why the internet is so addictive.

If you aren’t familiar with it, here’s the basic gist.  (There are probably better explainers somewhere if you feel like Googling for them.)

Say you are designing a machine that makes choices.  (I’m using a machine example here, rather than a person, so it’s clear that the tradeoff arises even when you don’t have a desire for instant gratification built in at the start.)

Your machine doesn’t know everything about the world.  So there are two reasons it might choose an action:

(1) it knows what will happen as a result, and likes it

(2) it doesn’t know what will happen as a result, so it will get new information from taking the action, even if the result is bad

Type 1 is called “exploitation” and type 2 is called “exploration.”

As a concrete example, imagine that the machine gets “points” every time it enters a room.  There are ten million rooms.  It knows that it will get 2 points for entering Room One, 1 point for entering Room Two, and 3 points for entering Room Three, but it doesn’t know how many points the other rooms deliver.

The best room it knows about is Room Three.  The “pure exploitation” strategy would just be to enter Room Three over and over again, forever.  But clearly this is a pretty silly strategy.  It only knows about 3 out of 1,000,000 rooms, and what if one of the others give more than 3 points?  Hence it may want to do some “exploration,” not just “exploitation.”

To learn the answer, though, it will have to take the risk of entering some rooms that might give less than 3 points, so it might lose out in the short term.  And there are so many rooms that it would make sense to settle down and exploit at some point, even if there are still rooms it doesn’t know about.

(The classic formal version of this problem is called the “multi-armed bandit,” BTW.  If you like math/CS there’s a lot of interesting stuff out there on it.)

This framework makes sense for thinking about instant vs. delayed gratification, although the connection can go in two opposing directions.  On the one hand, we need a tolerance for delayed gratification in order to explore, because exploration involves trying many things that aren’t as good as the best thing we already know about.  (Are you going to try a new recipe that might suck, or just make that easy dish you know you like?)  On the other hand, if we always stick with the same thing in the hopes that it’ll pay us delayed rewards sometimes, we are exploiting when we should be exploring.  (If you’ve been “working on” that unsatisfying job/relationship/whatever for 10 years, hoping that maybe next year it’ll finally improve, you might be too tolerant of delayed gratification.)*

When I read the anecdote about the American and Japanese students, I immediately thought back to my time in grad school.  Every day of work in grad school was a vivid explore/exploit dilemma: I was working on a project, and I had various ideas I was pursuing, many of which would probably not work out in the end.  Each day, I could plug away at my latest idea (exploitation), or I could spend time exploring – thinking about other possibilities, trying to critique my existing ideas, trawling Google Scholar to see if someone’s done it already.

This was never a Goofus vs. Gallant, good responsibility vs. bad hedonism sort of decision.  Exploration was scary: I could spend a whole day thinking or reading without anything to show for it, except even stronger misgivings about everything I’d done.  But exploitation was also scary: what if I was wasting time on something clearly misguided?  (I’ll never forget the time I spent weeks working on a beloved idea while my adviser was out of town, wrote up 10+ pages of TeXed notes on it, presented it proudly on the whiteboard to my adviser and a colleague, and watched them sit in impressed silence for about 30 seconds, after which my adviser walked up to the board and wrote a two-line proof that my idea could never possibly work.)

Indeed, one of the biggest psychological pitfalls was that exploitation felt so responsible.  If I was writing equations and doing math, I was “getting things done,” not just screwing around.  I was delaying gratification, putting a big project together step by step, carefully and dutifully.  This responsible feeling, though, got me into plenty of situations like the one I just described.  Delayed gratification can be dangerous!

*(Boring technicality paragraph: this is all a bit circular.  If we really didn’t care whether a reward happens now or in a million years, there wouldn’t be as much of a reason to prefer exploitation over exploration.  There still may be a reason if there’s an infinite choice set, but that’s a mathematical curiosity; more relevantly, any machine or human project has a finite time horizon, and besides, time is money.  If you spend 10 years agonizing over finding the perfect job posting to apply to, you are losing money by exploring too much.  No messy value judgments there, just concrete $$.)

Back to addictive games and the internet.

It’s tempting to lump together addictive video games and addictive internet behavior, since they’re both recent phenomena involving digital technology.  But I think they’re really very different.

Addictive games are all about exploitation.  They may have internal explore-exploit dilemmas: maybe you are trying to get the most points, and you have choices like the “rooms” above.  But when you’re entranced by the game, it makes you forget that you value things other than its points.  You end up spending three straight hours trying to get more Tacky Phonegame Points, not thinking “do I really have nothing better to do with the next three hours than mining Tacky Phonegame Points”?  This is why they’re draining and ultimately un-fun, like chemical addictions: you end up pursuing the game’s internal objectives even beyond the point where you stop truly wanting them.

Addictive internet behavior, though, is all about exploration.  That blog post mentions checking Facebook notifications, which do have a certain resemblance to Tacky Phonegame Points.  But the vast majority of the time I spend on sites like Facebook and tumblr isn’t spent following up on notification, it’s spent scrolling and refreshing the feed.  I am not pursuing any explicit score here; the website presumably records how much time I spend reading the feed, but it doesn’t give me any kind of feedback about it.  You have Follower counts and Friend counts, but you don’t have “Number of Posts Read” counts.  Yet I read a whole lot of posts.

Another distinctive feature of these sites, which is totally different from addictive games, is their heterogenous content.  When I scroll down my feed, I have no idea what I’ll see next – a joke? a serious political statement? a personal lament or celebration?  There is an “instant gratification” element here: I’m curious what the next post will be.  But there is also a strong exploration appeal.

The internet is better than anything else in the world at making you aware, at all times, there there may be better Rooms out there.  You tend to end up reading one tab of 20, acutely aware that there are 19 other possibly-better reading choices a mere click away.  Everyone knows those social media posts that make the plaintive appeal, “why isn’t anyone talking about this?”  But even when this is not explicitly said, it’s always the subtext of the internet.  There are always so many things you aren’t (yet) talking about, thinking about, reading about.

A typical breeze through my feeds in the morning might expose me to 2 important-sounding political issues I’ve never heard of, and 2 noteworthy developments in the lives of friends or acquaintances, and 2 things that trigger thoughts I want to write down in my own posts, and 5 or 6 articles or blog posts that I will open in new tabs because they give me an “I ought to know about this” feeling.  Which of these opportunities should I spend my day pursuing?  I’ve only just woken up, and already I have so many responsibilities – and that’s if I want to “waste the day on the internet,” instead of “getting real work done”!

The internet connects us to many people on ambiguous terms, in relations that are somewhere between friendship, acquaintance-ship, fan-ship, etc.  Unless we spend all day reading the feeds, we are constantly – by the standards of real-world friendships – “falling out of touch” with people.  So many personal situations I could pay more attention to, but at the cost of not reading up on the latest political thing I ought to know about, or reading that oh-god-so-interesting blog post.

And on the internet, we no longer have any excuses for not “educating ourselves.”  That important issue that more people should be talking about?  You can’t say you don’t have time to go to the library.  You probably haven’t even read the Wikipedia page, or all 10 of the must-read long-form journalism pieces, much less all the acerbic must-read blog critiques of those pieces (what kind of sucker are you, not being up on the acerbic critiques?).  Meanwhile, of course, you’ve been neglecting that tab about the different local variants of Polynesian mythology, which maybe isn’t as “important,” but which is super interesting, and which was giving you ideas for your fiction project earlier on, and wouldn’t it be a shame if you forgot about it and your fiction suffered as a result?

So much exploration to do.

Why isn’t anyone talking about Room #130,327?

BRYCE
What about the massacres in Sri Lanka, honey? Doesn’t that affect us, too? I mean don’t you know anything about Sri Lanka? About how the Sikhs are killing like tons of Israelis there? Doesn’t that affect us?

BATEMAN
Oh come on, Bryce. There are a lot more important problems than Sri Lanka to worry about. Sure our foreign policy is important, but there are more pressing problems at hand.

BRYCE
Like what?

BATEMAN
Well, we have to end apartheid for one. And slow down the nuclear arms race, stop terrorism and world hunger. But we can’t ignore our social needs, either. We have to stop people from abusing the welfare system. We have to provide food and shelter for the homeless and oppose racial discrimination and promote civil rights while also promoting equal rights for women but change the abortion laws to protect the right to life yet still somehow maintain women’s freedom of choice.

I think the internet has made it a lot harder for me to read books.  Really, it’s made it harder to focus on a lot of things, but reading books stands out, since it’s such a close substitute for what I do on the internet (mostly reading text of some sort), and because it’s something I want to do more of.  I don’t mean that in an aspirational, self-improvement way; the books are actually more interesting, in an immediate-fun way, than the internet.  Yet I choose the internet.

The problem here is not weak-willed hedonism.  Weak-willed hedonism is when, instead of reading a book or browsing the internet, I choose to play Starcraft.  When I make that choice, I am plunged into an almost non-conscious exploitation void, I get a lot of Points, and I come out on the other end 30 minutes later as though nothing had happened, without no learning or thinking in between.

No, this is something else.  You know what’s weird?  The reason I have so much trouble reading a book – any book – is that it feels complacent.  It feels like writing equations all day that my adviser is going to demolish on the board at our next meeting.  It feels like exploitation in a world full of visibly under-explored possibilities.  Even “important” reading, reading on “things I should know about,” feels palpably excessive, frivolous, an overblown luxury.

350 pages about Sri Lanka alone?  Come on, Bryce.  There are a lot more important problems than Sri Lanka to worry about.

The internet connects us to many many people, which makes us aware of all the possibilities for exploration.  To compound the problem, it also makes us aware of the worst pitfalls of exploitation.  It routes us directly to all the world’s crackpots and single-issue obsessives, who we all put in tabs so we can guiltily skim their work and think about how much more worldly we are than them, how much more aware.

The most superficially charismatic people on the internet (to me) are the people who are the opposite of these obsessives, who seem to know (a little) about everything, who always have the latest must-read links, who expose you to new things every day.  The Batemans, not the Bryces.  These people presumably have relatively shallow knowledge of any one thing, but since we’re all hyper-explorers on the internet, we don’t have enough time to exhaust their knowledge of any one thing.  The internet doesn’t show me “professor spends 10 years working on topic, contributes steadily to human knowledge”; it shows me “professor fields ignorant opinion on twitter; should have known better.”  Good thing I know better, I think, clutching my 20 tabs to myself like armor.

I don’t think traditional notions of addictiveness are sufficient to describe this particular problem.  It isn’t a matter of just choosing the responsible thing.  The problem is that Bateman is (figuratively) right.  There are a lot of things to worry about, and now we’re hyper-aware of this fact in a historically unusual (unique?) way.

And our traditional notions of responsibility, of instant and deferred gratification, were shaped in a world where there were much stronger barriers to exploration.  Where you could still say, truthfully, that you hadn’t educated yourself on an issue because you just didn’t have time to go to the library.

Now there really are millions of Rooms a few clicks away, and any one of them might be a very important Room.  And the question is what we should do in that situation.  And there is no good, traditional answer.  This is a new problem and we need new answers.