4point2kelvin:

nostalgebraist:

I can’t imagine I’m the first person to have this idea, but: I’m starting to think that, at least with currently existing technology, it’s always a bad idea to think of your software as an “agent” instead of a “tool.”  And, on the flipside, that many (if not all) useless “agents” could be repurposed into useful “tools.”

The distinction I’m making is between two types of software that try to save you time:

A “tool” saves you time by giving you a (literal or figurative) “button” you can press (could be a command line string, whatever) which will trigger a constrained, broadly transparent, broadly predictable (if perhaps quite complicated!) string of automated actions, which you then won’t have to do yourself.

An “agent” instead tries to do some task for you entirely on its own, up to and including deciding when the “button” should be pushed, and then pushing it.  A tool is never used by default, only when you push its button (perhaps by implication, through pushing the button of a higher-level tool).  Agents push their own buttons, whether you want them to or not, and usually the only way to push back against an agent that is behaving annoyingly or uselessly is to turn it off entirely, depriving yourself of all its functionality even when you do want it.

Agents and tools are often quite similar in their actual capabilities, and in what they do after the button is pushed.  But agents are more opaque to the user, often on purpose (to make them seem “smart” and/or effortless to operate).  And the model of time-saving is subtly, but importantly different in the two cases.

A tool wants to multiply your capabilities.  It wants to let you do more in any given ten minutes by giving you a button that will instantly do something that used to take you (say) five minutes.  Now, every time you want to do that thing, it’s as if you’re getting five extra minutes for free.

An agent wants to replace you.  It wants to let you do more in any given ten minutes by making a conceptual breakdown of your work, completely automating a subset of it, and posing as a new coworker who handles that entire subset so you don’t have to.

Why are agents worse than tools?  First, because we’re really good at making computers do complicated-yet-constrained tasks, but we’re really bad at making them anticipate our needs.  “Deciding when to push the button” is usually very hard to automate – and strangely pointless to automate, too, when it takes all of half a second to push one.

And second, because the space of conceivable, useful tools is much larger.  As long as you leave room for some human volition, you can have all sorts of great ideas that can multiply human productivity by orders of magnitude.  If you insist at the outset that the human is going to be cut off from the system, then you won’t even think about any of the ideas that necessarily involve a human participant, even if they’re great ones.

Think about compilers and interpreters.  Once upon a time, you (more or less) had to write byte code if you wanted to program a computer.  These days, the journey from “concept in your head” to “code you can run” takes orders of magnitude less time, because we now have tools that will automatically translate much higher-level descriptions into byte code.  After their buttons are pressed, these tools do all sorts of very complicated, very fancy things entirely on their own, in a way that is quite “smart” if you want to frame it that way – it seems to me that GCC and LLVM are as worthy of the title “AI” as anything on the market these days.  But these things don’t pose as coworkers or assistants, they only run when you tell them to, and they limit their behavior to an easily comprehensible scope.

Imagine if people in the days of byte code had thought about “automated programming” in the agent model instead of the tool model, with the goal of entirely replacing parts of the programmer’s workflow.  Would they have invented programming languages at all?  “Why translate into byte code from a language humans find easier, when the goal is to write code without the human needing to lift a finger?”

Compilers and interpreters are complex tools, and they are wonderful.  Are there any agents that are similarly wonderful?  When a software feature is marketed as being “smart” (which seems to be a term of art for “agent”), doesn’t that usually mean “useless”?

(The phone app that came with my sleep tracker has a feature called “Smart Coach.”  Each day, in semi-random fashion, it gives me a new piece of advice based somehow on my recent data.  The software capabilities behind the advice look like they might well be very useful to me, but they have been rendered useless by wrapping them in an agent.

“Smart Coach noticed you tend to sleep more on weekends.”  Okay – but how much, and over what time period, and is there any reason you told me that just now?  An ability to see my data averaged by day-of-week (which the app is already computing, apparently) would be so much more useful than Smart Coach.  “Smart Coach noticed you got less deep sleep last night than usual for your age cohort.”  Okay, so how much deep sleep does my age cohort get, and just how much less am I getting?  The developers put lots of interesting information at my fingertips, and then systematically hid it from me, because they wanted an agent.)

High-frequency stock traders. Airplane autopilots. Thermostats. Industrial control systems of all sorts. Game AI. Roomba. Ad-serving AI.

Is an ATM a “tool” I am using at some specific time to make a transaction, or is it an “agent” that the bank started running in the past to act on its own? The distinction is somewhat fuzzy.

The key to identifying useful agents, I think, is divorcing this category from “things built to pretend to be people.” The things we have built to pretend to be people must pretend to be agents, because people are (on average) agents, but the things that actually generate value by replacing human decision-making tend not to have to pretend to be people.

Yeah, I admit the distinction is pretty fuzzy, especially when we are looking at examples of well-established, successful automation.  Upon reflection, framing this as a dichotomy might be more confusing than helpful – although I still think there is an important concept there, and so I’ll look for a less dichotomous way to phrase it.

The principle could perhaps be phrased as, “don’t be afraid to prompt the user for input at points where you don’t yet have a good replacement for their decisions; fight the temptation to automate even at these points, just so you can say the thing is fully autonomous and can run without a user at all.”

Thermostats might be a useful example here.  Before thermostats, when you had to exert a lot of manual control to keep your house at the right temperature, it would of course have been pleasant to fantasize about a machine that would “do all that work for you.”  At this point it would not have been obvious that a thermostat was what was wanted – that is, a device that still relies on you to tell it the desired temperature, and then automatically adjusts the furnace.  Instead, one might well have just fantasized about a machine, any machine, which would “handle the furnace” for you.

But “handling the furnace” splits up into a part that is straightforward to automate – the thermostat – and a part that is not – deciding on the target temperature.  This is perfectly fine, because you can eliminate all of the tedium (manually stoking a furnace and adjusting its valves and dampers) by just automating the first part.  But this does require giving up on the fantasy of a machine that would simply “handle the furnace” for you and let you forget it even exists.

You have to give up on the fantasy of something that literally runs itself to achieve the reality of something that more or less runs itself, and I think that’s true in a lot of cases.

(via 4point2kelvin)