Recommendation Technology: Techno-Boon or Nightmare?

Collaborative Filtering Software is one of the biggest Web inventions to move off-line. But do we think enough about the things we make? Orwell would have jumped right off the nearest bridge. This "Big Brother" technology is coming soon to a supermarket and discount chain near you.

"Being in a minority, even a minority of one, did not make you mad. There was truth, and there was untruth, and if you clung to the truth even against the whole world, you were not mad." -- George Orwell, "1984"

Orwellian is a term used so often in so many knee-jerk ways it doesn't really have much meaning anymore. It's just another a cliche. Too bad, because it has plenty of relevance.

Orwell's hero Winston in "1984" lived in a world where Big Brother used technology to observe and police every minute of every day in the life of every citizen of Oceania.

The irony is that in our time, government is much too ham-handed and clunky to use technology so effectively in invade and threaten our lives. They don't know enough about it. Do we have a thing to fear from a government that has the House Rules Committee in it?

The scariest elements in our society -- the places Big Brother would feel the most at home -- are the mega-corporations, which Orwell never imagined and the existence of which would have sent him leaping right off of the nearest bridge. Microsoft is more disturbing on its best day than the federal government on its worst. (The Halloween document had to have been invented by Orwell, I thought when I first read it.)

Corporations gobble up data about us, take over our culture, seek to control our information, curb the things we say and the way in which we say them, gobble up free spaces wherever they find them, mass-market creativity and individuality out of existence whenever they can. And increasingly, they use technology to gather information and monitor our existences in ways Orwell never even dreamed off. All Winston faced was that tele-screen and some incompetent Thought Cops.

We have to deal with new advances like Recommendation Technology (RT), also called Group or Collaborative Filtering Software, also known as "suggestive selling." Almost everbody who's bought something online is familiar with these programs -- the recommend products for us on Amazon, CD Now, or Moviefinder.

And RT is only going to become more ubiquitous. This new technology is converging with very big bucks: according to Jupiter Communications, online sales could top $40.8 billion by 2002.

Jupiter says that in a recent study of online merchants, 40 per cent said they already use recommendation technology, and 93 per cent say they plan to. And this technology is moving rapidly off-line and into retail chains, hardware stores and supermarkets near you.

But "recommendation" is a misnome for this technology. The idea isn't just to suggest things to the consumer, but to gather unprecedented amounts of information digitally to influence how the consumer behaves.

Not only will websites suggest books and CD's but your neighborhood cashier will remind you that you forget toothpaste or toilet paper.

Perhaps this is just another techno-boon: useful information for the hard-pressed, bombarded consumer. But if history teaches us anything, it's that ill-considered technology is dangerous technology. The people who came up with all those pesticides had no doubt but that they were doing the world good, as did the makers of the Bomb.

Our media is much too busy, along with government, worrying about sex on the Web to consider behaviorial technologies like this collaborative filtering that might make it nearly impossible for any of us to live unobserved lives, or to retain our own personal definitions of dignity, privacy or indepedence.

And that is exactly what Orwellian means.

Collaborative filtering programs gather information about our shopping habits and use the information to convert browsers into buyers and buyers into more frequent buyers.

Unlike good old fashioned salesmanship -- Harry at the hardware store reminding us to store up on salt for the winter -- recommendation technology is designed to change our behavior, to get us to buy more of some things, or other things we might not have thought to buy. It is also organic, not a single static software program but a living, evolving one, one that becomes steadily more efficient and useful over time, as it learns more about us, in ever greater detail. If they know to remind us to buy beets after just one Spring, think of all the advice we'll get in five. And as companies share all of this personal information - as they invariably will -- think of the collective behaviorial sketches of all of us that they'll be able to pull together.

In this kind of a culture, why should government threaten us? All they have to do is sit back and rake in taxes?

Americans are notoriously gee-whiz and unthinking when it comes to technology. There's always a trade-off: cars give us mobility, but stink up the air.

Remember those miracle McCaughey sepuplets? Reporters and corporations were all so busy oohing and aaahing at the medical marvel that the warnings of physicians and ethicists about the extraordinarily high dangers of multiple pregnancies like that one -- birth defects, chronic disease, psychological problems -- were almost completely ignored. The McCaugheys were bombarded with free diapers, baby food and furniture. Let's see how gushy reporters and politicians are when a welfare Mom in Chicago gets some of that medication and has seven or eight babies she can't take care of? Will the President call her up?

It's impressive to see just how well my Recommended Books program on Amazon is getting to know me. The site can reliably pick books that I like to buy about subjects that it knows I'm interested in, something my wife and closest friends can't do. Each month, Amazon knows more about me and what I've bought and read. Amazon is my good literary friend. Is this any different than the old corner bookstore owner, who remembered what I liked and recommended new books to me?

Maybe not.

From Mary Shelley to Orwell, writers have been sounding the alarm about the dangers of unthinking technology for centuries now, and about intrusive forces of evil watching us and changing our lives. In our time, only the madmen -- people the Unabomber -- seem to want to talk about it at all. Because we can, we do. Because we do, it's great.

In an joyful piece on the spread of Recommendation Technology, American Demographics enthused about collaborative filtering in its November, l998 issue:

"Pumping out these recommendations is a technology called collaborative filtering. Unlike mass-marketing tools, collaborative filtering doesn't rely on democragphic or psychographic profiles. Instead it looks at individual consumer's behavioral data -- such as purchasing history and stated preferences -- to predict the future behavior of like-minded people."

And then this sub-hed: "This Big Brother technology records your buying habits and tastes online. And it could be coming to a store like you."

Jeez, even they call it "Big Brother" technology. That send me running for "1984."

As this technology spreads rapidly off-line, the little considered issues are these: Do we all want to be known that well? Will it be possible in coming months and years for us to choose to keep our personal lives and habits -- as revealed through the books, clothes, food and other things we buy -- private? Do we trust giant, powerful, obsessively greedy, amoral and market driven companies to use this information wisely and well? Do we want them to have the kind of power we would never give to government?

It's one thing for these questions to be pondered, and then put aside. It's another for this brave new world to be upon us without most of us people ever had the chance to make any real choices about their participation in it, or to understand what it will mean to them. Here on the Web, we're caught up in the revolutionary thrill of creating something new. We rarely take the time to wonder about about some of the things we're doing.

That was exactly the problem that landed poor Winston in an Oceania jail in "1984", brooding about his plight, trying to reassuring himself:

"He was safe, everything was all right. He fell asleep murmuring "Sanity is not statistical," with the feeling that this remark contained in it a profound wisdom." (You can e-mail me at [e-mail])