Database Nation: What did you do today?

When I was a teenager, I tried keeping a diary. I took out my fountain pen every night before I went to sleep and wrote down the details of the previous day. I had just started dating and soon the book's pages were filled with stories of my teenage romances: I'd write down the details of my dates themselves: who they were with, where we had gone, what we had eaten, and what we had done.

After a month or so I had created quite an impressive historical record of my teenage exploits. But as time passed, my entries started getting shorter and shorter. It was just too much work to write down all of the details. Ultimately, my project collapsed under the weight of its own data.

Keeping a diary in today's world would be much easier. Every time I buy something with a credit card, I get back a little yellow slip telling me the exact time and location of my purchase. I get a much more detailed receipt at my neighborhood supermarket that lists the name and size of everything in my shopping cart. My airline's frequent flyer statement lists every city that I've flown to over the past year. Should I accidentally throw out the statement, all of this information is stored safely in numerous computer databanks.

Even my telephone calls are carefully recorded, tabulated, and presented to me at the end of each month. I remember in college when my girlfriend broke up with me during a long-distance phone call. We talked for 20 minutes, then she hung up. I called her back again and again I got the answering machine each time. A few weeks later, the phone bill came in the mail, and there were the calls: one for 20 minutes, and then five calls in rapid succession, each one lasting just 15 seconds.

But by far he most detailed records of my life reside on my computer's hard drive: my stored email messages, going back to my freshman year in college. All told, there are more than 600 megabytes of information - roughly 315,000 pages of double-spaced text, or 40 pages of text for every day since September 3, 1983, when I got my first email account at MIT.

"Keep you old email messages," my friend Harold told me just before I graduated. "When historians look back at the 1980s, we are the ones they're going to be writing about." And he was right: with keyword searching and advanced text-processing algorithms, it will be a simple matter for some future historian to assemble a very accurate record of my life as a college student - and my life ever since - by examining the written electronic record I've left behind.

But this archive of facts and feelings is a rapier that can slice two different ways. More than my own digital diary, I have also been casting a vast "data shadow" that reveals the secrets of my daily life to anyone who can read it.

Alan Westin coined the term 'data shadow' in the 1960s. Westin, a professor at Columbia University in New York, warned that credit records, bank records, insurance records, and other information that made up America's emerging digital infrastructure could be combined to create a detail digital dossier. The metaphor, with its slightly sinister feeling, was uncannily accurate: just as few people are aware of where their shadows fall, few data subjects in the future, Westin conjectured, would be able to keep track of their digital dossiers.

In the three decades that have passed since then, the data shadow has grown from an academic conjecture to a concrete reality that affects us all.

We stand at the brink of an information crisis. Never before has so much information about so many people been collected in so many different places. Never before has so much information been made so easily available to so many institutions in so many different ways and for so many different purposes.

Unlike my email that's stored on my laptop, my data shadow is largely beyond my control. Scattered across the computers of a hundred different companies, my shadow stands to attention, shoulder-to-shoulder with an army of other data shadows inside the databanks of corporations and governments all over the world. These shadows are making routine the discovery of human secrets. They are forcing us to live up to a new standard of accountability. And because the information that makes up these shadows is occasionally incorrect, they leave us all vulnerable to punishment or retaliation for actions that we did not even commit.

The good news is that we can fight back against this wholesales invasion of personal privacy. We can fight to stop the capturing of everyday events. And where capture is inevitable, we can establish strong business practices and laws that guarantee the sanctity of our privacy - protection for our shadows to live by. We have done so before. All that's needed is for people to understand how this information is being recorded, and how to make that recording stop.

** All this week, Silicon.com is publishing exclusive extracts from Simson Garfinkel's latest book Database Nation, which poses a disturbing question: how can we protect our basic rights to privacy, identity and autonomy when technology is making invasion and control easier than ever before? The serialisation continues tomorrow, and on Friday, we've got 25 copies of the book up for grabs. See Silicon.com all week for further details.

Database Nation is available from all good bookshops, for more information see http://www.oreilly.com