How much data
As we start thinking of all this data, it’s easy to dismiss concerns about its retention and use based on the assumption that there’s simply too much of it to save, and in any caseit would be too hard to sift through for nuggets of meaningful information.
Some quick math. Your laptop probably has a 500-gigabyte hard drive. That big backup drive you might have purchased with it can probably store two or three terabytes. Your corporate network might have one thousand times that: a petabyte. There are names for bigger numbers.
A thousand petabytes is an exabyte (a billion bytes), a thousand exabytes is a zettabyte, and a thousand zettabytes is a yottabyte. To put it in human terms, an exabyte of data is 500 billion pages of text.All of our data exhaust adds up. By 2010, we as a species were creating more data per day than we did from the beginning of time until 2003. By 2015, 76 exabytes of data will travel across the Internet every year.
As we start thinking of all this data, it’s easy to dismiss concerns about its retention and use based on the assumption that there’s simply too much of it to save, and in any caseit would be too hard to sift through for nuggets of meaningful information. This used to betrue. In the early days of computing, most of this data—and certainly most of the metadata—was thrown away soon after it was created. Saving it took too much memory.
But the cost of all aspects of computing has continuously fallen over the years, and amounts ofdata that were impractical to store and process a decade ago are easy to deal with today. In2015, a petabyte of cloud storage will cost $100,000 per year, down 90% from $1 million in 2011.
The result is that more and more data is being stored.You could probably store every tweet ever sent on your home computer’s disk drive. Storing the voice conversation from every phone call made in the US requires less than300 petabytes, or $30 million, per year. A continuous video lifelogger would require 700gigabytes per year per person. Multiply that by the US population and you get 2 exabytes per year, at a current cost of $200 million. That’s expensive but plausible, and the pricewill only go down. In 2013, the NSA completed its massive Utah Data Center in Bluffdale. It’s currently the third largest in the world and the first of several that the NSA is building.
The details are classified, but experts believe it can store about 12 exabytes ofdata. It has cost $1.4 billion so far. Worldwide, Google has the capacity to store 15exabytes.What’s true for organizations is also true for individuals, and I’m a case study. My e-mail record stretches back to 1993.
I consider that e-mail archive to be part of my brain.It’s my memories. There isn’t a week that goes by that I don’t search that archive for something: a restaurant I visited some years ago, an article someone once told me about,the name of someone I met. I send myself reminder e-mails all the time; not just reminders of things to do when I get home, but reminders of things that I might want to recall years in the future. Access to that data trove is access to me.