The cost of clicks: The cloud as sociophysical problem

Several recent posts have dealt with the physical elements of connectivity, such as lost broadband at the home office, Internet cabling and how it gets installed, but this week's New York Times investigative report by James Glanz on the power demands, pollution and waste associated with data centers (a.k.a., 'the cloud' or 'big data'), is sobering reading.

For example, the power consumed by data centers is roughly 30 billion megawatts, or the equivalent of 30 nuclear power stations.

And, it is not just the shear amount of power consumed, it is the fact that most data centers are idling wastefully in order to avoid our rapid surge from users. As Ganz reports,

"Stupendous amounts of data are set in motion each day as, with an innocuous click or tap, people download movies on iTunes, check credit card balances through Visa’s Web site, send Yahoo e-mail with files attached, buy products on Amazon, post on Twitter or read newspapers online.

A yearlong examination by The New York Times has revealed that this foundation of the information industry is sharply at odds with its image of sleek efficiency and environmental friendliness.

Most data centers, by design, consume vast amounts of energy in an incongruously wasteful manner, interviews and documents show. Online companies typically run their facilities at maximum capacity around the clock, whatever the demand. As a result, data centers can waste 90 percent or more of the electricity they pull off the grid, The Times found."

The report insightfully critiques how the social drivers of Internet culture (including providers and users) play themselves out in real physical terms, i.e., power consumption, pollution and waste.  In short, the web-based industry has grown fearful of servers crashing or going off-line, while users have come to expect that every email and billions and billions of songs, movies, photos and other documents can all be stored and/or backed up on-line.  We think we no longer need big hard drives, because of the cloud, but the cloud itself is just a lot of big hard drives. (use quote)

More from the New York Times Report:

"Today, roughly a million gigabytes are processed and stored in a data center during the creation of a single 3-D animated movie, said Mr. Burton, now at EMC, a company focused on the management and storage of data.

Just one of the company’s clients, the New York Stock Exchange, produces up to 2,000 gigabytes of data per day that must be stored for years, he added.

EMC and the International Data Corporation together estimated that more than 1.8 trillion gigabytes of digital information were created globally last year.

“It is absolutely a race between our ability to create data and our ability to store and manage data,” Mr. Burton said. With no sense that data is physical or that storing it uses up space and energy, those consumers have developed the habit of sending huge data files back and forth, like videos and mass e-mails with photo attachments. Even the seemingly mundane actions like running an app to find an Italian restaurant in Manhattan or a taxi in Dallas requires servers to be turned on and ready to process the information instantaneously.

The complexity of a basic transaction is a mystery to most users: Sending a message with photographs to a neighbor could involve a trip through hundreds or thousands of miles of Internet conduits and multiple data centers before the e-mail arrives across the street.

“If you tell somebody they can’t access YouTube or download from Netflix, they’ll tell you it’s a God-given right,” said Bruce Taylor, vice president of the Uptime Institute, a professional organization for companies that use data centers.

To support all that digital activity, there are now more than three million data centers of widely varying sizes worldwide, according to figures from the International Data Corporation.

Nationwide, data centers used about 76 billion kilowatt-hours in 2010, or roughly 2 percent of all electricity used in the country that year, based on an analysis by Jonathan G. Koomey, a research fellow at Stanford University who has been studying data center energy use for more than a decade. DatacenterDynamics, a London-based firm, derived similar figures."

*This is not to say that companies like Apple, Facebook and Google aren't working hard to lower emissions and power.  For more on energy efficiency initiatives at data centers, see Katie Fehrenbacher's report at gigaom.