The cost of clicks: The cloud as sociophysical problem

Several recent posts have dealt with the physical elements of connectivity, such as lost broadband at the home office, Internet cabling and how it gets installed, but this week's New York Times investigative report by James Glanz on the power demands, pollution and waste associated with data centers (a.k.a., 'the cloud' or 'big data'), is sobering reading.

For example, the power consumed by data centers is roughly 30 billion megawatts, or the equivalent of 30 nuclear power stations.

And, it is not just the shear amount of power consumed, it is the fact that most data centers are idling wastefully in order to avoid our rapid surge from users. As Ganz reports,

"Stupendous amounts of data are set in motion each day as, with an innocuous click or tap, people download movies on iTunes, check credit card balances through Visa’s Web site, send Yahoo e-mail with files attached, buy products on Amazon, post on Twitter or read newspapers online.

A yearlong examination by The New York Times has revealed that this foundation of the information industry is sharply at odds with its image of sleek efficiency and environmental friendliness.

Most data centers, by design, consume vast amounts of energy in an incongruously wasteful manner, interviews and documents show. Online companies typically run their facilities at maximum capacity around the clock, whatever the demand. As a result, data centers can waste 90 percent or more of the electricity they pull off the grid, The Times found."

The report insightfully critiques how the social drivers of Internet culture (including providers and users) play themselves out in real physical terms, i.e., power consumption, pollution and waste.  In short, the web-based industry has grown fearful of servers crashing or going off-line, while users have come to expect that every email and billions and billions of songs, movies, photos and other documents can all be stored and/or backed up on-line.  We think we no longer need big hard drives, because of the cloud, but the cloud itself is just a lot of big hard drives. (use quote)

More from the New York Times Report:

"Today, roughly a million gigabytes are processed and stored in a data center during the creation of a single 3-D animated movie, said Mr. Burton, now at EMC, a company focused on the management and storage of data.

Just one of the company’s clients, the New York Stock Exchange, produces up to 2,000 gigabytes of data per day that must be stored for years, he added.

EMC and the International Data Corporation together estimated that more than 1.8 trillion gigabytes of digital information were created globally last year.

“It is absolutely a race between our ability to create data and our ability to store and manage data,” Mr. Burton said. With no sense that data is physical or that storing it uses up space and energy, those consumers have developed the habit of sending huge data files back and forth, like videos and mass e-mails with photo attachments. Even the seemingly mundane actions like running an app to find an Italian restaurant in Manhattan or a taxi in Dallas requires servers to be turned on and ready to process the information instantaneously.

The complexity of a basic transaction is a mystery to most users: Sending a message with photographs to a neighbor could involve a trip through hundreds or thousands of miles of Internet conduits and multiple data centers before the e-mail arrives across the street.

“If you tell somebody they can’t access YouTube or download from Netflix, they’ll tell you it’s a God-given right,” said Bruce Taylor, vice president of the Uptime Institute, a professional organization for companies that use data centers.

To support all that digital activity, there are now more than three million data centers of widely varying sizes worldwide, according to figures from the International Data Corporation.

Nationwide, data centers used about 76 billion kilowatt-hours in 2010, or roughly 2 percent of all electricity used in the country that year, based on an analysis by Jonathan G. Koomey, a research fellow at Stanford University who has been studying data center energy use for more than a decade. DatacenterDynamics, a London-based firm, derived similar figures."

*This is not to say that companies like Apple, Facebook and Google aren't working hard to lower emissions and power.  For more on energy efficiency initiatives at data centers, see Katie Fehrenbacher's report at gigaom.

Requisite Connectivity

Requisite connectivity is the state of having robust and reliable communication and/or transportation media/modes, with operable alternative work-around options, so that contact may be initiated or maintained at the rate, richness and intensity required for a given task or social outcome. Too little connectivity can obviously be a problem. Most of us have experienced the frustrations of working from home without technical support, travelling abroad where your cell phone doesn’t work, and so on. Indeed, for some, the difficulties of keeping connected are simply more trouble than they are worth. Most of us, however, struggle to stay connected, even though it is not always. Fortunately, like social networks, where thin networks can work as well as dense ones, ‘thin’ connective density (number of links) can be sufficient in many circumstances.

So, in terms of connectivity, more is not necessarily better. Indeed, we can think of connectivity and security as opposite ends of a continuum, whereby highly connective systems bring higher risk of hacking into technical systems and/or intrusions into our private lives. We might call this the weakness of strong ties. Rather than a single ideal level of connectivity, what most of us seek is not necessarily more and more connectivity, but requisite connectivity, that is enough connectivity for our intended purposes, but not so much that it undermines performance, invades our personal space and/or produces stress and anxiety. It involves the interplay of both technical and social systems, including human agency (choice, free will) to connect or disconnect.

Requisite connectivity is particularly important for organisations relying on cross-cultural communication for successful collaboration in a global economy. Ultimately, we still need to manage distance and isolation better. Moreover, with our history (until recently) of high uptake of information and communication technologies and our distant location, New Zealand could take the lead in understanding the potential and limitations of connective technologies.

Since 1999, I have been probing the literatures of globalization, the sociology of work, socio-technical systems, technology adoption, human-computer interaction, virtual teams, distributed work, and social networks. I have forged relationships with scholars at Boston College, Carnegie Mellon University, Duke University, University of California, Irvine, and the University of Washington, Seattle. See ‘Who’s Who in Understanding Distance’ on this site.

With the basic theoretical foundations in place, I have begun conducting empirical research, beginning with data on distributed leadership from the New Zealand Leadership Institute and data to come from the Global Executive MBA programme at Duke University. I have manuscripts under review in top journals and have also started a book on the subject of ‘Managing Distance and Isolation,’ which investigates how organizations compete from far away places, like New Zealand.