Hurricanes and the limits of data (and technology)

We write a lot of stories that show how data is helping to solve big problems and improve people’s lives. But the New York City post-Sandy experience is also a testament to the limits of data.

Certainly, sharing data about where you can get food and gasoline, and optimizing traffic routes based on user-reported information can help the Sandy recovery effort run more smoothly. But there are many things in a post-distaster area that reside solely in the physical world and data analysis can’t replace infrastructure.

Gasoline shortages are one example. Gas stations are either running out of fuel or don’t have power to pump the fuel that they do have. Meanwhile, delivery of new fuel to places that need it isn’t going to happen if a debris-clogged port is blocking tankers, or roads are impassable.

The bigger issue here is that there is a mismatch between our online experiences and our expectations about how those experiences translate into the physical world. We’ve grown so accustomed to the relatively frictionless movement of bits that we forget that the physical distribution of goods can’t possibly keep up.

Sure, this may be a ‘no, duh’ kind of observation, but the expectations of many people in New York seem to be that with all of our technology and resources, the current aftermath is inexplicable. I’ve lived through hurricanes, and the aftermath of Sandy is exactly what happens when a major storm hits.

This mismatch between Internet expectations and the real world is found everywhere, not just in the wake of disasters. You see it in frustration during big nights in San Francisco when finding a cab or an Uber car is impossible. You see it in slow order fulfillment in online retailers whose viral launch success may have depleted their inventory.

Finding ways to bridge that divide between the real and digital worlds represent the next big opportunity in commerce, but expecting the real world to match the online world’s speeds and scalability is still a pipe dream.

I thought about this today when I got an email from the University of Illinois asking for people in New York to share traffic data via an app on a smartphone, for an experiment the university is doing on post-disaster traffic patterns. It’s a cool idea — think Root Metrics asking for data to measure cell phone network quality or Rick Smolan and his The Human Face of Big Data app.

But the reality of disasters is this: You lose power. Homes flood. Gas is in short supply. Sometimes water is undrinkable. Looting and violence occur. Technology can tell people if they should evacuate, help pre-stock supplies near the edge of the anticipated disaster zone, spread the word about relief efforts and help people connect after the disaster itself.

But it can’t replace fallen telephone poles and downed utility lines. It can’t get oil or food supplies to the center of the disaster zone quickly, and it can’t reconstruct physical infrastructure at online speeds.


GigaOM