Mining Data for Meaning

More and more people are using the tremendous explosion of available data and cheap-but-powerful computing capabilities to do amazing things with data.

This May 20 New York Times article highlights some of the ways people are using data, including a very interesting way of combining a set of neighborhood measures to fight crime:

The programs cull through information that the department already collects, like “911” and police reports, but add new streams of data — about neighborhood demographics and payday schedules, for example, or about weather, traffic patterns and sports events — to try to predict where crimes might occur.

...

The technology, for example, pointed to a high rate of robberies on paydays in Hispanic neighborhoods, where fewer people use banks and where customers leaving check-cashing stores were easy targets for robbers. Elsewhere, there were clusters of random-gunfire incidents at certain times of night. So extra police were deployed in those areas when crimes were predicted.

The crime rate in Richmond declined about 20 percent last year, and it is down again this year. The Richmond experience is part of a wave of sophisticated computing and mathematical analytics that is moving into the mainstream.

The opportunities afforded by wider uses of data-driven decision-making cut two ways, however. For community indicators efforts to build on these opportunities, we're going to have to get more geographically specific (taking data down to the neighborhood and sub-neighborhood level, where appropriate and available.) We're going to have to be more timely with the data, so that the information is as current as possible. We're also going to have to be more specific with what we measure, so that the information can move decision-making rather than just point out large-scale trends. And we're going to have to market and share that information in a more compelling fashion to help make change possible.

This means getting the right information to the right people at the right time for the right places -- not an insurmountable challenge, but one that suggests we take a careful look at our processes and procedures to ensure they fit a changing reality.

We've been pushing for more data-driven decision-making. If we get what we ask for, are we ready to meet the data demands of the decision-making processes?