State of the USA and 1,000 Points of Data

The State of the USA has responded to the New York Times editorial calling for a national indicator program. That's a good thing, because I'm a big fan of the work they've been doing, and I'm getting pretty excited about their official launch, scheduled for some time this fall.

Chris Hoenig, President and CEO of The State of the USA, writes:

In response to a growing call for new ways to achieve shared understanding, increased transparency and improved accountability, a new dialogue is unfolding among Americans and in the media about how to measure the performance and progress of our society. The State of the USA’s mission is to help Americans better assess the progress of the country. We aim to provide key national indicators on the Web as a public service, bearing on major issues ranging from health and the economy, to education and the environment. In the next few months, we will begin by publishing data on key national health indicators, followed by a steady progression of information on other issues. This is a daunting task, which will take some time to accomplish. A working partnership between the public and private sectors will be central to accomplishing this mission.

As a private, non-profit organization dedicated to free information dissemination in the public interest, we rely in particular on working with the people and the products of the U.S. federal statistical system. Without them, we and thousands of other institutions like ours would be unable to make our contributions to better informing the American people. Our role as a private institution allows us to perform an important function that government cannot: combining in one place both public (i.e., official) statistics from federal sources as well as the best quality data from other organizations – commercial, state or international. It is also possible for us to be more selective in our choices of measures that are especially important for Americans to focus on. To do this in a responsible fashion, we rely on an editorial process with a variety of inputs – from polling research and public workshops run by America Speaks and Westat, to expert advice from institutions such as the National Academies. (The National Academies is the umbrella organization for the National Academy of Sciences, the National Academy of Engineering and the Institute of Medicine.)

Conversely, there are vital functions that public institutions, like the Federal government, have a unique ability to provide to help Americans assess the progress of the nation. These go beyond the tremendous service of managing the nation’s multi-billion dollar statistical system – the lifeblood of so many decisions in our democracy. From the moment of the State of the USA’s founding, we have encouraged the federal government to create a key national indicator system to ensure that Americans get even better access to valuable data on the changing nature of American society. (The State of the USA engages in limited advocacy on behalf of an independent key national indicator system, which is allowed under the Internal Revenue Service code and governed according to the respective policies of its funders.) Such a system has been recommended to Congress by the GAO, and recent media coverage has highlighted growing bipartisan support for the Key National Indicators Act – a bill to be introduced by Senators Edward M. Kennedy and Michael Enzi. This legislation would establish a bi-partisan commission to create such a system. The commission would then charge the prestigious National Academies with the task of creating and managing the system.

Ever since its founding, America’s democracy has grown and strengthened when the best of the public and private sectors are brought out in appropriately constructed partnerships. As we have also seen recently, it can weaken when these partnerships are neglected or poorly constructed. The historical lesson is not to avoid them or ignore them, but to design and manage them well. It is our greatest hope that, should the Key National Indicators Act be enacted into law, we and many other public and private organizations can make contributions to such a partnership in the interest of the American people and future generations.

Read more ...

NYTimes Calls For National Indicator Set

Have you seen the editorial today in the New York Times? Kenneth Duberstein writes about the State of the Union address:

But given today’s challenges and the rapid pace of change, a yearly formal address is no longer sufficient to measure the true state of our Union. To recapture the spirit of the founders — and to fulfill President Obama’s own promise to provide greater accountability in Washington — another tool is needed, one that enables all Americans to gauge whether we are making progress as a nation.

What we need now is a Web-based system for measuring our changing society with key national indicators — in a free, public, easy-to-use form.

That sounds like a pretty good idea, doesn't it? It's such a great idea that "Senators Edward M. Kennedy, Democrat of Massachusetts, and Michael Enzi, Republican of Wyoming, plan to soon introduce a bill that would allocate about $7.5 million a year for such a comprehensive database of key national indicators, and the idea already has wide bipartisan support. The data, selected by the National Academy of Sciences, would come from public and private sources of information on issues like education, the environment, the economy, energy use, housing, health care and the state of our roads and other public works."

Which would be fantastic -- except Congress already did this. Through the GAO and the National Academies of Science. And they spun it off, and it's now called the State of the USA project. And it's scheduled to be done and online by Fall 2009.

Here's more:

The State of the USA, Inc. (SUSA) is a new nonprofit organization that will assemble high-quality measures and data that can be used to assess the progress of the United States; it will display those measures—as a public service—on its website. SUSA will serve nongovernmental organizations, the media, policy makers, business leaders, foundations, scientists, educators and citizens by providing valuable information so all Americans can educate themselves about the progress of the United States.

With advice from the National Academies, SUSA is assembling a set of key national indicators to measure specific conditions or trends in this country. Data will be drawn from this country’s most respected public and private statistical sources (e.g., the Bureau of Labor Statistics, the Census Bureau and the Bureau of Economic Analysis) and will help the public track the nation’s progress on a wide variety of issues at many demographic and geographic levels, to the extent such data are available.

So what's up with the editorial? The legislation? Any ideas?

Read more ...

Comparables, Context, and Community Indicators

As I write this, I'm at cruising altitude looking down as the morning sun highlights land-use development patterns that appear remarkably standardized from the ones I saw yesterday in another airplane heading toward a completely different part of the country. (I'm spending much too much of my time in airplanes these days.) So when I look down from above the clouds and can't tell if I'm above Tallahassee or Thomasville, my mind naturally turns to ... community indicators.

More specifically, I start thinking about some comments yesterday about comparability and scalability of indicators, especially as it relates to the development of the State of the USA project. The Community Indicators Consortium was hosting a conversation between SUSA and community indicators practitioners, all held over at the Urban Institute offices in D.C. (special thanks to Tom Kingsley and Kathy Pettit of the National Neighborhood Indicators Partnership for providing meeting space and refreshments.) The discussion flowed easily, as Charlotte Kahn of the Boston Indicators project facilitated exploration into civic engagement strategies, technology and tools, and indicator development. I deeply appreciate the opportunity to participate in such a thoughtful discussion.

One challenge faced by communities as they design their indicator efforts is that of comparability. If the local unemployment rate is 5.5, as someone in one community asked me, is that good or bad? That's not an easy question to answer. I think we've talked about this before, but I'll recap the issues around comparability and context.

One option for communities to answer “is that good or bad?” is to develop a peer comparison set. This would be a set of roughly similar communities, at least in demographics or other key points, and compare one's own indicators to the results from those communities. This poses several challenges, including selecting which communities could act as reasonable peer comparisons, finding indicators that are measured the same way throughout the peer community set, and avoiding significant confounding factors that make the comparisons irrelevant or problematic. For example, if a peer community just faced the closing or transfer of a large employer, that spike in unemployment may not be a fair comparison. At the end of the day, however, using peer comparisons doesn't answer “is it good or bad?” It can only answer “is it better or worse than someone else?” The danger is the TGFM effect (Thank Goodness for Mississippi): there's always somebody doing worse, and so your indicator may be met with complacency, depending on the choice of comparables. In fact, things might be getting worse in your community, but as long as things are getting relatively more awful somewhere else, you can feel pretty good about doing poorly. Not an ideal recipe for change.

The other side is also true: real progress in the local community can be overshadowed by more progress somewhere else, turning celebrations into discouragement. The benefit of comparables, however, is to avoid local complacency in making incremental changes when the rest of the world is making rapid progress on an issue. Ken Jones, of the Green Mountain Institute, points out that making small steps toward reducing lead poisoning in children would be a travesty, since most communities have made huge leaps in lead reduction and childhood exposure. Only by examining the context in which these numbers are changing can you get an accurate read on what your progress means.

A second option for communities to answer “is this good or bad?” is to compare themselves against themselves – and against a community vision. This is the decision Jacksonville made with its indicators project 23 years ago. So the question of the unemployment rate becomes “is it better than it was last year?” and “how far away are we from our goal of full employment?” The advantage of this approach is that communities look inward, measuring themselves against their own standards of where they want to be, and can galvanize action or celebrate success accordingly. The disadvantage is that this approach can miss significant national confounding factors out of the control of the local community, or the community might not reach far enough or fast enough for improvement when the rest of the country is moving rapidly toward success.

A third option, and one that Jacksonville is moving toward, is that of using peer communities and national measures as context, not comparison. This means not asking “are we doing better than Nashville?” but instead “what does our trend line look like in the context of the shared movement of peer communities?” The question is not “where do we rank among these cities?” but instead asks what is happening across the country in relationship to the indicator being measured, and how can we understand local efforts and progress better.

This approach allows us to consider the national decrease in teen birth rates or violent crime and see if our local efforts at teen pregnancy prevention are making progress beyond that which would have been otherwise expected if we were just following national patterns. It allows us to see the local murder rate in context – although it is lower than it was a decade ago, something is happening in our community that makes it more violent than it should be if all else were equal. These are much harder questions to ask, and require more work to put together the information to provide useful context. But it may be getting easier, if a few new national project provide the tools for selecting peer comparables like I hope they will.

Imagine, if you will, being able to select your own comparable communities from an extensive national dataset, based on the sociodemographic variables you select. That would be useful, yes? Now imagine being able to select comparable communities based on a wider set of indicators than just socioeconomic status or demographics. Imagine putting together different peer sets depending on which variables/indicators are under consideration. Now imagine adding a time dimension to the equation, so that you could look at communities that five years ago were where your community is today on an issue, and then see which communities improved significantly. How much more useful is that to see who's actually solved the problem your community faces today? How exciting would it be to then examine their public policy approaches, their human service programs, or their community initiatives to discover what they did that made a difference, and then to replicate that success in your community?

This is, I think, one of the fascinating new possibilities that scalable, searchable, comparable national indicator data sets will be providing in the next few years. I think it will greatly enhance the capacity of local communities to understand their own indicators in a larger context and to identify successful practices for community change. And it is for community improvement that we measure all this stuff, isn't it? We want things to get better, and we know that they get better faster if we make decisions based on accurate information.

You may think my head's in the clouds. Right now, as I look out the airplane window, you would be right. But that won't keep these tools from being developed, and soon. More to come later ... right now, my tray table needs to be placed in an upright and locked position.

Signing off somewhere above a patchwork America.

Read more ...

Creating Community Indicators Reports

The last couple of weeks, I've been working with several communities in developing their first community indicators reports. After these reports move from draft stage to publication, I'll provide links so you can see what these communities are doing. I'm quite impressed with the depth of thinking and creativity that has gone into these projects, which clearly reflect the priorities and issues specific to their people and geography.

Seeing multiple projects come together at the same time was a welcome reminder of the importance of local input and local variability in community indicators. I applaud, as I'm sure you do, the work of the State of the USA project, which will be of tremendous benefit to local community indicators work, especially if the indicators reported are scalable local through national. The project will accelerate the use of data in decision-making and highlight the need for community indicators, as well as provide welcome data resources, spotlight the importance of tracking data for issues where data gaps exist, and decrease the upfront costs in developing local indicator sets.

The concern is, of course, that a national set of indicators may be seen as creating a homogenous template against which all communities can be compared, ranked, and tracked. The local projects I've been working with have shown once again the incredible variability and strength that comes through local communities who define for themselves what's important and how they want to measure it.

In talking with one community yesterday after reviewing their draft report, I was asked for suggestions on how to release the report to the community. I'll share with you my response, but I really would like your input as well. After a community develops a set of indicators, creates a framework for sharing them, and researches the trend lines and plugs in the data, what's an effective way to take that set of numbers and charts and graphs and present them in a way that informs, excites, and inspires a community to action?

My reply, in part, was: You have data. You have trends. You have a constellation of [a certain number of] indicators telling you something about [your community.] So what’s the story? What picture does this draw for your community? What has been the reaction of the steering committee to the numbers?

If a news reporter showed up with a camera and a microphone and asked [the community volunteer who chaired the indicator effort] what the report said about the region, what’s the 15-second sound bite?

(I see some really interesting stuff in the report, but the key story really should be drawn from the steering committee’s reaction to the data … and then from that reaction, the key phrase, quote, talking points, picture, and story can be drawn to make the data resonate in the community. [some examples here] There are several stories that jump out – what did the committee see as most important?)

[C]apturing the “a-ha” moments from the committee will be important in describing how the report is framed/translated for the community at large. The key questions to consider often are:
1. What does the data tell us?
2. Why should we care?
3. What should we do with that information? or How does that information affect decision-making in the region?

So what did I not tell them that they need to know? What's your advice to a community presenting their report for the first time -- how can they engage the community around the data in ways that make a difference?

Read more ...