Posts Tagged ‘transparency’

“You can’t do the math that way”

April 12, 2010

One of the best sources for news reporting about the stimulus is ProPublica. Its staff is particularly savvy about how to dig into the data and seems happy to share its knowledge with journalists elsewhere.

Jennifer LaFleur

ProPublica, which draws its funding from the Sandler Foundation and other contributors, is an independent, non-profit newsroom, created with the purpose of covering “truly important stories, stories with moral force.” It has aggressively covered the stimulus from its passage in February 2009. We recently talked with Jennifer LaFleur, who is director of computer assisted reporting there.

How did ProPublica get started covering the Recovery Act?

In the beginning, there was this huge amount of money and very little coverage. Initiated by our stimulus reporter Michael Grabell, we started pulling together as much data as we possibly could. Before the data was on Recovery.gov, we had to go to individual states to get it. In some states, if you’re not a resident, they won’t provide public records – even though this was federal money.

Early on, we spent a lot of time with the state websites and the state department of transportation sites. Initially, a lot of states looked like they were giving you a lot of information, but they really weren’t. Texas had a pretty page and lots of graphics and flashy things, but it didn’t provide a list of actual recipients. I remember that Washington was a state that did a really good job on its state DOT site.

With all the data on Recovery.gov, is the stimulus a lot easier to cover now?

It is easier, though there is a problem that not all the data is in one place. There’s Recovery.gov and USAspending.gov and there’s duplication in those sites. The information in the latter covers the parts of the stimulus that weren’t intended to create jobs – Section 8 housing money, Small Business Administration loans.

We’ve collected all the data in one place, so we have double the records in our data set that you’d find in Recovery.gov.

What do you think of the Recovery.gov website?

For the speed with which they had to set this up, Recovery.gov is one of the better sites as far as providing information. But it was a very big promise to say we’d be able to follow every dollar. That’s not the case.

In the beginning, there were definitely data quality issues.  There were zip codes that didn’t exist and data entry errors on Congressional districts. They’ve done a pretty good job of fixing the data issues.

Are there improvements you’d make in Recovery.gov?

It’s been responsive and fixed a lot of the big problems that came up early on.

But there is information you’d like to see. They don’t include counties in the data and that’s kind of a standard thing. They have other location information, but people want to look at what money is going to their county. There are some identifier issues I’d fix.  You can connect a project to a prime recipient, who doles out the money to sub-recipients, but you can’t really hook up vendors to sub-recipients.

For example, a reporter called yesterday.  He saw that a sub-vendor in his area was a car dealership and  wanted to know who had hired that car dealer. But the data doesn’t show that.

There’s also no information on anything below sub-recipient. In weatherization, for example, the money goes from the Department of Energy to states and states disperse the money to community action agencies and then they hire contractors to do the work.  The data doesn’t show those relationships.

I know that at some point you have to draw the line on how many levels you go down.

What do you think of the quality of local reporting on the stimulus?

It’s also gotten a lot better. The questions we get from reporters are more sophisticated than they used to be, but understanding the data is still complicated.

I’ve noticed that a few organizations have made the mistake of taking the total project money, dividing by the number of jobs and then writing a story about the $500,000 job. You can’t do the math that way. A lot of the money goes to other things – like buying asphalt or trucks.

Also, when I do training sessions for reporters, I ask how many people have read the recovery bill and nobody raises their hands.

Do you have any other issues with what you read?

There’s a tendency to jump on a problem as if it were a big conspiracy. Early on, there was a perception that a bad zip code was a signal that there was a big conspiracy of money being dumped in bad places. But it was just bad data.

What aspects of the ProPublica coverage are you particularly proud of?

Michael Grabell has done a great deal of very good work. He dug in very quickly. In one example, he found that stimulus money was going to teeny tiny airports in the middle of nowhere and then the DOT  inspector general came out with an advisory that said the same thing.

We’ve put a lot of work on some of the tools on our website. Our Recovery Tracker lets people track down to the county level and Stimulus Speed Chart looks at how fast money goes out the door.

What are your plans for future coverage?

We’ll continue to improve our Recovery Tracker and we likely will develop new interactive tools for covering the stimulus. We’ll also continue to dive into the data to do investigations.

Note:  A late afternoon congratulations to ProPublica, which became the first online non-profit news organization to win a Pulitzer. The award, announced today, was in the investigative journalism category and was shared with The New York Times for an article published in its Sunday magazine last August. The article, by Sheri Fink, dealt with choices faced by doctors at a New Orleans hospital immediately following Hurricane Katrina.

Advertisements

The transparency balancing act

April 9, 2010

As we talk to people about the reporting requirements for the stimulus act, we hear two significant complaints:

1) Important information is missing
2) The task of supplying the information already there is onerous.

It’s a tricky balance, to be sure.  In our search for thoughtful commentary on questions of transparency, we’ve been entertained and educated by  the The Fine Print, a terrific blog  associated with OMBWatch.com. It has been doing a great job this week of staying on top of the Obama Administration’s Open Government Directive.

Consider the following comment from the federal Office of Management and Budget that The Fine Print highlights:

“A clear lesson from the Federal government’s experience with the Recovery.gov website is that, given the numerous stakeholders involved in the federal spending process and the complexity of underlying systems, all efforts to improve transparency must include thoughtful consideration of the costs and benefits of various implementation approaches.”

And while you’re visiting The blog, be sure to check out its breakdown of Open Government plans and memos and thoughts about some of the limits within those plans.

Trying to meet “a gold standard”

April 5, 2010

We recently interviewed Chris Patton, Recovery Act director in Wisconsin, to find out more about the improvements to the Wisconsin stimulus website.

Stimulus-supported school construction bonds in Wisconsin

Q. Wisconsin did very well on the Good Jobs First evaluation of state Recovery Act websites. What motivated you to make more improvements?

CP: We wanted to have as robust a tool as we could to explain to the public where the Recovery Act money was going and the impact it had on our economy. It’s a monumental effort. We’re really emphasizing that this has to be presented in a way that is easily understood by the average citizen.

Q. Will this experience have an impact on how the state reports on other programs?

CP:  I think the emphasis on customer service, the emphasis on listening to the public will really pay dividends going forward.

Historically, this kind of information has been available, but not in a user friendly manner.

Q. What have you learned in doing this?

CP:  We really tried to respond to what the public wants and to respond quickly. People are asking for information in a real time manner and typically, things can move slowly in state government.

We needed to learn how to be much more responsive in real time with the information we’re making available.

Initially, the federal Recovery website and ours was going to be updated quarterly. We realized and the public realized that there’s a lot going on and that they want to know what’s happening up-to-the-minute. Our website is now refreshed on a much more frequent basis, sometimes daily.

The public also had a much greater desire for details as to where the money is going, not only who is receiving it, but who the vendors and contractors are. We retooled the website to have that level of detail. We go down to basically any vendor payment that goes out the door. The federal payment standard is above $25,000, but we track spending down to a $10 hammer at Home Depot.

Q. What would you point out on your website as the key improvements?

CP: We give people the ability to download data – so you can take the information we present and get at the raw information and sort and compare. We had requests that made us consider different ways to access the information – for example, through local government identifiers or commercial districts.

There’s also an expanded search. You can input your zip code or look at it graphically on the map and you can zoom right in and see where recipients are, right down to your own street.

There’s a public policy side to having the level of detail that we do. With maps that overlay the unemployment rates in various communities, you can start to see where you are having impact and where you need to redouble effort. You can look at the per capita distribution.

Q. Wisconsin’s website seems to move faster than others. Do you know why?

CP: We charged our staff with meeting a gold standard. We have a set of products and tools that give us cutting edge technology on the back end of our website.

Q. Very few states have included information on where the bond money is going. Why do you feel this is important?

CP: The public doesn’t always understand that there’s a whole part of the Recovery Act that includes the bond provisions and tax credits. These are very important to stimulating our economy. We took it upon ourselves to make that information available to the public. This is a portion of the Recovery Act that doesn’t get a lot of attention.

With the bond program, we noticed that certain communities had expended their bonds and desired additional bonding allocations, but others hadn’t. We worked with the legislature to pool the unused bonds and target the communities that had the greatest need with products that were ready to go.

The geographic element, combined with the different data elements from the Commerce Department, was very helpful.

Q. What would you like to improve in the future?

CP: So much attention has gone to tracking the money, but there are a lot of other performance measures that the programs are undertaking. For example, the number of meals on wheels served. We hope to have additional details – not just what jobs are being funded, but truly understanding how programs are performing and the other benefits of the Recovery Act funding.

Q. What are the biggest challenges?

CP: Bringing the data together centrally is the challenge.

As we ramp up our performance monitoring efforts and collect other data elements and performance measures, we want to show unique program details. Are we meeting program goals in training dislocated workers? Are they not only being trained, but becoming employed?

The biggest barrier is the overall complexity of all the programs and the volume of data.

“Are you spending more time to report than to deal with programmatic responsibilities?”

April 1, 2010

We like transparency. We like statistics. We like up-to-date data.

But even we feel pity for stimulus grantees faced with a monsoon of reporting requirements. Beginning with March 30th reports, the federal Department of Energy has started to require monthly updates for weatherization assistance grants, energy efficiency grants, and state energy program grants. So many state officials are concerned about this that the Council of State Governments, the National Conference of State Legislatures, and the National Governors Association sent a joint letter to the Department of Energy in order to register their frustration with the proposed change.

Chris Whatley, who heads the Council on State Governments’ Washington office, is concerned that the additional reporting burden could stymie the growth of green jobs – an area already lagging thanks to labor and procurement guidelines and other factors. A recent CSG report found that barely more than 2% of all ARRA jobs in the first quarter of reporting had been green jobs. “They have been slow and have been criticized and now you are going to triple their reporting requirements,.” Whatley told us. Perhaps more than triple; from what we understand, these monthly reports are in addition to the quarterly reports–which means some redundant reporting every third month.

Evan Curtis, who works in the Utah Governor’s Office of Planning and Budget, notes that, for the most part, the people doing the work in areas like weatherization are also the people doing the reporting. That means, he says, “For every hour you take away for these reports you are literally taking away an hour that could have been spent weatherizing.”

Matt Fritz, an ARRA coordinator in Connecticut, understands the DOE’s motivation, “There is so much criticism of these major numbers and they want to offset that by saying ‘these are the things that are really happening, here’s what the money is really used for.’” But Fritz is nonetheless anxious that the reporting is beginning to crowd out the doing: “You end up spending a week every month to gear up to report. Are you spending more time to report than to deal with programmatic responsibilities?”

While it’s the DOE that has the states jittery now, there’s also concern about other agencies making the same requests. “Once the DOE does it,” Curtis asks, “what is to stop the other agencies from doing monthly too?”

Wisconsin: Better and better

March 31, 2010

Back when we started to evaluate the management of state and local governments about twenty years ago, we noticed a curious phenomenon. Governments that were weak in this field, tended to be relatively satisfied with what they were doing. By contrast, ambitious governments were often the most aggressive at figuring out how to improve still further. We thought of this phenomenon the other day when we saw that Wisconsin had just announced improvements to its stimulus website.

In the Good Jobs First evaluation of websites that we wrote about in early March, Wisconsin was ranked fourth (tied with Minnesota, and Colorado, and just behind Maryland, Kentucky and Connecticut). We turned to our friends at Good Jobs First to see which of the new Wisconsin features they found most worthy of notice.

Here’s what research analyst Thomas Cafcas says:

One great new feature  is a search box that combs the recovery act data for the state:

They also posted some distress metrics including county foreclosure rates:

Another feature we like is the mapping of recovery act bond allocations:

A big improvement is the disclosure of wages related to recovery act jobs across all sectors and a distribution of disadvantaged contracts.  (Cafcas believes the labeling could be clearer, though, and he said he’d also like to see more demographic information relating to workers who benefited.)

Coming up soon: A short interview we did with Wisconsin’s Recovery Act Director Chris Patton

California’s take on Recovery.gov and the State’s New Stimulus Website

March 23, 2010

Shortly after we posted our last  blog entry (“Before the Phone Starts Ringing. . .) we heard from Eric W Alborg the  Communications Director for the California Recovery Task Force.

He had some very germane thoughts about the federal government’s  Recovery.gov website — and improvements he’d like to see made. We thought we’d share them with you here:

“There are several improvements that could be made to Recovery.gov that the California Recovery Task Force feels would be of great benefit to Californians in our effort to provide the utmost transparency,” he told us. “California would like to be able to download or obtain all of the inferred data, which includes everything in the posted data dictionary.  We’d also like all of the Geocoded information, so that we can ensure we map projects and data correctly and have the ability to cross check.  Lastly, we’d like to have a direct contact at Recovery.gov to address any errors or problems with California’s data that we find.”

Any thoughts on this topic from other states? We’d love to hear them.

Meanwhile, kudos to California for the redesign of its own Recovery Act website, which went live today.  Included in the new site are:  success stories that highlight the human impact of Recovery Act dollars in California; a resource that outlines special opportunities for small business, non-profits and disadvantaged business enterprises; and, most notably, a map that features ways to sort by county, city or area of investment such as education or transportation. It also features locations of every Recovery Act project in the state with project-specific information such as budget, recipient and project description.

“Before the phone starts ringing. . . .”

March 23, 2010

Anyone  interested in stimulus spending quickly grows familiar with the federal government’s Recovery.gov website. It’s a comprehensive spot for citizens to get information on what’s happening in individual states. Curiously, it also turns out that Recovery.gov is the go-to place for state officials to find out what’s happening in their own backyards.

You might think they’d have their own, independent, sources of information. In fact, many states have no firm handle on what is going on with money that flows directly from the federal government to local governments or school districts within the state. When local governments direct their Recovery Act reporting straight to the federal government, that information is neither sent to the states nor are state officials allowed “early access” to Recovery.gov.  For states that are decentralized, this one-way information flow can also cause a headache if they want to get the information reported to the feds by their own state agencies.

“States ability to articulate what is happening on the federal website is lacking. What we’re struggling with as states,” says Michelle Weber, Minnesota’s ARRA coordinator, “is to look at Recovery.gov and validate that information.” Worse yet, reports Weber, “It is sometimes hard to understand where they got the information that is on Recovery.gov.”

Don Winstead, Florida’s recovery czar, says that with early access to the statewide numbers, he could have spotted some obvious errors in previous reporting periods, such as outcomes reported for congressional districts that don’t actually exist in the real world.

As it currently stands, Winstead has to wait with everyone else until the information is published. “I wish they’d have a way to give the governor’s designee broader rights to go into the system and get aggregate information, so we could have a process in place to analyze the data before the phone starts ringing.”

INTERVIEW WITH: Beth Blauer, director of Maryland’s StateStat

March 16, 2010

Blauer: "The Governor lives and breathes StateStat"

Beth Blauer is director of the Maryland Executive Department’s StateStat office. In that role, she oversees Maryland’s much-praised efforts to manage the state through widely disseminated performance measures. This was modeled, to some extent, on ground-breaking efforts in Baltimore, when  Governor Martin O’Malley was mayor there.

As in Washington and a few other states, Maryland has connected its tracking of stimulus dollars to other performance reporting mechanisms.  In a move that makes good sense to us, it launched its stimulus website on the back of the work it had already done with StateStat – and Blauer was put in charge. She also became, in her words, the “de facto stimulus czar.”  Largely as a result of the work done in the past, the online stimulus material was ranked as best in the country in Good Jobs First’s evaluation of state stimulus websites.

We thought it might be interesting to hear what Blauer had to say about  this work, and she was kind enough to spend a chunk of time chatting. Following some excerpts from our conversation.

Q. Maryland’s stimulus tracking was top of the pack in the Good Jobs First evaluation.  What separates Maryland from other states, in your view?

BB: A lot of states are struggling to bridge the gap between accountability with the financial data and the need for accountability and transparency on a management level. We had this highly scrutinized accountability and performance measurement program in place already.  So, we were able to immediately gear up.

Q. In what ways have you linked the spending of stimulus dollars in Maryland to program results?

BB: As much as possible, we’ve connected the tracking of the stimulus dollars with the performance information we have through StateStat. I’m director of StateStat and the de facto stimulus czar. It’s useful to have the same person have both these responsibilities because they’re both concerned with performance and reporting. We want to use this information so we can make the right choices of where to spend the money and we want to use the data on a management level.

The Governor lives and breathes StateStat and he’s looking at the map that shows stimulus activity all the time. I think that’s the most important message – if you don’t have your leaders buying in to it and heavily relying on the tools, you’re not going to have a tool that’s fully used.

The Governor was very interested in the creation of the map. He was sitting next to me and involved down to the choices of the icons. He wanted the map to show more than just the distribution of dollars, he wanted it tied to performance measurement. That’s the principle. He gets it.

Q. States are required to track the jobs that are being funded with the stimulus dollars. What are some of the other ways you’re tracking the use of the stimulus dollars on your website?

BB: We’ve been measuring how quickly our contracts are going out to bid and we’re looking at the percentage of minority business enterprises that are getting our Recovery Act contracts. We’re not relenting on that. We have a goal of 25 percent.

On weatherization, we wanted to connect the weatherization program with the state’s energy assistance program. We’re making it a goal to prioritize the people who get weatherization money, by looking for those who are also receiving cash assistance from the state to help pay energy bills. That way the state’s costs will go down when individual utility costs drop. So, we’ll track who gets the service and we’ll track the impact that has on energy bills. I’m working out how to depict this on the map without having to go down to the individual house level.

We’re also tracking the number of people who are going through weatherization training, so they can work in those jobs.

Q.  A lot of states are now using the same map that you’re using. How did that happen?

BB: We worked with ESRI, a company that specializes in geographic information systems, to create the tools under the condition that they’d share it with other states. We worked very hard to develop it. Now there are over 20 other states with the exact same map

But that doesn’t mean they’ve necessarily linked to performance. Washington State has some performance linkages and Massachusetts is working on it. New York City is doing a great job.

Q. Are there areas in which you think Maryland has gone beyond other states?

BB: We’re the only state right now that’s extensively using needs data. For example, we had to make decisions about where transit money was going.  If you look at our map under transportation, you can see we’ve color coded the areas in the state where there are high percentages of individuals who don’t have motor vehicles. Those are the spots that may have more need of transit dollars. We knew that Baltimore would be one of those areas, but the map also shows us that there was a justification for Garrett County, in the western part of the state, getting some new buses.

Q. What do you have planned going forward?

BB: For every single funding area, I have a wish list of performance metrics. I’d like to continue to draw connections, as we did with weatherization and energy assistance. I’d like to think about how we can speed up some of the milestones and goals that we’ve had. For example, with an infusion of dollars going to water projects, can we create a faster-paced change in the health of the Chesapeake Bay?

Q. What’s the biggest challenge for you with tying Recovery Act dollars to performance?

BB: This is hard, because it’s a fast-paced program and the majority of the dollars that are coming to the state are for programs that are very well developed, like the increase in the Medicaid match.

The biggest struggle I have in mapping the recovery data is trying to isolate the direct impact that the federal investment has had.

Q. Do you have any disappointments with what you’ve done so far?

A. I’ve been surprised that there hasn’t been more public engagement. Every place on our map, we have a way that a user can directly communicate with us. I was looking forward to that public engagement as part of the transparency. But we’re not getting the response we were expecting. That’s an area in which we can really strengthen our program.

Ranking the websites — additional thoughts

March 10, 2010

Though we found the website rankings done by Good Jobs First really interesting, it didn’t surprise us that some state officials think the effort missed the mark.

Gerry Oligmueller, the astute and thoughtful state budget administrator of Nebraska, for example, questioned whether states should be supplying much of the same information already available on the federal Recovery.gov site. “We aren’t trying to emulate or duplicate recovery.gov,” he told us. “We don’t feel that is in the interest of the U.S. or Nebraska taxpayer. If people want to track it down to the GIS dot, then recovery.gov provides that feature.”

To be clear, Nebraska’s site – recovery.nebraska.gov — does provide some state-level ARRA expenditure information, but it primarily serves as a conduit for resource and contact information—what funds are available and who to call about them. “We knew the web allowed us to eliminate a thousand phone calls a week. People were asking, ‘What is the money available for and where do I go to find out information for it,’” Oligmueller explains, “and a lot of that was driven by the fact that the federal government didn’t have all that information out there.”

We asked Greg LeRoy and Phil Mattera of Good Jobs First what they thought about this. Their response was, essentially, that state websites should be supplying even more details than appear on the federal website on how they are using the money.

LeRoy and Mattera strike an “if-not-now-when” note, saying that all levels of government are facing rising expectations for transparency and performance reporting. “The recovery act websites are fueling that expectation but it was already underway,” says LeRoy, who is the executive director of Good Jobs First. “We view this is as a two-year crash course in state disclosure,” adding that Good Jobs First is also ramping up to revisit its 2007 report “The State of State Disclosure.”

Other state officials we’ve talked to note that Good Jobs First gives credit to states that had pre-existing performance reporting websites and punishes those that didn’t. While we understand the logic there, it feels to us like however a state develops a superior site, it deserves to get credit for it.

Ranking the stimulus websites

March 9, 2010

All fifty states have set up websites that report on the way they’ve implemented the stimulus bill. Anyone who visits more than a handful of these sites will be immediately struck at the remarkable variety in quality. Some have clearly taken a great deal of thought, time and effort. Others seem like afterthoughts.

After a first stab at evaluating websites last July, Good Jobs First, tackled the task again in January. The organization is devoted to promoting government and corporate accountability in economic development. The broad conclusion of its new website evaluation: “A growing number of state ARRA sites deliver on President Obama’s promise that the stimulus plan would be carried out with an ‘unprecedented level of transparency and accountability.’ Yet some remain half-hearted efforts that provide taxpayers little useful data on the largest federal stimulus since the new deal.”

Some states may well have improved their sites since the release of this report. We, personally, know of a handful that are in the process of doing so now. But we think that the report’s interesting findings are still valid. Some of the points that jumped out at us:

  • “More than half the states (28) now have some kind of project mapping feature on their ARRA site. Of these, 14 have interactive maps with significant project details, while 13 have such maps with more limited details.”
  • “Only three states – – Kentucky, Maryland and Wisconsin – juxtapose the geographic distribution of spending with patterns of economic distress or need within the state.”
  • “Despite the ready availability of Recovery.gov ARRA employment data, 10 states have no jobs data on their websites: Hawaii, Kansas, Louisiana, Mississippi, Missouri, New York, North Carolina, North Dakota, South Carolina and the District of Columbia.” (Note: the report counts Washington D.C. as a 51st state)
  • Only five states – Connecticut Kentucky, Massachusetts, Mississippi and New Hampshire provide the full texts of at least some ARRA contract awards.”

As of the report’s publication, the half dozen top rated states were: Maryland, Kentucky, Connecticut, Colorado, Minnesota and Wisconsin. The bottom six were North Dakota, Washington D.C., Missouri, Alaska, Vermont and Louisiana.

Stay tuned. In coming days we’ll be sharing an interesting conversation with Beth Blauer, the director of the Maryland StateStat program. She oversees the website that was rated number one by Good Jobs First.