Pick a Number

Factcheck.org, sponsored by the Annenberg Public Policy Center of the University of Pennsylvania,  is designed to reduce confusion and deception in U.S. politics. We’ve been following its commentary for some time, and have been generally impressed. As you’d expect, it has included a fair amount of coverage of the Recovery Act — often  calling into question exaggerated statements from all sides.

A couple of weeks ago, we featured a post about the latest job impact report by the President’s Council of Economic Advisers. We just came across Factcheck.org’s  look at that report, which questions the Council’s statement that “a wide range of private and government analysts concur with our estimates” of 2.2 million to 2.8 million in job impacts.”

In its assessment, Factcheck.org points out that only one of the five other estimates cited by the report falls into the range of jobs cited by the Council’s economists. It notes that of the private economists cited, all were “well below the range estimated by the White House. “ For example, “Mark Zandi of Moody’s Economy.com put the number at 1,896,000, HIS/Global Insight put it at 1,707,000 and Macroeconomic Advisers estimated 1,462,000. While the nonpartisan Congressional Budget Office came nearly up to the CEA estimate of job impact on the high side (2.7 million) its low side estimate was 1.2 million – about a million short of the Council’s lower estimate.

Accurate or not?

An audit that came out of the Louisiana Legislative Auditor’s Office several months ago frets that the federal government’s job reporting system “provides little assurance that the information state agencies report to the federal government is accurate. While there are criteria that relate to the specific reporting requirements of state agencies, there are no criteria on how states as single entities should manage the overall reporting of ARRA funds.”

Louisiana, itself, is an interesting case. For one thing, it is one of five states, along with Alaska, Arkansas, Maryland, and Texas, that haven’t established an official czar. In addition, Louisiana has opted for a decentralized stimulus data reporting system. In other words, in Louisiana, agencies are self-verifying their numbers and it isn’t clear to the auditors how and by whom the accuracy of those numbers is being checked at the end of the process. “Louisiana,” the audit reads, “is primarily depending on guidance provided by the federal government to oversee the ARRA program and ensure the information state agencies report is accurate.”

Of course, that doesn’t mean that the process is or isn’t working, only that the auditor doesn’t seem able to be certain. From the auditor’s perspective, policies are being followed, and yet, as Nicole Edmonson, Performance Audit Manager, says, “In our report we just pointed out that it is kind of hard to say who is responsible for the funds. As the legislative auditor we’re not going to say how things should be done—we’re only going to say how things are being done. Is there going to be enough accountability in there? I would have to say I hope so.”

A Stark Difference

This week, the Council of Economic Advisers released its third quarterly report (for Q1 2010) on the impact of the federal stimulus with respect to jobs created or saved. The headline numbers: ARRA stimulated between 2.2 and 2.8 million jobs.

NPR’s blog “The Two-Way” pointed out that adding those jobs to the March unemployment figures suggests that there could have been 5 million more people out of work in March 2010 than in March 2009, if not for the stimulus.

The Council is careful to note the challenge of accurately estimating the effects of a large policy like ARRA. In the Daily Brief on Portfolio.com, Sarah Krouse writes, “The report said the stark difference between the White House calculations and recipient-reported data — which estimated a total number of 1.2 million jobs created or saved through the fourth quarter of 2009 — is the result of several factors, including the fact that reporting requirements only apply to about a third of ARRA funding and the direct spending parts of the stimulus requiring reports are moving slower than other portions of the stimulus.”

Trying to meet “a gold standard”

We recently interviewed Chris Patton, Recovery Act director in Wisconsin, to find out more about the improvements to the Wisconsin stimulus website.

Stimulus-supported school construction bonds in Wisconsin

Q. Wisconsin did very well on the Good Jobs First evaluation of state Recovery Act websites. What motivated you to make more improvements?

CP: We wanted to have as robust a tool as we could to explain to the public where the Recovery Act money was going and the impact it had on our economy. It’s a monumental effort. We’re really emphasizing that this has to be presented in a way that is easily understood by the average citizen.

Q. Will this experience have an impact on how the state reports on other programs?

CP:  I think the emphasis on customer service, the emphasis on listening to the public will really pay dividends going forward.

Historically, this kind of information has been available, but not in a user friendly manner.

Q. What have you learned in doing this?

CP:  We really tried to respond to what the public wants and to respond quickly. People are asking for information in a real time manner and typically, things can move slowly in state government.

We needed to learn how to be much more responsive in real time with the information we’re making available.

Initially, the federal Recovery website and ours was going to be updated quarterly. We realized and the public realized that there’s a lot going on and that they want to know what’s happening up-to-the-minute. Our website is now refreshed on a much more frequent basis, sometimes daily.

The public also had a much greater desire for details as to where the money is going, not only who is receiving it, but who the vendors and contractors are. We retooled the website to have that level of detail. We go down to basically any vendor payment that goes out the door. The federal payment standard is above $25,000, but we track spending down to a $10 hammer at Home Depot.

Q. What would you point out on your website as the key improvements?

CP: We give people the ability to download data – so you can take the information we present and get at the raw information and sort and compare. We had requests that made us consider different ways to access the information – for example, through local government identifiers or commercial districts.

There’s also an expanded search. You can input your zip code or look at it graphically on the map and you can zoom right in and see where recipients are, right down to your own street.

There’s a public policy side to having the level of detail that we do. With maps that overlay the unemployment rates in various communities, you can start to see where you are having impact and where you need to redouble effort. You can look at the per capita distribution.

Q. Wisconsin’s website seems to move faster than others. Do you know why?

CP: We charged our staff with meeting a gold standard. We have a set of products and tools that give us cutting edge technology on the back end of our website.

Q. Very few states have included information on where the bond money is going. Why do you feel this is important?

CP: The public doesn’t always understand that there’s a whole part of the Recovery Act that includes the bond provisions and tax credits. These are very important to stimulating our economy. We took it upon ourselves to make that information available to the public. This is a portion of the Recovery Act that doesn’t get a lot of attention.

With the bond program, we noticed that certain communities had expended their bonds and desired additional bonding allocations, but others hadn’t. We worked with the legislature to pool the unused bonds and target the communities that had the greatest need with products that were ready to go.

The geographic element, combined with the different data elements from the Commerce Department, was very helpful.

Q. What would you like to improve in the future?

CP: So much attention has gone to tracking the money, but there are a lot of other performance measures that the programs are undertaking. For example, the number of meals on wheels served. We hope to have additional details – not just what jobs are being funded, but truly understanding how programs are performing and the other benefits of the Recovery Act funding.

Q. What are the biggest challenges?

CP: Bringing the data together centrally is the challenge.

As we ramp up our performance monitoring efforts and collect other data elements and performance measures, we want to show unique program details. Are we meeting program goals in training dislocated workers? Are they not only being trained, but becoming employed?

The biggest barrier is the overall complexity of all the programs and the volume of data.

“Are you spending more time to report than to deal with programmatic responsibilities?”

We like transparency. We like statistics. We like up-to-date data.

But even we feel pity for stimulus grantees faced with a monsoon of reporting requirements. Beginning with March 30^th reports, the federal Department of Energy has started to require monthly updates for weatherization assistance grants, energy efficiency grants, and state energy program grants. So many state officials are concerned about this that the Council of State Governments, the National Conference of State Legislatures, and the National Governors Association sent a joint letter to the Department of Energy in order to register their frustration with the proposed change.

Chris Whatley, who heads the Council on State Governments’ Washington office, is concerned that the additional reporting burden could stymie the growth of green jobs – an area already lagging thanks to labor and procurement guidelines and other factors. A recent CSG report found that barely more than 2% of all ARRA jobs in the first quarter of reporting had been green jobs. “They have been slow and have been criticized and now you are going to triple their reporting requirements,.” Whatley told us. Perhaps more than triple; from what we understand, these monthly reports are in addition to the quarterly reports–which means some redundant reporting every third month.

Evan Curtis, who works in the Utah Governor’s Office of Planning and Budget, notes that, for the most part, the people doing the work in areas like weatherization are also the people doing the reporting. That means, he says, “For every hour you take away for these reports you are literally taking away an hour that could have been spent weatherizing.”

Matt Fritz, an ARRA coordinator in Connecticut, understands the DOE’s motivation, “There is so much criticism of these major numbers and they want to offset that by saying ‘these are the things that are really happening, here’s what the money is really used for.’” But Fritz is nonetheless anxious that the reporting is beginning to crowd out the doing: “You end up spending a week every month to gear up to report. Are you spending more time to report than to deal with programmatic responsibilities?”

While it’s the DOE that has the states jittery now, there’s also concern about other agencies making the same requests. “Once the DOE does it,” Curtis asks, “what is to stop the other agencies from doing monthly too?”

Ranking the websites — additional thoughts

Though we found the website rankings done by Good Jobs First really interesting, it didn’t surprise us that some state officials think the effort missed the mark.

Gerry Oligmueller, the astute and thoughtful state budget administrator of Nebraska, for example, questioned whether states should be supplying much of the same information already available on the federal Recovery.gov site. “We aren’t trying to emulate or duplicate recovery.gov,” he told us. “We don’t feel that is in the interest of the U.S. or Nebraska taxpayer. If people want to track it down to the GIS dot, then recovery.gov provides that feature.”

To be clear, Nebraska’s site – recovery.nebraska.gov — does provide some state-level ARRA expenditure information, but it primarily serves as a conduit for resource and contact information—what funds are available and who to call about them. “We knew the web allowed us to eliminate a thousand phone calls a week. People were asking, ‘What is the money available for and where do I go to find out information for it,’” Oligmueller explains, “and a lot of that was driven by the fact that the federal government didn’t have all that information out there.”

We asked Greg LeRoy and Phil Mattera of Good Jobs First what they thought about this. Their response was, essentially, that state websites should be supplying even more details than appear on the federal website on how they are using the money.

LeRoy and Mattera strike an “if-not-now-when” note, saying that all levels of government are facing rising expectations for transparency and performance reporting. “The recovery act websites are fueling that expectation but it was already underway,” says LeRoy, who is the executive director of Good Jobs First. “We view this is as a two-year crash course in state disclosure,” adding that Good Jobs First is also ramping up to revisit its 2007 report “The State of State Disclosure.”

Other state officials we’ve talked to note that Good Jobs First gives credit to states that had pre-existing performance reporting websites and punishes those that didn’t. While we understand the logic there, it feels to us like however a state develops a superior site, it deserves to get credit for it.

Ranking the stimulus websites

All fifty states have set up websites that report on the way they’ve implemented the stimulus bill. Anyone who visits more than a handful of these sites will be immediately struck at the remarkable variety in quality. Some have clearly taken a great deal of thought, time and effort. Others seem like afterthoughts.

After a first stab at evaluating websites last July, Good Jobs First, tackled the task again in January. The organization is devoted to promoting government and corporate accountability in economic development. The broad conclusion of its new website evaluation: “A growing number of state ARRA sites deliver on President Obama’s promise that the stimulus plan would be carried out with an ‘unprecedented level of transparency and accountability.’ Yet some remain half-hearted efforts that provide taxpayers little useful data on the largest federal stimulus since the new deal.”

Some states may well have improved their sites since the release of this report. We, personally, know of a handful that are in the process of doing so now. But we think that the report’s interesting findings are still valid. Some of the points that jumped out at us:

• “More than half the states (28) now have some kind of project mapping feature on their ARRA site. Of these, 14 have interactive maps with significant project details, while 13 have such maps with more limited details.”
• “Only three states – – Kentucky, Maryland and Wisconsin – juxtapose the geographic distribution of spending with patterns of economic distress or need within the state.”
• “Despite the ready availability of Recovery.gov ARRA employment data, 10 states have no jobs data on their websites: Hawaii, Kansas, Louisiana, Mississippi, Missouri, New York, North Carolina, North Dakota, South Carolina and the District of Columbia.” (Note: the report counts Washington D.C. as a 51st state)
• Only five states – Connecticut Kentucky, Massachusetts, Mississippi and New Hampshire provide the full texts of at least some ARRA contract awards.”

As of the report’s publication, the half dozen top rated states were: Maryland, Kentucky, Connecticut, Colorado, Minnesota and Wisconsin. The bottom six were North Dakota, Washington D.C., Missouri, Alaska, Vermont and Louisiana.

Stay tuned. In coming days we’ll be sharing an interesting conversation with Beth Blauer, the director of the Maryland StateStat program. She oversees the website that was rated number one by Good Jobs First.