How deep do you want to dive?

by

After years of looking at performance measurement in a whole variety of fields, we’ve come up with two basic principles that we think apply in most cases:

1) You can’t have enough performance measurement and evaluation.

2) Too many performance measures are a bad thing.

“You’re going to have to do a sampling and you’re going to have to be pretty choosy as to what you go and look at, and then make some fundamental decisions about how deep you want the dive to be. ” – Jeffrey A. Simon, director of the Massachusetts Reinvestment and Recovery Office.

These may not seem entirely consistent, we know. But, at heart, they just represent the conflict between an ideal world and the real world. In the ideal world, there would be sufficient time and resources to measure the effectiveness and the efficiency of every government program, and to report those findings to as many people as possible. In the real world, on the other hand, executive managers, auditors, legislators and their staffs will probably never have enough time to look at a fraction of all the metrics potentially available. What’s more, just creating all that information can wind up being an impossible burden on the men and women who are trying to keep up.

Truth is, creating a measure that nobody looks at is a lot like building an energy-efficient  car that can only move on gold-paved roads. It might be a great car, but it’s never going to go anyplace.

What does this have to do with the stimulus? Simple. Even as states move forward in the direction of measuring the actual outcomes of these federally-funded activities, they’re not going to have the time and resources to measure outcomes across the board in a timely way. We started thinking about this after a terrific conversation with Jeffrey A. Simon, the director of the Massachusetts Reinvestment and Recovery Office.  Although his office has been largely concerned with the basics of the Stimulus Act thus far (making sure the money is properly spent and so on), he’s enthusiastic about understanding what’s actually been done with those dollars, beyond job preservation and creation.

Still, he’s convinced that the states are going to have to carefully pick and choose what things to measure or be overloaded by an impossible task. “We’ve just begun to spend some serious time in my office talking about what it means moving into the implementation phase,” he told us, “as opposed to project selection or spending so much time on federal reporting requirements. All of this is important . . .But then you need to step back and see, from an implementation point of view, what’s being accomplished.

Jeffrey Simon

“You’re going to  have to take a close look at the skill sets that you need to do that kind of thing, and the sheer volume of projects. You’re going to have to do a sampling and you’re going to have to be pretty choosy as to what you go and look at, and then make some fundamental decisions about how deep you want the dive to be. How far in the weeds to we want to get?

“I’d tend to take fewer projects and go farther down in the expenditure to the last dollar and understand what it did, and look at the product it produced, whether it’s a weatherized home or a child care voucher.

“One of the distinguishing characteristics of the recovery program is it goes all across state government. It’s not like I can have two or three different people and say, ‘today you’re going to look at housing vouchers and tomorrow you’ll look at road paving.’ Those are fundamentally different. So, I’d need an army of people to do a full examination. And that’s why we’re going to have to make choices.”

Advertisements

Tags:

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: