Measures for Workforce Training . . . A Minnesota Audit’s Critique

Last week, the Department of Labor announced the availability of $90 million in added Recovery Act funding for on-the-job training opportunities.  “States and their partners will use this funding to create on-the-job training experiences, which will improve the employment prospects of dislocated workers in areas that have been hard hit by the recent economic downturn,” said Secretary of Labor Hilda Solis.

Secretary of Labor Hilda Solis

It seems to us that there needs to be a solid system of measurement that will allow state and federal observers to see the impact this money has. Maybe one has already been developed for this newly revitalized program.

But we sure hope it is better than the federal measures that are currently in place for tracking the success of more traditional workforce training programs. The latter were recently panned in a Minnesota legislative audit, which cited problems that will be very familiar  to long-term government observers who recognize some of the same measurement headaches that afflicted the Job Training Partnership Act back in the 1980s.

The audit mentioned four major issues.

1.       Federal measures allow programs to be selective in whom they help. As the audit says “The measures do not distinguish among clients with different capabilities.” If a workforce training program places a former inmate with little schooling, the accomplishment is viewed as having the same value as  placing a skilled person with a college degree. One program director told the audit crew that her agency “avoided enrolling individuals in the adult program if they appeared unlikely to succeed.”

2.       Gaming program data is easy. The Minnesota audit found that workforce training programs routinely timed the entrance and exit from programs to generate the most favorable results. The fact that there are no rules for establishing when someone enters or exits a program makes it impossible to accurately compare different entities. (If a person gets a job, some service areas wait until the last day of the quarter to decide to exit the person from the program. If he still has a job on the last day of the quarter, the program can count on a record of employment the subsequent quarter – which starts the next day.)

3.       Measurement and evaluation continues to focus on the short-term. There is currently no requirement to look at the impact of the workforce training programs beyond a nine-month period. In our minds, the real goal of training programs is to keep people in jobs, not just get them into jobs in the first place.

4.       There are no requirements to test the impact of programs by comparing individuals who receive services with those who have similar characteristics but didn’t receive services.

The Government Accountability Office has commented on these issues frequently in the past and we know the Department of Labor has tried to make improvements in its approach. But it’s clear from this audit, that there’s a great distance that still needs to be traveled to arrive at a useful way of evaluating workforce training.  In the Minnesota legislative audit, at least one program director characterized federal measurement requirements as “one of the biggest impediments to the effective delivery of workforce programs. “

The cloud inside the silver lining

Curiously, sometimes having more money available for an entitlement program – through ARRA, for example –  can actually mean that while access improves, quality of service declines on certain measures.

Stimulus dollars have been used across the country to expand the Supplemental Nutrition Assistance Program (formerly known as food stamps).  But the  Texas auditor’s office discovered that with the increased case loads, wait times for applicants have gone way up. In 2006, 92 percent of applications were processed in a timely way, according to the audit. In the first part of Fiscal year 2010, that number had fallen to less than 70 percent.

One of the problems cited in the report is the fact that many of the workers hired to serve the growing population of food stamp recipients are new. Those who have less than two years experience are now 41 percent of the front line workforce – compared to four percent in Fiscal year 2005.

We wonder. Are these problems unique to Texas. Or does the Texas experience reflect what’s happening in other states as well?

From the auditor’s seat

In upcoming weeks, we’ll be talking with local and state auditors about their observations about the Recovery Act and how it’s affecting their work. If you’re curious to see some examples of how local auditors are approaching the challenges of monitoring Recovery Act dollars, the Association of Local Government Auditors has set up a web page to collect local audits relating to the stimulus.  To date, a similar repository has not been established on the state side, according to the National State Auditors Association. But the Recovery Board has asked states to send in Recovery Act audits and a small collection is developing in the Recovery.gov accountability section.