Evaluating the Impact of a Training Program
Government Agency with 9000+ employees Issue: The client wanted to measure the impact of a company-wide full-day “Mutual Respect” training program that they were implementing.
We conducted a four level evaluation to determine the impact of this training program. The evaluation consisted of the following steps:
- Level 1: We evaluated the content of the training and the facilitators through questionnaires given to all attendees.
- Level 2: We measured the level of learning by developing and administering a pre-training and post-training questionnaire
- Level 3: We measured behavioral changes by developing a questionnaire and conducting approximately 40 focus groups with employees, supervisors and managers.
- Level 4: We utilized all the data collected in Levels 1-3 to answer the following questions: Was the training worth the resources expended? Did the training meet its goals?
The evaluation team determined that based on the initial criteria established by the client and the quantitative and qualitative results, the employees did benefit from the training and that the goals established prior to the training were met.
We also recommended that our client develop a strategic vision and plan to leverage the momentum they had from this initial round of training. It was further recommended that further training be given to the supervisors to reinforce the concepts covered during the training as approximately 2/3 of them had not made any changes as a result of the initial training.
Specific data collected included:
- Level 1: The content of the training and the trainers received an overall approval rating of approximately 90%.
- Level 2: There was a 25% increase in the knowledge from the pre-test to the post-test. Pre-test scores were 72% and post-test scores were 97% (respondents who correctly answered the questions).
- Level 3:
- 41% stated that their organization has an inclusive environment (59% stated that it did not).
- 49% stated that they personally were doing something different after attending the training class
- Of the individuals who reported that they are doing something different: (top 3)
- 42% reported that they were more careful about what they said
- 18% reported they were paying more attention
- 15% reported they were addressing issues
- 69% stated that their supervisors/managers had not made any changes after the training
- Level 4: The training had the desired impact and the data revealed areas where further work was necessary.