The New World Level 1 Reaction Sheets
Don Kirkpatrick developed the four levels of evaluation in the mid 1950s. He coined the term ‘reaction’ back then to describe what soon be came to know as “Level 1”. Don defines Level 1 (L1) as “The participants’ reactions to a training event. It is basically a measure of (internal) customer satisfaction.” Today, organizations around the world conduct some form of Level 1 using what they call “reaction sheets”, “smile sheets”, or “happy sheets”. They are surveys typically administered immediately after the conclusion of any type of training event, and are either completed with ‘paper and pencil’ or online.
These reaction sheets have been passed down through the generations, modified, automated, and are so common that many organizations simply refer to then as “evals.” Much data and information can be gleaned from well-constructed reaction sheets to be used to either improve the training or supply evidence for the beginning stages of demonstrating the value of training to the bottom line.
I am considered an expert in training evaluation, which is a fair statement since I learned from the legend (my father, Don). Over the past few months, I have had a nagging feeling in the back of my mind about most L1 evaluations not quite hitting the mark. It was just yesterday, as a matter of fact, when the nagging feeling materialized into a crystal clear set of conclusions. In keeping with the Kirkpatrick four level approach, I shall start with the end in mind. It is time for a radical change in the way we construct Level 1 reaction sheets!
Trainers are some of the most caring people on the planet. Nothing makes them happier than to see training participants enjoy their training and learn the knowledge and skills that the program was designed to teach. Unfortunately, as a profession, we are quite self-centered. “Nonsense” you say? I shall not only provide you with compelling evidence as to the truth of that statement, but provide a rather easy way out of our self-absorption. Why? To increase our effectiveness to our business partners.
Take a look at your Level 1 reaction sheet. Go on. I dare you. I will bet that you will find that it is mostly about us and our environment. Check for phrases like, “the facility was ...”, “the facilitator ...”, “the exercises were ...”, the materials were ...”. Do you want some more? “The course content was ...”, “which of the topics covered were ...”, “which of the methodologies were ...” You may not get where I am going, but you will. What we do is to ask participants – our customers – their thoughts about us and our methods, our buildings, and our coffee. Instead, we should be asking them about them – and how they are experiencing the training event in relation to their needs.
Below, I have developed a chart that will, hopefully, show you what I mean. I have differentiated these two contracting approaches as ‘trainer-centered, and learner-centered. Please assume for me, if you will, a four point Likert scale for each question defined by the terms, ‘strongly disagree’, ‘disagree’, ‘agree’, and ‘strongly agree’.
Evaluation Category |
Trainer-centered |
Learner-centered |
Program objectives |
The program objectives were clearly defined. The program objectives were covered by the instructor. The material was the right level of complexity for my background. |
I understood the learning objectives. I was able to relate each of the learning objectives to the learning I achieved. I was appropriately challenged by the material. |
Course materials |
The course materials were well organized. The course materials complemented the course content. |
I found the course materials easy to navigate. I felt that the course materials will be essential for my success. |
Content relevance |
The material was relevant to my needs. |
I will be able to immediately apply what I learned. |
Facilitator knowledge |
The facilitator demonstrated a good understanding of the material. The facilitator shared his/her experiences in regards to the content. |
My learning was enhanced by the knowledge of the facilitator. My learning was enhanced by the experiences shared by the facilitator. |
Facilitator delivery |
The facilitator effectively delivered the program material. The facilitator did a good job of generating participant interaction. The facilitator used a good variety of instructional methods. The pace of the program was good. The duration of the session was good. |
I was well engaged during the session. It was easy for me to get actively involved during the session. I was comfortable with the pace of the program. I was comfortable with the duration of the session. |
Facilitator style |
The facilitator managed the program well. The facilitator allowed for questions during the program. The exercises and activities were useful. |
I was well engaged during the session. I was given ample opportunity to get answers to my questions. I was given ample opportunity to practice the skills I am asked to learn. |
Program evaluation |
The test was a fair representation of the program content. The role plays or simulations were a fair representation of the program content. |
I was given ample opportunity to demonstrate my knowledge. I was given ample opportunity to demonstrate my skills. |
Breaks |
The breaks were spaced at the right times during the session. |
I felt refreshed after the breaks. |
Facility |
The lighting was adequate. The temperature was comfortable. The coffee was hot. |
I found the room atmosphere to be comfortable. I was pleased with the room set up. I experienced minimal distractions during the session. |
I trust you see where I am coming from. We need to make Level 1 less about us and more about the impressions and feelings about our programs. Learners have enough trouble with thinking that they are being “sent” to training as a reward, punishment, or to earn a ‘check’ on some training activity sheet. I urge you to consider at least some of these questions in your L1 reaction sheet. It will help put the emphasis more on the learner and less on us.
SMR USA's Public Sessions, 2008
|
