SMR USA E-News - January 2008

If you cannot view this E-News, please copy this url and paste into your browser.
http://www.smr-usa.com/enews/january_2008.htm

Greetings from SMR USA - by Jim Kirkpatrick

I would like to wish all of you a very happy and productive new year. I have taken the opportunity of a few days off to write out my thoughts of how SMR’s Kirkpatrick Four Level Evaluation has evolved during the past year, and how you might benefit from those changes in 2008 and beyond.

“Don’t be fooled into thinking that just because our senior business partners are not specifically asking for us to demonstrate the impact of training that they don’t care!”. These words from a senior training manager for a major automobile manufacturer should strike a note of fear in your heart. Or at least, a flutter of caution. Tammy went on to say, “Here is how it works. They do not ask for us to demonstrate the value of our learning initiatives, and even off-handedly mention that they hear good things about our programs. But when budget time comes, and there has been issued a directive to tighten the belt a bit, they show their true colors. Guess who they look to for the first line of cuts? Training!!”

My father, Don Kirkpatrick, developed the four levels of evaluation over 50 years ago, and they are still with us today. Don identified four words that described how to evaluate the effectiveness of a training program back in the mid-1950’s while working on his doctoral dissertation – reaction (Level 1, or how training participants thought of their training program); learning (Level 2, or what knowledge, skills, and attitudes were acquired through the training); behavior (Level 3, or to what degree did participants apply on the job what they learned); and results (Level 4, or what results came to be from the training). Don’s model remains the most familiar and the most effective evaluation model on the planet. Oh, there are scoffers and skeptics who think otherwise, but they have not apprised themselves as to how the four levels have evolved over the past few years. There are other theories, mostly modifications of Don’s original work, which work effectively with the elite few organizations which think they have mastered the four levels. Don and I are truly glad for them, as they do offer variety and a look into using evaluation to actually predict where HRD monies and resources should best be applied for future business success.

What is it, then, that makes Kirkpatrick’s four levels the most useful and critical approach going forward? It is the focus on what we call the Business Partnership Model. Our friends and colleagues, Jim and Dana Robinson, have written recent books and articles about this model, and nowhere is it more applicable than with evaluation. I was tempted to entitle this article, “You had better develop and leverage your own business partnership model or you will soon be out of business!”, but it contained too many words. In this instance, the business partnership model specifically refers to the relationship between the learning function and the business.

Below are six keys to the successful implementation and execution of the Business Partnership Model, through the four levels.

1. Learning is a process, not an event.
There remains the strong myth today that training departments, corporate universities, learning functions – whatever you want to call us – are in the business of developing and delivering training programs. Continued thinking along this line will form the first nail in your training coffin. Instead, “training programs” needs to be replaced with “learning events.” Research has proven over and over that a single training or learning event is not as effective as a collection of short, targeted, blended learning initiatives. This is especially true when participants have the opportunity to apply what they learn – on the job – in between sessions.

2. Learning initiatives need to be tied to the needs of the business.
Now, don’t skip over this and say, “Yea, we know that.” I am quite sure that most of you know this, but how well are you actually making it come to life? If you are honest, very few. A quite training needs analysis, or a brief discussion with business leaders prior to a major training initiative is typically not enough. You really need to put on your internal consultant hat for this one, earn a proverbial seat at the table with your business partners, and target each learning initiative to actual business needs, problems, or opportunities to get the job done. We have developed a specific methodology for this by taking a fresh look at the popular yet misunderstood ADDIE Model. The letters of this model, used mostly by instructional designers, stand for Analysis, Design, Development, Implementation, Evaluation. It may come as no surprise to you that the “A” is where this point either stands of falls. A model that depicts the relationships between strategy, instructional design, and evaluation is shown below.

3. “More training!!” is not always the answer.
In my travels, I do find that most training professionals are well aware of this. I worked with a healthcare organization recently that is in its fourth rendition of a patient care initiative in six years. All four have similar content, similar delivery methods, and similar desired outcomes. What changes each time is the name. Sound ridiculous? It is, but here is the problem. Each year or so, this training is rolled out, yet it never achieves the desired results? And each time, the learning professionals know that the content was targeted to the desired needs, and that participants left their learning events with the necessary knowledge and skill to achieve the targeted results, but something happened along the way that led to underachievement of targeted results. Unfortunately, this organization had no data to determine what factors – other than at Level 1 and Level 2 – actually led to the failure. So, each year when senior executives said, “We need more training (i.e., new learning initiatives), the learning professionals had no choice but to say, “OK”, then go about their ADDIE business and reinvent the patient care wheel. Effective implementation of all four levels can easily rectify that situation.

4. Return on Expectations is the key to business partnership success.
Once the needs of the business have been detailed, and it becomes clear that new training is at least an ingredient in the recipe for business success, it is time to really leverage the business partnership model. A colleague of ours, Diederick Stoel, talks about “negotiating value propositions”. What he means by that is this. Learning leaders must dialogue with key business stakeholders to identify and agree on expectations for the learning events. Often, what the business leaders expect are vastly different from what the learning leaders believe they can deliver on. These differences must be negotiated so that the expectations are not only satisfying to the former, but are realistic in the eyes of the latter. Once these rather generic expectations are identified, they need to be converted into success indicators. In other words, you must ask and help answer the question, “What will success look like?”. This must be done in such a way as to be able to convert them into metrics that then become important Level 4 outcome goals. In addition to determining how to demonstrate the return on expectations, you must also manage stakeholder expectations along the way. This means providing Level 2 and Level 3 metrics and indicators along the way, so that a) you can be sure that the Level 4 goals will ultimately be reached, and b) so that you can double check with you stakeholders along the way to ensure that they are confident in what is happening.

5. You had better create value before trying to demonstrate it.
Nothing contributes more to the inadequacy of and threat to the future of training than this. It actually causes my fingers pain as I type this. Training leaders and even some training associations believe that if training is done in a manner that is enjoyable, relevant, and effective, that somehow it will lead to significant business results. This myth is perpetuated by both learning professionals and business leaders. Many learning professionals truly believe that huddling amongst their own kind, and working diligently on the “DDI” of the ADDIE model will ultimately (and magically) lead to positive business impact. No. It will lead to some impact, but likely not enough for you to keep your job. Execution is the key to actually achieving the results that the business is looking for. At the risk of alienating non-sports fans, let me illustrate what I am talking about by using an example of the 2007 Super Bowl champion Indianapolis Colts. Here is how the Colts approached their 2006-2007 season. They conducted training camps for all of their players, handed out playbooks for all to memorize, and decided on and posted (probably on a flip chart) end-of-season goals (e.g., win the division, win the conference, and reach the Super Bowl). Then, head coach Tony Dungy addressed the team just prior to the first game, “Well men, we have 16 tough games ahead of us. I want you all to do your very best, and at the end of the season, let’s see how well we did.” Why, this is ludicrous! But, many businesses actually conduct themselves that way. They ‘send people to training’, then send them back to their jobs, then hope for the best. Anyone who knows anything about any sport knows that proper execution on the field is the key to success. The same is true of business, and targeted Level 3 evaluation (along with coaching, alignment of desired behaviors with incentives, accountability, reinforcement and support) can and will make all the difference between mediocre and exceptional Level 4 performance (just ask the Colts!).

6. Once you have leveraged learning to business success, you better know how to demonstrate it.
Don and I recently wrote a book called Implementing the Four Levels (Berrett-Koehler, 2007). In it, we use the metaphor of a law firm presenting a case to a court room jury. We have received much feedback that this metaphor accurately and strikingly represents what learning functions need to do to demonstrate their value to the business. While details are provided in the book, I would like to point out a few main principles. First, we need to know that there is a corporate jury lurking out there whether we can clearly identify it or not, and whether they are actively reviewing our case or not. And what is our case? That learning is making a significant contribution to the business. Second, that jury is made up of different types of people with very different personalities. We need to know who is on that jury (i.e., our key business stakeholders and those who determine our budgets), and gather evidence for value that will be meaningful to them (see ROE above). Third, we need to present our evidence in such as way that it is compelling to them, and will lead them to return a favorable verdict to us.

Well, there you have it. What started out in 1955 as four simple words that can be used to evaluate the effectiveness of training programs have now, believe it or not, training is on trail, and there is a storm coming that will likely lead to massive outsourcing of training unless we become true business partners.

There are some upcoming SMR events that I would like to direct your attention to. First, we are co-sponsoring a Kirkpatrick Evaluation Summit with Training Magazine. This event, to be held in Atlanta, Georgia, USA, will take place the first week in February, 2008. This will be a cutting edge, 2-day event that wil address all of the issues presented above. Our Asia HRD Congress 2008, to be held in Jakarta, Indonesia from 22nd - 24th July, 2008, will also address all of the topics mentioned above by a number of world class learning professionals.

Best regards,
Jim Kirkpatrick, PhD
VP Global Training and Consulting
SMR USA, Inc.