Editorial Type: research-article
 | 
Online Publication Date: 12 Jun 2023

CONSIDERING CHANGE MANAGEMENT WHEN CONDUCTING HPT EVALUATION

PhD
Article Category: Research Article
Page Range: 52 – 58
DOI: 10.56811/PFI-22-0011
Save
Download PDF

This article describes how Human Performance Technology (HPT) practitioners can use HPT evaluation models to evaluate organizational change. Change management encompasses all the processes identified in the ISPI HPT model. Practitioners must conduct an effective change management implementation to be successful. However, many change management models do not describe how to evaluate a change implementation. This article identifies six HPT evaluation models and describes how practitioners can use these models to evaluate performance improvement and change management implementations simultaneously.

CONSIDERING CHANGE MANAGEMENT WHEN CONDUCTING HPT EVALUATION

Change management is a crucial part of performance improvement that helps organizations maintain productivity and motivation (Bakari et al., 2017). The ISPI Human Performance Technology (HPT) model uses change management techniques to manage resistance and promote readiness for change (Van Tiem et al., 2012). It is common for people to resist change, and resistance may arise from administrators (Block, 2000), individuals (Oreg, 2003), or the context in which the change takes place (Burnes, 2015). Organizations use performance improvement to identify and reach goals and change management to get employees ready to achieve them.

Dessinger et al. (2012) describe how change management is necessary for each phase of the HPT model. The phases of the HPT model include the following: (a) performance analysis of need or opportunity; (b) intervention selection, design, and development; (c) intervention implementation and maintenance; and (d) evaluation. However, there is no change management model that aligns specifically with the HPT model. The field of change management contains different schools of thought, various models, and numerous strategies. Additionally, change managers select models based on the context of the change because one model is not appropriate for all change implementations (Pettigrew, 1987; Phillips, 2021). Although the HPT model provides a series of steps to promote new performance, the field of change management provides an abundance of implementation options.

Bernardez (2009) explains that a performance improvement implementation will have poor results if an organization does not align interventions that occur simultaneously. In the HPT model, performance improvement and change management are overlapping systems. Because change management occurs throughout all the steps of the HPT model, change managers and HPT practitioners should be aware of how change management and HPT systems interact. It is beneficial when practitioners from different fields share a common language (Abbott & Adams, 2016) and when the practitioners work together to monitor the effects of the change (Van Tiem et al., 2012). For example, HPT evaluators should serve as change agents because the outcomes of evaluation lead to change (Geis & Smith, 1992). Evaluators may provide a solution to improve an organization, but their efforts may fail if they do not promote the change that the evaluation encourages.

This paper describes the overlapping evaluation strategies found in performance improvement and change management implementations. Because change management is considered a component of performance improvement, this paper focuses on how HPT evaluation models relate to change management implementations. Whereas the field of HPT contains multiple evaluation models, change management literature often details the strategies that practitioners should use without describing how to evaluate the strategies. I will describe the change management models identified in ISPI literature, the HPT evaluation models listed in ISPI literature, and how the HPT evaluation models align with the goals of change management. Hopefully, the results will increase collaboration between the HPT and change management practitioners by describing similar evaluation strategies used by both fields.

CHANGE MANAGEMENT

ISPI-Identified Change Management Models

Change management encompasses the entire HPT process, and Van Tiem et al. (2012) identify the following eight change management models to promote organizational change:

  • 8-Step Model of Organizational Change

  • The ADKAR Model

  • Business Process Reengineering

  • Kaizen

  • Leadership Culture-Change Actions

  • McKinsey 7-S Framework

  • Three Stages of Change

  • Total Quality Management

Change management models offer prescriptive strategies to promote buy-in for the change. However, many models in the field of change management do not provide details on how to assess employees' attitudes toward a change or evaluate the effectiveness of the implementation. For example, only two of the eight change management authors mentioned by Van Tiem et al. (2012) describe assessment in their foundational publications. Hiatt (2006) provides an assessment tool that contains six open-ended questions. The questions ask participants to self-identify their acceptance of the change implementation and identify how the organization considered elements of ADKAR. Unlike Hiatt's (2006) post-implementation assessment, Lewin's (1951) force field equation, \(\def\upalpha{\unicode[Times]{x3B1}}\)\(\def\upbeta{\unicode[Times]{x3B2}}\)\(\def\upgamma{\unicode[Times]{x3B3}}\)\(\def\updelta{\unicode[Times]{x3B4}}\)\(\def\upvarepsilon{\unicode[Times]{x3B5}}\)\(\def\upzeta{\unicode[Times]{x3B6}}\)\(\def\upeta{\unicode[Times]{x3B7}}\)\(\def\uptheta{\unicode[Times]{x3B8}}\)\(\def\upiota{\unicode[Times]{x3B9}}\)\(\def\upkappa{\unicode[Times]{x3BA}}\)\(\def\uplambda{\unicode[Times]{x3BB}}\)\(\def\upmu{\unicode[Times]{x3BC}}\)\(\def\upnu{\unicode[Times]{x3BD}}\)\(\def\upxi{\unicode[Times]{x3BE}}\)\(\def\upomicron{\unicode[Times]{x3BF}}\)\(\def\uppi{\unicode[Times]{x3C0}}\)\(\def\uprho{\unicode[Times]{x3C1}}\)\(\def\upsigma{\unicode[Times]{x3C3}}\)\(\def\uptau{\unicode[Times]{x3C4}}\)\(\def\upupsilon{\unicode[Times]{x3C5}}\)\(\def\upphi{\unicode[Times]{x3C6}}\)\(\def\upchi{\unicode[Times]{x3C7}}\)\(\def\uppsy{\unicode[Times]{x3C8}}\)\(\def\upomega{\unicode[Times]{x3C9}}\)\(\def\bialpha{\boldsymbol{\alpha}}\)\(\def\bibeta{\boldsymbol{\beta}}\)\(\def\bigamma{\boldsymbol{\gamma}}\)\(\def\bidelta{\boldsymbol{\delta}}\)\(\def\bivarepsilon{\boldsymbol{\varepsilon}}\)\(\def\bizeta{\boldsymbol{\zeta}}\)\(\def\bieta{\boldsymbol{\eta}}\)\(\def\bitheta{\boldsymbol{\theta}}\)\(\def\biiota{\\boldsymbol{\iota}}\)\(\def\bikappa{\boldsymbol{\kappa}}\)\(\def\bilambda{\boldsymbol{\lambda}}\)\(\def\\bimu{\boldsymbol{\mu}}\)\(\def\binu{\boldsymbol{\nu}}\)\(\def\bixi{\boldsymbol{\xi}}\)\(\def\biomicron{\boldsymbol{\micron}}\)\(\def\bipi{\boldsymbol{\pi}}\)\(\def\birho{\boldsymbol{\rho}}\)\(\def\bisigma{\boldsymbol{\sigma}}\)\(\def\bitau{\boldsymbol{\\tau}}\)\(\def\biupsilon{\boldsymbol{\upsilon}}\)\(\def\biphi{\boldsymbol{\phi}}\)\(\def\bichi{\boldsymbol{\chi}}\)\(\def\bipsy{\boldsymbol{\psy}}\)\(\def\biomega{\boldsymbol{\omega}}\)\(\def\bupalpha{\bf{\alpha}}\)\(\def\bupbeta{\bf{\beta}}\)\(\def\bupgamma{\bf{\gamma}}\)\(\def\bupdelta{\bf{\delta}}\)\(\def\bupvarepsilon{\bf{\varepsilon}}\)\(\def\bupzeta{\bf{\zeta}}\)\(\def\bupeta{\bf{\eta}}\)\(\def\buptheta{\bf{\theta}}\)\(\def\bupiota{\bf{\iota}}\)\(\def\bupkappa{\bf{\kappa}}\)\(\def\\buplambda{\bf{\lambda}}\)\(\def\bupmu{\bf{\mu}}\)\(\def\bupnu{\bf{\nu}}\)\(\def\bupxi{\bf{\xi}}\)\(\def\bupomicron{\bf{\micron}}\)\(\def\buppi{\bf{\pi}}\)\(\def\buprho{\bf{\rho}}\)\(\def\bupsigma{\bf{\sigma}}\)\(\def\buptau{\bf{\tau}}\)\(\def\bupupsilon{\bf{\upsilon}}\)\(\def\bupphi{\bf{\phi}}\)\(\def\bupchi{\bf{\chi}}\)\(\def\buppsy{\bf{\psy}}\)\(\def\bupomega{\bf{\omega}}\)\(\def\bGamma{\bf{\Gamma}}\)\(\def\bDelta{\bf{\Delta}}\)\(\def\bTheta{\bf{\Theta}}\)\(\def\bLambda{\bf{\Lambda}}\)\(\def\bXi{\bf{\Xi}}\)\(\def\bPi{\bf{\Pi}}\)\(\def\bSigma{\bf{\Sigma}}\)\(\def\bPhi{\bf{\Phi}}\)\(\def\bPsi{\bf{\Psi}}\)\(\def\bOmega{\bf{\Omega}}\)\(| {f_{\left( L + \right),L}^*} | = | {f_{L,n}}|\), explains how much force (\(f)\) is required to move the present level (\(L\) ) in a different direction (\(n\)). Because people are resistant to change (Kirkpatrick & Kirkpatrick, 2006), the force field equation helps practitioners identify the level of force required to break a social habit.

Van Tiem et al. (2012) mention three models that align with performance improvement more than with change management. Business Process Reengineering (Davenport, 1993), Kaizen (Imai, 1986), and Total Quality Management (Deming, 1986) describe the effectiveness of promoting change. However, the purpose of these models is to change how an organization performs instead of focusing on the process of promoting change. For example, Total Quality Management (TQM) suggests that improving quality increases employee motivation and satisfaction. TQM appears to be a performance improvement model because the effectiveness of TQM is dependent on a performance outcome and not employee motivation and readiness for change.

The remaining three change management models—the 8-Step Model of Organizational Change (Kotter, 2012), Leadership Culture-Change Actions (Schein, 1969), and McKinsey 7-S Framework (Waterman et al., 1980)—identify specific strategies that change managers should follow. Each model uses a different collection of strategies, and practitioners must consider different evaluation measures based on which model they implement. Further, each model may implement the phases of the change in a different order. Whereas change management models consist of a sequential series of steps, change management frameworks are an interconnected series of strategies that do not follow a linear prescription. This requires practitioners to adjust the order in which they evaluate different phases. For example, Kotter (2012) suggests implementing the 8-Step Model in the order in which the phases are listed. However, Waterman et al. (1980) describe how the phases of the McKinsey 7-S Framework are interconnected without a starting point or order. Choosing an effective method to evaluate an organizational change would differ between models and frameworks because of the order in which the practitioner implements the phases.

Change Management Strategies

Each phase of a change management model consists of measurable strategies that describe how to promote change. Although change managers implement different strategies depending on the desires of the organization, there are similar strategies found in multiple models. Phillips and Klein (2022) identified the following change management strategies found in at least eight change management models:

  • Provide all members of the organization with clear communication about the change

  • Have open support and commitment from the administration

  • Focus on changing organizational culture

  • Distinguish the differences between leadership and management

  • Create a vision for the change that aligns with the organization's mission

  • Reward new behavior

  • Listen to employees' concerns about the change

  • Include employees in change decisions

  • Prepare for unexpected shifts

  • Generate short-term wins

  • Create groups or subsystems to tackle the change

  • Provide employees with training

Many change management models contain strategies that align with HPT's systems-based phases. For example, both fields consider (a) focusing on changing organizational culture, (b) providing employees with incentives, (c) providing employees with training, and (d) listening to employees' concerns. Because change management encompasses performance improvement in the HPT model, practitioners should evaluate similar performance improvement and change management strategies simultaneously (e.g., providing employees with incentives meets the goals of both change management and performance improvement). Instead of evaluating the strategies separately, practitioners should consider how these strategies systemically affect both performance improvement and change management implementations.

HPT EVALUATION

Whereas assessment is a measurement of a learner's knowledge, skills, or attitudes, evaluation is using those measurements to make a judgment. Van Tiem et al. (2012) define evaluation as (a) “a way to compare results with intentions and delve into the usefulness of methods and resources so that we may move towards the required results,” and (b) “the act of passing judgment on the value of a problem and its proposed solution” (p. 529). They list seven tasks that performance improvement practitioners conduct during an evaluation:

  • Establish the purpose, goals, objectives, criteria, and scope

  • Determine the feasibility

  • Select the evaluation methods

  • Collect data

  • Analyze data

  • Report the findings

  • Make recommendations

To achieve these tasks, the HPT model considers four types of evaluation—formative, summative, confirmative, and meta. Performance improvement practitioners use formative evaluation to continuously improve the implementation. Formative evaluation keeps the implementation aligned with the organizational mission and the objectives of the performance improvement effort. Van Tiem et al. (2012) describe summative evaluation as “the immediate effectiveness, efficiency, impact, and value of the intervention when it is implemented” (p. 553). Practitioners use summative evaluation to determine if the desired outcomes are met. Confirmative evaluation expands upon formative and summative evaluation to determine if the implementation continues to be effective. It identifies the lasting quality of the implementation and how the long-term effects of the performance improvement package align with the purpose of the implementation. Meta evaluation determines if the three types of evaluations were effective. It can be conducted during formative, summative, and confirmative evaluation to guide the implementation (Type One Meta Evaluation) or afterward to improve the reliability and validity of the evaluations (Type Two Meta Evaluation).

HPT Evaluation Models

There are numerous evaluation models in the field of performance improvement. Van Tiem et al. (2012) identify six “ageless” evaluation models used in HPT. The models vary in how practitioners should approach evaluation. However, each model can effectively evaluate both performance improvement and change management. Table 1 identifies the models and details the steps of each model.

TABLE 1 The Steps of the Six “Ageless” Evaluation Models
TABLE 1

Kirkpatrick's Four Levels

The Kirkpatrick Model consists of four levels of evaluation—reaction, learning, behavior, and results (Kirkpatrick & Kirkpatrick, 2006). When implementing a training intervention, the levels of evaluation align with elements of change management. Practitioners evaluate the participant's reaction to the change to determine their readiness and motivation toward the change. A positive reaction to the intervention encourages employees to support the change. Learning determines if the employees had a change in knowledge, skill, and attitude. By evaluating learning, practitioners can determine the effectiveness of implementing the change. Behavior is an aspect of organizational culture, which consists of employee behavioral patterns, demeanors, and attitudes (Craig, 2020). Although employees may have a change in knowledge, skill, and attitude, the purpose of change management is for employees to continuously engage in the new behavior. The results of the intervention describe the effectiveness of the change management implementation by providing administrators with evidence-based results on how change management strategies led to a change in behavior.

Table 2 identifies how Kirkpatrick and Kirkpatrick's (2006) levels of evaluation align with strategies found in the change management models.

TABLE 2 Similar Strategies Identified in Kirkpatrick's Four Levels and Change Management Models
TABLE 2

Brinkerhoff's Six-Stage Model

Brinkerhoff's (1987) six-stage model improves the efficiency and worth of training. Practitioners evaluate the stages of the model linearly and return to the first stage after completing the final stage. Brinkerhoff (1987) believes that organizational change should be considered throughout the stages of evaluation. Some examples include:

  • Stage 1 forecasts potential problems that arise when the change is implemented. Practitioners should treat the effect of the change as a cost of the program.

  • Stage 2 promotes buy-in from employees by including the employees in the program development process.

  • Stage 3 determines how communication between participants affects attitudinal change.

  • Stage 4 identifies if a change in knowledge, skills, and attitudes has occurred from the implementation.

  • Stage 5 determines if that change in behavior has transferred to the workplace.

  • Stage 6 evaluates the value of the change management implementation.

Geis and Smith Model

Geis and Smith (1992) use the ADDIE model to describe how human performance interventions consider evaluation. ADDIE consists of five phases—analyze, design, develop, implement, and evaluate. Geis and Smith (1992) modify the order of the five phases of the ADDIE model to increase the rate at which practitioners evaluate the phases. Instead of evaluation being the final phase of a linear model, they suggest that practitioners should evaluate throughout all the phases of the ADDIE model.

Geis and Smith (1992) focus on how evaluation leads to change. They suggest that evaluation occurs during and after each step of HPT evaluation and is both formative and summative. They believe that evaluators should have an active role in promoting change because evaluation influences change. A change is not permanent when the intervention ends, it requires continual monitoring and adjustment to promote new behavior. Geis and Smith (1992) mention several strategies found in change management models such as aligning people and functions with the mission of the organization, getting buy-in from decision-makers, and listening to stakeholders throughout the organization.

Kaufman-Keller-Watkins Model

Kaufman et al. (1995) believe that practitioners should evaluate worthiness inside and outside an organization. Their model evaluates employees' acceptance of change at each of the five levels:

  • Level 1 considers the availability and quality of resources and the employees' reactions to resources, methods, and tools.

  • Level 2 assesses the learner's acquisition of knowledge and evaluates how the intervention relates to the objectives.

  • Level 3 determines if the employees are using the new knowledge and skills on the job and identifies if the desired change in behavior occurred.

  • Level 4 identifies how the change has improved the organization and provides administrators with evidence to support the change management implementation.

  • Level 5 identifies how the change has improved society, which reinforces retention and desire for the change.

Dessinger-Moseley Full-Scope Evaluation Model

Dessinger and Moseley (2006) explain that evaluation should occur throughout all the HPT processes. Their Full-Scope Evaluation Model considers performance improvement and change management strategies. Practitioners use formative evaluation to evaluate the employees' reactions throughout the change and use the employees' feedback to help promote the change implementation. Further, practitioners use summative evaluation to determine how employees react to the outcome of the change implementation. Confirmative evaluation determines if the change in behavior continues after the implementation. This reinforces the change by promoting the retention of the new status quo. Because practitioners use change management strategies contextually, meta evaluation determines how reliable and valid the measurements were for different implementations. Meta evaluation helps promote an effective alignment between performance improvement and change management evaluations.

Combs and Falletta Targeted Evaluation Model

The Targeted Evaluation Model is a process-oriented approach to evaluation. The first three processes focus on the effectiveness of the intervention, while the final three processes consider how to implement the evaluation. Combs and Falletta (2000) promote organizational change by (a) evaluating the organization's readiness for change prior to the implementation, (b) implementing a predetermined change management plan, (c) delivering communication about the change throughout the organization at the optimal time, and (d) creating a training program that teaches employees about new their new roles and processes. Additionally, the steps of the Targeted Evaluation Model align with several change management strategies. For example, partner with stakeholders relates to three change management strategies—provide all members of the organization with clear communication about the change, have open support and commitment from the administration, and distinguish the differences between leadership and management.

CONSIDERING CHANGE MANAGEMENT WHILE CONDUCTING HPT EVALUATION

Evaluating Performance and Behavior

Change management addresses organizational performance and employee behavior (Army & Army, 2008; Griffith-Cooper & King, 2007). Whereas organizational performance focuses on meeting the desired goals, behavior focuses on sustaining the desired goals (Army & Army, 2008). All six HPT models evaluate a change in behavior by determining if the employees transferred and retained new knowledge, skills, and attitudes. Additionally, the HPT evaluation models include strategies found in numerous change management models. For example, evaluating the clarity of communication is a strategy of Kirkpatrick's Four Levels, Brinkerhoff's Six-Stage Model, and The Targeted Evaluation Model. These similarities allow practitioners to use HPT evaluation models as a foundation for change management evaluation. However, performance improvement and change management are different disciplines that sometimes evaluate discipline-specific processes. Using multiple evaluation models simultaneously can help practitioners evaluate processes that are not relevant to both performance improvement and change management.

Content-Focused and Process-Focused Models

HPT evaluation models are content-focused or process-focused (Chyung, 2015). Whereas content-focused models identify what practitioners should evaluate, process-focused models describe how practitioners should conduct an evaluation. For example, Kirkpatrick's Four Levels identifies what practitioners should evaluate (i.e., reaction, learning, behavior, and results), while the Targeted Evaluation Model identifies the steps that practitioners should use to evaluate the intervention (e.g., target stakeholders and gather and analyze data). Chyung (2015) explains that HPT practitioners should use both content-focused and process-focused evaluation models. By considering both types of models, HPT practitioners can (a) recognize the essential content required to evaluate performance improvement and change management simultaneously, and (b) have the necessary guides and rubrics to conduct a discipline-specific evaluation. For instance, an HPT implementation may use Kirkpatrick's Level 1 evaluation to evaluate an employee's reaction to the change while simultaneously using a process-focused evaluation model that judges the effectiveness of different forms of communication strategies.

CONCLUSION

The purpose of this article is to describe how HPT evaluation models can evaluate change management interventions. Because many change management models do not explain how to evaluate the change implementation, practitioners can use HPT evaluation models to evaluate both implementations simultaneously. Combining performance improvement and change management evaluations helps practitioners avoid initiating phases too early and duplicating evaluation efforts. Neither practitioner should switch to a new phase of the HPT model before identifying that the other has also completed the phase. For example, employees may become resistant to the change if an HPT practitioner moves to the cause analysis before the change manager has received buy-in for the purpose of the change. Additionally, change managers should reassess any phases that HPT practitioners adjust to promote employee buy-in for the change. Practitioners should implement HPT evaluation strategies that consider the effects of performance improvement and change management holistically. Although performance improvement and change management are separate disciplines, their overlapping strategies and similar goals encourage collaborative evaluation measures.

Copyright: © 2023 International Society for Performance Improvement 2023

Contributor Notes

JEFF PHILLIPS is an Assistant Professor at the University of West Florida. His areas of expertise stem from his work in academia, librarianship, and information technology. He has taught graduate courses at Florida State University, the University of West Florida, and the University of West Georgia. These courses were available in departments such as Instructional Systems and Learning Technologies, Information Studies, Information Technology, and Instructional Design and Technology.

Dr. Phillips earned a PhD and MS in Instructional Systems and Learning Technologies and an MS and BS in Library Information Studies. His research interests include human performance improvement, change management, and systematic instructional design. You can reach him at jbphillips@uwf.edu.

  • Download PDF