The Essentials of Coaching Program Evaluation: Formative, Summative and Four Ds

Campbell and Stanley’s “time-series” design requires that some standard measure be taken periodically throughout the life of the organization: for example, rates of executive turnover, average duration from product conception to delivery, percentage of product rejection in a production line – or more obvious measures such as profit and loss, sales volume or number of customers. If such a measurement relates directly to one of the anticipated outcomes of the coaching program being evaluated, then we are looking for a significant change in this measurement over time. Hopefully, this change will occur after the program has been in place for a given amount of time among those units of the organization that are participating in the program. With this design, a sufficient number of measures must be taken before and after the program is initiated in order to establish a comparative base. At least three measures should be taken before and two measures after program initiation.

The second quasi-experimental design, “nonequivalent control group design,” is a bit more complex; however, it will in some cases help the evaluator partially overcome the Hawthorne effect among experimental group members and the sense of inferiority and “guinea pig” status among control group members. Rather than randomly selecting people into an experimental or control group, the evaluator can make use of two or more existing units (teams, departments or divisions). Two programs being offered by the HR Department, for instance, might be offered to several units in the organization. One or more of these programs would be those already provided by the HR Department, such as a management development program or online technology update seminars. The new coaching program would be the additional option.  Client units would select one of the program offerings on the basis of time preference, convenience of location, specific need at the moment, etc. It is hoped that these reasons would function independently of the outcomes being studied in the evaluation. One of the units would be given the new coaching program (the experimental group), while the other unit(s) (the control group/s) receive the program(s) already provided by HR.