The Essentials of Coaching Program Evaluation: Formative, Summative and Four Ds

The three stages of progressive focusing have been summarized by Parlett:

Obviously, the three stages overlap and functionally interrelate. The transition from stage to stage, as the investigation unfolds, occurs as problem areas become progressively clarified and re-defined. The course of the study cannot be charted in advance. Beginning with an extensive data base, the researchers systematically reduce the breadth of their inquiry to give more concentrated attention to the emerging issues. This progressive focusing permits unique and unpredicted phenomena to be given due weight. It reduces the problem of data overload and prevents the accumulation of a mass of unanalyzed material.

These three appreciative characteristics of diagnostic evaluation (qualitative analysis, systematic perspectives and progressive focusing) are often troublesome for both inexperienced and traditional evaluators. These characteristics appear to fly in the face of a contemporary emphasis on precision, measurement, objectivity and the discovery of deficits. Such is not the case, however, for these three characteristics can serve to enhance rather than take the place of a more traditional “scientific” evaluation.

In looking appreciatively at cause and effect relationships in a complex social setting and working with a complex set of activities (such as coaching), a whole variety of tools and concepts must be considered. In attempting to better understand the workings of a specific coaching program, the evaluator, like the cultural anthropologist, uses a variety of data collection methods, ranging from participant-observation and interviews to questionnaires and activity logs. Parlett suggests that the experienced evaluator also emulates the anthropologist in making use of various data analysis methods, ranging from narration and metaphor to multivariate statistics. This approach to evaluation was identified by Parlett several decades ago–yet it continues to challenge most approaches to evaluation and points to the “new” kinds of program evaluation that are needed if we truly intend to better understand how professional coaching operates when successful.