Energy efficiency and Demand Side Management Program ... - Eskom

3) They quantify results and cost‐effectiveness, as necessary, to help better communicate the value
of the program.
Some key definitions are described below:
Program: A project or group of projects with similar characteristics and installed in similar
applications, where multi‐faceted effects are observed and evaluated.
Evaluation: The performance of studies and activities aimed at determining the effects of a program;
any of a wide range of assessment activities associated with understanding or documenting program
performance, assessing program or program‐related markets and market operations; any of a wide
range of evaluative efforts including assessing program‐induced changes in energy efficiency
markets, levels of demand or energy savings, environmental impacts, social and economic impacts
and program cost‐effectiveness.
Program evaluations: Program evaluations are systematic and objective studies, conducted
periodically or on an ad hoc basis, to assess how well a program is achieving its intended goals. These
evaluations have a retrospective focus, with a view to assessing past performance and developing
recommendations for improvements, with an exception of the evaluations of market needs that can
have a current or prospective focus. Some evaluations usually require certain level of details in data
collection and analytical methodology that goes beyond routine performance‐monitoring reporting.
This helps the decision makers determine what kinds of timely adjustments may be needed in
program design or implementation to improve the rate or quality of achievement relative to the
committed resources. It is not necessary to have in‐depth familiarity with these methods to benefit
from a general program evaluation, but program managers need to have enough familiarity to select
and monitor an evaluation contractor who will make decisions about evaluation methodologies.
The POET guideline:
1) Explains how to establish an evaluation marking system with the help of the POET classification
approach
2) Provides guidance on how to evaluate a program from not only engineering aspects, but also
social, economic, and environmental aspects; and
3) Gives a vision of future M&V projects to evaluate social, economic, and environmental issues.
4.2 Purpose
This program evaluation guideline should serve two critical purposes – program improvement and
accountability. Many evaluations will be designed to serve both of these purposes.
The purpose of this guide is to create and manage objective, high quality, independent and useful
program evaluations. The guide could be used by those without prior training or experience in
program evaluation and should make it easier for them to take advantage of this useful and
increasingly required program‐management tool. Such a guide is intended for use by all stakeholders
involved in any EEDSM program.
This guide does not answer all of the technical questions about evaluation and M&V methodology,
but it will provide enough information to help program managers to:
Identify the questions that they need to answer to fulfil a general program evaluation
Monitor the evaluation progress
Implement credible quality assurance (QA) controls
7