pdf - Nyenrode Business Universiteit

3.3. SAMPLE AND MEASUREMENT 69
3.3.3 Measurement
This section describes the measurement of the variables used for this study. I discuss the
variables related to the research hypothesis (i.e., noise in the performance evaluation, and
RPE-Use), as well as the noise drivers (i.e., environmental uncertainty, goal ambiguity,
measurability of outputs, emphasis on personal-level measures, and emphasis on disaggregated
measures). Additionally, this section motivates and describes the control variables
for size and sector. When references are made to the questionnaire items, please see the
appendix at the end of the thesis for the exact phrasing, response categories, and, if applicable,
sources.
Noise in the Performance Evaluation The dependent variable noise in the performance
evaluation is based on a single question that asks the respondent to indicate the
extent to which the performance evaluation that he receives is based on factors over which
he has control (Q34). Throughout this chapter, this proxy is referred to as ‘NOISE’.
NOISE is measured by taking the reverse of the respondents’ answers. In other words,
NOISE measures the extent to which the performance evaluation is influenced by factors
that lie beyond the business unit manager’s control. I acknowledge that relying on a single
indicator construct is less than ideal, especially when measuring the dependent variable
in the model. Therefore, I seek additional validation for NOISE. The questionnaire provides
additional items that support this metric. Specifically, the questionnaire provides
two items that are associated with the variable of interest, though they do not measure
noise in the performance evaluation per se. For one, I asked the respondents to indicate
the extent to which the business unit’s performance is substantially influenced by factors
beyond his control (labelled: exogenous-performance-effects, Q35). Similar to NOISE, this
question refers to the impact of exogenous factors. However, this time on the difficulty of
the business unit performance instead of on the performance evaluation. Several reasons
exist why NOISE and exogenous-performance-effects are not perfectly correlated. Firstly,
the performance evaluation can be informed by things other than the business unit’s performance
(e.g., contributions to firm performance, or personal development of skills). If
so, the exogenous events that impact the business unit’s performance, impact the performance
evaluation only to a certain degree. Second, the performance evaluation can be
insulated from exogenous events that influence business unit’s performance through, for
example, RPE. As argued before, RPE filters noise out of the performance evaluation.
However, RPE does not remove the impact of exogenous events on the BU performance itself.
Therefore, I expect NOISE and exogenous-performance-effects to correlate positively
with one another, though not to a great extent.
The second questionnaire item to validate NOISE captures more broadly and informally
whether the respondent is held accountable for uncontrollable factors, instead of whether