pdf - Nyenrode Business Universiteit

80 CHAPTER 3. NOISE REDUCTION EFFECTIVENESS
the performance. The next subsection uses an alternative operationalization of RPE use
to further assess the robustness of the findings.
3.4.2.2 Analyses with RPE-based-Targets Measure
To further buttress the findings of the base model analyses, I run a robustness check. The
robustness check consists of rerunning the base model analysis with a different operationalization
of RPE use (i.e., RPE-based-Targets).
RPE-based-Targets specifically measures the explicit use of RPE for target setting purposes
(i.e. how peer performance influences the difficulty of the performance standards
used to evaluate the manager’s performance). I measure RPE-based-Targets as the impact
of peer performance on the performance target. I ask to what extent the performance
of the peers influences the difficulty of the target and/or budget for a variety of performance
measures as described in the appendix (Q2). To interpret the impact of peers on
the evaluation as a whole, I weigh the various performance metrics. I weigh the impact
of peers on each of the metrics by multiplying the scores with their individual importance
for the performance evaluation (Q3). RPE-based-Targets does not solely measure ex-post
peer comparison. The influence of peer performance on the agent’s target can also take
place at the beginning of the period. In this case, the target for the upcoming period is
based on the actual performance of peers during the current period. I expect the second
RPE measure to correlate with the primary measure ‘RPE-Use’, because they both include
explicit RPE use and ex-post application of the peer comparison information.Because of
the differences between the measures presented above, the correlation is not necessarily
strong. Correlation analysis confirms this; the two measures for RPE are correlated, but
not strongly (0.498; p < 0.01).
The analyses with the RPE-based-Targets measure support the findings of the base model’s
analyses. The robustness check yields results that are qualitatively similar to those of the
model with the primary RPE measure (RPE-Use). The model is significant (chi 2 statistic
p = 0.274). Additionally, the goodness-of-fit statistics suggest a sufficient model fit. The
explained variance in NOISE is slightly higher than the explained variance in the base
model at 13%. Similar to the base model, the model with the RPE-based-Targets proxy
supports the main research hypothesis regarding the noise mitigating effect of RPE (H1)
and the additional expectations regarding the drivers of noise. Additionally, in accordance
with the base model, environmental uncertainty does not increase the amount of noise
in the performance evaluation. The results of the robustness check are summarized in
table 3.11. According to the kurtosis, the current model also has potential non-normality
issues. Similar to the base model, a Bolen-Stine bootstrap (p = 0.305) indicates the fit
between the data and the model. In other words, the kurtosis does not significantly affect
the results of the estimated model. In sum, the robustness check supports the validity of
the findings in the base model’s analysis. Although minor quantitative changes occur, the
implications of the models are qualitatively similar.