Contents of It's not All in Your Head by Asmundson and Taylor

3.2.2. Inter-rater reliability coding
The Head Researcher and a graduate level volunteer conducted inter-rater
reliability coding following the procedure outlined in Hruschka et al., (2004). After
creating response units and subcategories the procedure involved: creating definitions
(descriptions) of the subcategories; two or more coders independently coding a portion of
the responses; comparing the coding results; making any necessary modifications to the
theme subcategories; and then coding the entire data set. The Head Researcher created
the theme subcategory definitions. Response units (i.e., phrases and words; 61 in total),
subcategories (i.e., themes; three to seven per category), and categories (i.e., open-ended
questions; five in total) were then compiled into a multiple choice online survey (using
Survey Monkey.com). Each category was presented on a separate page (i.e., five pages
for five questions) and each response unit was presented as a survey question. The
subcategories identified for each question were presented as multiple-choice options with
descriptions of the subcategory in parentheses beside each option. The coding task was to
select one subcategory for each response unit. For each response unit (i.e., survey
question) there was also a text box where an alternative subcategory could be entered if
none of the presented options were appropriate.
Before coding began, the study and coding procedure were explained to the
volunteer. The Head Researcher and volunteer individually completed a random selection
of five questions and the coding responses were identical. Then each coder individually
73