Developing a Self-Assessment Toolfor Culturally - Office of Minority ...

In an effort to make the telephone interviews with senior executives even more efficient,the final question posed would be to ask the executive to identify or recommend appropriate staffwithin the organization to complete the Staffing and Membership Questionnaires. Because aminimum of 480 Staffing and Membership respondents—i.e., two respondents in each of the 240MCOs—would be asked to complete a questionnaire (that would be much lengthier than thesenior executive’s inquiry), these data would be collected by mail rather than telephone, toreduce costs. Once Staffing and Membership respondents were identified by the seniorexecutive at the conclusion of the telephone interview, a personalized introductory letter wouldbe mailed to invite the individuals to participate in the study, along with a copy of the respectivequestionnaire and instructions for submitting the completed questionnaire.The need for follow-up contacts with respondents who do not submit a completedquestionnaire in a timely manner was considered during the data collection planning phase. Theresearch team, OMH staff, and members of the advisory groups agreed that multiple follow-upcontacts may be required with these respondents. As such, a series of reminders in variousforms were determined necessary for this population. Reminder postcards would bedisseminated to respondents as a first step, and up to three reminder telephone calls perrespondent would become part of the data collection plan.2.1 Conducting the Pilot TestIn order to test the appropriateness of the data collection process and the study instruments,a pilot assessment using cognitive testing techniques was conducted with nine respondentrepresentatives—one Senior Executive respondent, one Staffing respondent, and oneMembership respondent from each of three MCOs—between March and June 2001. The sixStaffing and Membership respondents completed a Pilot Test Response Form (see Appendix F)that was designed to provide respondent feedback on the format and content of the instruments.For each questionnaire component (cover letter; questionnaire cover sheet with instructions; eachof four questionnaire sections), respondents were asked to rate certain elements (e.g., layout anddesign; clarity of instructions; clarity of question wording and response categories; ability toprovide requested information) based on the level of appropriateness for presenting eachcomponent to someone in a similar staff position as themselves.An average rating score was used to summarize the responses to the 4-point scale(i.e., 1 = very appropriate; 2 = appropriate; 3 = somewhat appropriate; and 4 = not appropriate).The three Staffing pilot respondents reported an average rate of 1.4, and the three Membershippilot respondents reported an average rate of 1.3. Moreover, there were no responses of “notappropriate” for any component of the Staffing or Membership Questionnaires. These resultsdemonstrated that for the pilot participants, the Staffing and Membership Questionnaires werefound to be understandable and appropriate—in both format and content—for the purposes ofthis study.By implementing the initial data collection plan with a test group of respondents, severalmethodological changes were found to be necessary. These modifications are described below.COSMOS Corporation, December 2003 G-5