In-Basket Design: Developing and Implementing an On-Line ... - IPAC
In-Basket Design: Developing and Implementing an On-Line ... - IPAC In-Basket Design: Developing and Implementing an On-Line ... - IPAC
Step 4: Develop Performance Benchmarks Specific E-mail Benchmarks • Are developed by well qualified SMEs • Have held the target position • Are not eligible to take the test – Are currently at target level or higher • Are recognized as competent • Define possible actions that can be taken in response to each e-mail message • Form basis of multiple-choice response options • Competency benchmarks help to calibrate response options IPAC Annual Conference 2011 20
Step 4: Develop Performance Benchmarks SME Content Review and Benchmark Development • SMEs provide preliminary review of in-basket materials • Respond to in-basket in ―candidate mode‖ • Rate job-relatedness, technical accuracy, format, clarity • Suggest changes to improve fidelity • SMEs develop performance benchmarks • Focus SME responses with competency-based probes Specific issues (e-mail messages) Broader issues (overall in-basket) • Define performance levels Highly effective Marginally acceptable Ineffective IPAC Annual Conference 2011 21
- Page 1 and 2: In-Basket Design Developing and Imp
- Page 3 and 4: What is the In-Basket Assessment
- Page 5 and 6: The Steps • Step 1: Determine Cri
- Page 7 and 8: Step 1: Determine Critical Tasks, D
- Page 9 and 10: Step 1: Determine Critical Tasks, D
- Page 11 and 12: Step 2: Collect Source Materials
- Page 13 and 14: Step 2: Collect Source Materials Wo
- Page 15 and 16: Step 3: Develop Overview and E-mail
- Page 17 and 18: Step 4: Develop Performance Benchma
- Page 19: Step 4: Develop Performance Benchma
- Page 23 and 24: Step 5: Write Multiple-Choice Quest
- Page 25 and 26: Step 5: Write Multiple-Choice Quest
- Page 27 and 28: Step 5: Write Multiple-Choice Quest
- Page 29 and 30: Step 5: Write Multiple-Choice Quest
- Page 31 and 32: Step 5: Write Multiple-Choice Quest
- Page 33 and 34: Step 6: Confirm Fidelity and Job Re
- Page 35 and 36: Step 6: Confirm Fidelity and Job Re
- Page 37 and 38: Step 7: Assemble, Review, and Final
- Page 39 and 40: Step 8: Prepare for Administration
- Page 41 and 42: Step 9: Administer the In-Basket Ca
- Page 43 and 44: Step 10: Conduct Key Clearance Ques
- Page 45 and 46: References American Educational Res
- Page 47 and 48: Questions IPAC Annual Conference 20
- Page 49 and 50: Gast/Votraw -- In-Basket Design IPA
- Page 51 and 52: Gast -- In-Basket Design PTC-SC Ann
- Page 53: The ITEM as a whole should: It
Step 4: Develop Perform<strong>an</strong>ce Benchmarks<br />
SME Content Review <strong><strong>an</strong>d</strong> Benchmark Development<br />
• SMEs provide preliminary review of in-basket materials<br />
• Respond to in-basket in ―c<strong><strong>an</strong>d</strong>idate mode‖<br />
• Rate job-relatedness, technical accuracy, format, clarity<br />
• Suggest ch<strong>an</strong>ges to improve fidelity<br />
• SMEs develop perform<strong>an</strong>ce benchmarks<br />
• Focus SME responses with competency-based probes<br />
Specific issues (e-mail messages)<br />
Broader issues (overall in-basket)<br />
• Define perform<strong>an</strong>ce levels<br />
Highly effective<br />
Marginally acceptable<br />
<strong>In</strong>effective<br />
<strong>IPAC</strong> Annual Conference 2011<br />
21
Change language