Accreditation Standards Self-Evaluation Tool - AAMC's member profile
Accreditation Standards Self-Evaluation Tool - AAMC's member profile
Accreditation Standards Self-Evaluation Tool - AAMC's member profile
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
LCME Database: Moving to a<br />
Pre-Populated Web Service—<br />
ASSET (<strong>Accreditation</strong> <strong>Standards</strong><br />
<strong>Self</strong>-<strong>Evaluation</strong> <strong>Tool</strong>)<br />
Robby Reynolds, AAMC (Moderator)<br />
Heather Lent, AAMC<br />
Terri Cameron, AAMC<br />
Latha Chandran, Stony Brook<br />
Julia Pet-Armacost, University of Central Florida
Session Overview<br />
•Welcome and Introductions (Robby Reynolds)<br />
•<strong>Accreditation</strong> Standard Challenges (Latha Chandran)<br />
•School Systems (Julia Pet-Armacost)<br />
•ASSET (Terri Cameron)<br />
•MedAPS (Robby Reynolds<br />
•Discussion (Heather Lent)<br />
2012 AAMC Annual Meeting
LCME <strong>Accreditation</strong> Today<br />
8 copies<br />
2012 AAMC Annual Meeting
Leveraging Technology To Address<br />
Challenging LCME <strong>Standards</strong><br />
Latha Chandran MD, MPH<br />
Vice Dean for UGME<br />
Professor of Pediatrics<br />
Stony Brook University School of Medicine<br />
2012 AAMC Annual Meeting
WHAT WE DID AT OUR SCHOOL<br />
• Process challenge: Many teams working with various data<br />
• sets, Process new data challenge: Many teams working<br />
• ALFRESCO with various - with central data control sets, new data<br />
• Finalization of document by hand<br />
• ALFRESCO - with central control<br />
• Homegrown System-CBase: Timeliness of grades,<br />
• mistreatments, Finalization search of document function using by MeSH hand terms, course<br />
evaluations, etc.<br />
• Homegrown System-CBase: Timeliness of<br />
grades, mistreatments, search function<br />
using MeSH terms, course evaluations, etc.<br />
2012 AAMC Annual Meeting
2012 AAMC Annual Meeting
2012 AAMC Annual Meeting
2012 AAMC Annual Meeting
2012 AAMC Annual Meeting
2012 AAMC Annual Meeting<br />
NEW REPORT FORMATS
2012 AAMC Annual Meeting<br />
MID-CLERKSHIP FEEDBACK- MULTISOURCE
2012 AAMC Annual Meeting<br />
ESTABLISHING BENCHMARKS
2012 AAMC Annual Meeting<br />
CLINICAL CONDITIONS ED2/8
COLOR CODED DASHBOARDS<br />
Survey Items:<br />
Themes Across<br />
Clerkships<br />
A B C D E F G<br />
Faculty observed me<br />
taking patient history<br />
Faculty observed me<br />
performing a physical<br />
exam<br />
N/A<br />
Residents and fellows<br />
provided effective<br />
teaching<br />
N/A<br />
Faculty provided<br />
effective teaching<br />
Understood how I<br />
would be evaluated<br />
Feedback received was<br />
useful<br />
2012 AAMC Annual Meeting
ED 8 SITE COMPARISONS<br />
Internal Medicine<br />
(2012)<br />
Site A (n=14) Site B (n=19) Site C (n=43) Site D (n=37)<br />
Faculty observed me<br />
taking patient history<br />
- + + -<br />
Faculty observed me a<br />
performing physical<br />
exam<br />
- + + +<br />
Residents and fellows<br />
provided effective<br />
teaching<br />
- + -- /<br />
Faculty provided<br />
effective teaching<br />
Understood how I would<br />
be evaluated<br />
Feedback received was<br />
useful<br />
++ ++ -- /<br />
++ ++ -- +<br />
-- ++ ++ +<br />
2012 AAMC Annual Meeting
ED 8 GRADE COMPARISONS<br />
Site<br />
Letter Grade (% of Students)<br />
Honors High Pass Pass<br />
Site A 22.9% 22.9% 51.9%<br />
Site B 29.9% 55.1% 14.2%<br />
Site C 12.7% 31.3% 49.3%<br />
Site D 32.6% 20.2% 41.9%<br />
Site E 31.3% 32.8% 32.0%<br />
Site F 34.7% 32.3% 29.1%<br />
Site G 31.5% 40.2% 26.8%<br />
2012 AAMC Annual Meeting
2012 AAMC Annual Meeting<br />
SEARCH FUNCTION FOR ED33
Innovative Systems to<br />
Support <strong>Accreditation</strong><br />
Julia Pet-Armacost, Ph.D.<br />
Associate Dean
April 2012<br />
State of the College of<br />
Medicine<br />
Deborah C. German, M.D.<br />
September 24, 2012<br />
Emerging “Medical City.”.<br />
Lake Nona site in 2012.
Systems UCF COM Uses to<br />
Support LCME <strong>Accreditation</strong><br />
• Development of database and ongoing<br />
compliance with standards: Compliance<br />
Assist from Campus Labs<br />
• Management of <strong>Self</strong>-study process and<br />
communication with the Survey Team<br />
Chair: Dropbox<br />
20
Compliance Assist<br />
21
Manage Entire Database Set<br />
22
Edit Narratives and Manage<br />
Appendices<br />
23
Print and View Narratives<br />
24
Archive <strong>Self</strong>-study Documents<br />
25
Manage Users and Track Progress<br />
26
<strong>Self</strong>-study Process and<br />
Communication with Survey Team<br />
• Share documents with people outside of<br />
the university domain<br />
• Access documents from anywhere at any<br />
time<br />
• Easy to locate<br />
• Log-in from anywhere<br />
27
Dropbox for <strong>Self</strong>-study Teams<br />
28
Dropbox for Survey Team Chair<br />
29
Share Documents<br />
30
ASSET: <strong>Accreditation</strong> Standard<br />
<strong>Self</strong>-<strong>Evaluation</strong> <strong>Tool</strong><br />
Terri Cameron<br />
Director of Curriculum Management<br />
AAMC
ASSET<br />
• Pre-populate accreditation documentation<br />
• Ensure consistent data reporting<br />
• Facilitate school-side accreditation<br />
processes<br />
• Foster qualitative data collection<br />
• Facilitate the work of LCME survey<br />
teams and staff<br />
www.aamc.org/medaps<br />
2012 AAMC Annual Meeting
ASSET<br />
• Began development process with Focus Groups for<br />
Curriculum Inventory in 2010<br />
• Vision from Focus Groups went beyond Curriculum<br />
Inventory<br />
• Develop tool for Continuous Quality Improvement<br />
• Include options for benchmarking against peer<br />
institutions using demographic data<br />
• Reduce number of requests for same information<br />
from schools<br />
• Pre-populate requests for information<br />
• Use data collected from schools to help with<br />
accreditation process<br />
2012 AAMC Annual Meeting
ASSET GOALS<br />
• Shorten timeline for data gathering<br />
• Remove need for each school to develop a<br />
sharepoint site or website to track progress and<br />
documents<br />
• Create a tool for constant monitoring of status of<br />
DCI Completion<br />
• Allow Users to tag questions for internal or external<br />
discussion<br />
• Provide for electronic assignment/monitoring of<br />
responsibilities<br />
• Provide a repository for appendices and other<br />
reference documents<br />
2012 AAMC Annual Meeting
ASSET GOALS<br />
• Remove the need for schools to track and/or sync<br />
information reported on various surveys and AAMC<br />
systems throughout the year, as well as during<br />
accreditation<br />
• Equip schools with a tool that supports and enhances<br />
Continuous Quality Improvement<br />
• Provide an online starting place for mission reviews,<br />
curriculum reviews, and mid-cycle and full-cycle<br />
accreditation documentation<br />
2012 AAMC Annual Meeting
ASSET<br />
<strong>Accreditation</strong><br />
<strong>Standards</strong><br />
<strong>Self</strong> -<strong>Evaluation</strong> <strong>Tool</strong><br />
Section: Institutional Settings<br />
Part: Background<br />
Standard: Revenue<br />
Question: d<br />
Complete the following table with data from the previous and current full<br />
surveys:<br />
Year of Survey: *2003-2004 **2011-2012<br />
Total Revenue from Tuition and Fees $8.0M $12.0M<br />
University and Government Appropriations $2.6M $3.4M<br />
Research/Training Grants, Direct $45.0M $34.0M<br />
Indirect Cost Recoveries $3.4M $5.3M<br />
Practice Plan Income $46.3M $59.2M<br />
Revenue from Clinical Affiliates $25.0M $35.5M<br />
Other Revenues $4.3M $2.5M<br />
Gifts and Endowment $2.1M $1.3M<br />
Total Revenues $136.7M $153.2M<br />
*Data Source:<br />
**Data Source:<br />
2003-2004 LCME Part I-A Annual Financial Questionnaire<br />
Administered by: AAMC | Medical School Contact: Emily Wilkerson<br />
2011-2012 LCME Part I-A Annual Financial Questionnaire<br />
Administered by: AAMC | Medical School Contact: Emily Wilkerson<br />
Provide narrative for survey team (optional):<br />
Navigation<br />
Institutional Settings<br />
Background<br />
Directory<br />
Changes in Personnel<br />
History<br />
Revenue<br />
Part A: Quantitative Indicators<br />
Vacant Department Chairs<br />
Students Enrolled<br />
Residents and Clinical Fellows<br />
Research Project Participation<br />
Service-Learning Project Participation<br />
Part B: Narrative Data and Tables<br />
IS-1 Planning<br />
IS-2 Not-for-Profit Status<br />
IS-3 Institutional <strong>Accreditation</strong><br />
IS-4 Organization<br />
IS-5 Governance<br />
IS-6 Board Members<br />
IS-7 Administrative Officers<br />
IS-8 Chief Official<br />
IS-9 Authority and Responsibility<br />
IS-10 Chief Official Qualifications<br />
IS-11 Mission Leadership<br />
IS-12 Student’s Learning Environment<br />
IS-13 Education Program Environment<br />
IS-14 Research and Scholarly Activities<br />
IS-15 Service-Learning Activities<br />
IS-16 Diversity<br />
MD Program<br />
Medical Students<br />
Faculty<br />
Educational Resources
LCME <strong>Accreditation</strong> Tomorrow<br />
www.aamc.org/medaps<br />
2012 AAMC Annual Meeting
MedAPS: Suite of Services<br />
Provide AAMC <strong>member</strong> medical schools with the<br />
tools necessary to assess, maintain and fulfill<br />
accreditation standards and promote continuous<br />
quality improvement.<br />
Curriculum Inventory & Reports<br />
(Replacing CurrMIT)<br />
ASSET<br />
(<strong>Accreditation</strong> <strong>Standards</strong> <strong>Self</strong>-<strong>Evaluation</strong> <strong>Tool</strong>)<br />
ASSET Dashboard<br />
www.aamc.org/medaps<br />
2012 AAMC Annual Meeting
Curriculum Inventory & Reports<br />
• Streamline and simplify curriculum data<br />
collection and exchange utilizing internationally<br />
adopted standards<br />
• Provide graphical<br />
interpretations of<br />
aggregate and historical<br />
curriculum-related data<br />
(includes LCME A/Q Part II data)<br />
• Serve as the source<br />
for benchmarking and<br />
educational research in<br />
medical education<br />
www.aamc.org/medaps<br />
2012 AAMC Annual Meeting
ASSET (<strong>Accreditation</strong> <strong>Standards</strong> <strong>Self</strong>-<strong>Evaluation</strong> <strong>Tool</strong>)<br />
• Pre-populate online accreditation documentation<br />
• Ensure consistent data reporting<br />
• Facilitate school-side accreditation<br />
processes<br />
• Foster qualitative data collection<br />
• Facilitate the work of LCME survey<br />
teams and staff<br />
www.aamc.org/medaps<br />
2012 AAMC Annual Meeting
ASSET Dashboard<br />
• Review performance on LCME standards<br />
annually<br />
• Compare performance<br />
and curricula with national<br />
data<br />
• Compare performance<br />
and curricula with peer<br />
institutions<br />
• Link to relevant AAMC tools<br />
and solutions to help schools address<br />
deficiencies<br />
www.aamc.org/medaps<br />
2012 AAMC Annual Meeting
ASSET Dashboard<br />
• Monitor compliance with LCME accreditation standards<br />
on a continuous basis<br />
• Link to relevant resources and<br />
solutions to address deficiencies<br />
• Compare performance with<br />
national data and peer institutions<br />
• Budget range<br />
• Gender and ethnicity ratios<br />
• Number of affiliated hospitals<br />
• Number of faculty, students, and/or residents<br />
• Public or private and/or academic health centers<br />
• Use Demographic Options to narrow peer group for<br />
curriculum content, research effort, clinical practice, etc.<br />
www.aamc.org/medaps<br />
2012 AAMC Annual Meeting
www.aamc.org/medaps<br />
2012 AAMC Annual Meeting
Populating MedAPS<br />
LCME AQ Part I-A<br />
Data Sources<br />
LCME AQ Part II<br />
ASSET<br />
(1/3 Pre-Populated)<br />
LCME AQ Part I-B<br />
Curriculum<br />
Inventory<br />
AAMC Data Warehouse<br />
Curriculum<br />
Inventory Reports<br />
Student Record System<br />
Graduation<br />
Questionnaire<br />
2012 AAMC Annual Meeting<br />
Faculty Database<br />
ASSET Dashboard
MedAPS Roadmap<br />
FY 2012<br />
(July 2011-June 2012)<br />
FY 2013<br />
(July 2012-June 2013)<br />
FY 2014<br />
(July 2013-June 2014)<br />
FY 2015<br />
(July 2014-June 2015)<br />
ASSET<br />
ASSET Development<br />
ASSET Business<br />
Analysis<br />
Data Mapping & Migration<br />
to Data Warehouse<br />
Identifying & Aligning Data Relevant to LCME <strong>Accreditation</strong><br />
Curriculum Inventory (CI) <strong>Standards</strong> Development,<br />
Business Analysis<br />
AAMC Data Warehouse<br />
Curriculum Inventory<br />
& Reports<br />
ASSET Dashboard<br />
2012 AAMC Annual Meeting<br />
ASSET Dashboard Development
Discussion<br />
• What are the challenges encountered in preparing for LCME<br />
accreditation survey visits?<br />
• What solutions have been implemented, and do those solutions<br />
integrate LCME processes into continuous quality improvement<br />
efforts at schools?<br />
• What are some best practices for leveraging existing technology with<br />
current accreditation processes to achieve quality improvement?<br />
• What are some of the most difficult ‘data’ questions to answer, and<br />
what are data sources to respond to those questions?<br />
• What is the general approach used to organize school accreditation<br />
process and what are the general technologies used to develop and<br />
monitor workflows?<br />
• What are some of the most tedious parts of the accreditation<br />
workflow, and what tools would make those tasks less tedious?<br />
2012 AAMC Annual Meeting