How to Perform Laboratory Internal Audits
How to Perform Laboratory Internal Audits
How to Perform Laboratory Internal Audits
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
<strong>How</strong> <strong>to</strong> <strong>Perform</strong> Labora<strong>to</strong>ry <strong>Internal</strong> <strong>Audits</strong><br />
Scott D. Siders<br />
Division of Labora<strong>to</strong>ries<br />
Illinois EPA<br />
1
Training i Disclaimer<br />
i<br />
The material presented in this training session<br />
is for informational purposes p only. It is<br />
designed <strong>to</strong> promote understanding,<br />
consistency and clarification of internal<br />
auditing practices. It should not be<br />
considered a change or alteration of any<br />
accreditation standards, the published<br />
methods, a regula<strong>to</strong>ry agency requirement or<br />
the position of the Illinois EPA. The opinions<br />
expressed by the speaker are his own.<br />
2
Training Goals<br />
• At the completion of this training presentation<br />
you should have a basic understanding of:<br />
• what an internal audit is,<br />
• why one might perform an audit,<br />
• how <strong>to</strong> plan and conduct an audit,<br />
• the end results of the audit process, and<br />
• corrective actions.<br />
3
Training Goals (cont.)<br />
• By the end, you should be familiar with the<br />
steps in the auditing i process, including:<br />
• Establishing requirements against which <strong>to</strong> audit,<br />
• Assembling an audit team,<br />
• Developing relevant audit process/procedures,<br />
• Documenting the audit, and indentifying<br />
corrective actions and follow-up activities.<br />
4
Course Agenda<br />
• <strong>How</strong> <strong>to</strong> perform and internal audit<br />
• Introduction <strong>to</strong> internal auditing,<br />
• Planning an audit<br />
• <strong>Perform</strong>ing the audit<br />
• Assessment and documentation<br />
• Corrective actions<br />
• Follow-up, and<br />
• Closure<br />
5
Definitions (EL-V1M2-ISO-2009)<br />
• Assessment: The evaluation process used <strong>to</strong><br />
measure or establish the performance,<br />
effectiveness, and conformance of an<br />
organization and/or its systems <strong>to</strong> defined<br />
criteria (<strong>to</strong> the standards and requirements of<br />
labora<strong>to</strong>ry accreditation).<br />
6
Definitions (EL-V1M2-ISO-2009)<br />
• Audit: A systematic and independent<br />
examination of facilities, equipment,<br />
personnel, training, procedures, recordkeeping,<br />
data validation, data management,<br />
and reporting aspects of a system <strong>to</strong><br />
determine whether QA/QC and technical<br />
activities are being conducted as planned and<br />
whether these activities will effectively<br />
achieve quality objectives.<br />
7
TNI Standard – ISO 17025<br />
• V1M2 4.14.1 – The labora<strong>to</strong>ry shall<br />
periodically, and in accordance with a<br />
predetermined schedule and procedure,<br />
conduct internal audits of its activities <strong>to</strong><br />
verify its operations continue <strong>to</strong> comply with<br />
the requirements of the management system.<br />
• V1M2 4.14.1 – The internal audit program<br />
shall address all elements of the management<br />
system, including the testing and/or<br />
calibration activities.<br />
iti<br />
8
TNI Standard – ISO 17025<br />
• V1M2 4.14.1 – It is the responsibility of the<br />
quality manager <strong>to</strong> plan and organize audits<br />
as required by the schedule and requested by<br />
management.<br />
• V1M2 4.14.1 – Such audits shall be carried<br />
out by trained and qualified personnel e who<br />
are, wherever resources permit, independent<br />
of the activity <strong>to</strong> be audited.<br />
9
TNI Standard – ISO 17025<br />
• V1M2 4.14.2 – When audit findings cast<br />
doubt on the effectiveness of the operations<br />
or on the correctness or validity of the<br />
labora<strong>to</strong>ry’s test or calibration results, the<br />
labora<strong>to</strong>ry shall take timely corrective action,<br />
and shall notify cus<strong>to</strong>mers in writing if<br />
investigations show that the labora<strong>to</strong>ry results<br />
may have been affected.<br />
10
TNI Standard – ISO 17025<br />
• V1M2 4.14.3 4143 – The area of activity audited,<br />
the audit findings and corrective actions that<br />
arise from them shall be recorded.<br />
• V1M2 4.14.4 – Follow-up audit activities<br />
shall verify and record the implementation<br />
and effectiveness of the corrective action<br />
taken.<br />
11
TNI Standard – ISO 17025<br />
• V1M2 4.15.5 4155 a) – The labora<strong>to</strong>ry shall have<br />
a policy that specifies the time frame for<br />
notifying a client of events that cast doubt on<br />
the validity of the results.<br />
• V1M2 4.15.5 b) – The labora<strong>to</strong>ry<br />
management shall ensure that t these actions<br />
are discharged within the agreed time frame.<br />
12
TNI Standard – ISO 17025<br />
• V1M2 4.15.5 c) – The internal audit<br />
schedule shall be completed annually.<br />
13
Oh the troubles I have seen…<br />
• Significant issue for some labora<strong>to</strong>ries<br />
• Lack of senior management commitment<br />
• QA Manager wearing <strong>to</strong> many hats<br />
• Lack of internal audit process<br />
• <strong>Internal</strong> audit <strong>to</strong>o limited <strong>to</strong> encompass<br />
quality system issues<br />
• <strong>Internal</strong> audit findings not followed by<br />
corrective action process<br />
• Corrective actions not implemented<br />
14
Benefits<br />
• Identify issues and resolve them more quickly<br />
than bi-annual external on-site assessment<br />
• Demonstrates pro-active or continual<br />
improvement quality system approach<br />
• Enhance quality of reported data<br />
• Correct non-conformances before they<br />
become the cus<strong>to</strong>mers problem<br />
• Helps avoid loss of control (QA, QC or<br />
otherwise)<br />
• Reduces risks <strong>to</strong> organization<br />
15
Why Audit and When <strong>to</strong> Audit<br />
• Routine oversight as part of your established<br />
quality system<br />
• Prior <strong>to</strong> starting a project or “new work”<br />
• During the course of a project<br />
• High-visibility project that require extra<br />
attention<br />
• Audit for cause<br />
• Other activities have called practices in<strong>to</strong><br />
question<br />
• Overall failure <strong>to</strong> perform<br />
16
Basic Audit Concepts<br />
• Objective<br />
• Scope<br />
• Planning<br />
• On-site<br />
• Reporting<br />
• Corrective Action<br />
• Follow-up<br />
• Closure<br />
17
Planning an Audit - Steps<br />
• Planning the audit objectives<br />
• Planning the audit scope<br />
• Establishing the requirements against which<br />
<strong>to</strong> audit<br />
• Developing framework or technical approach<br />
• Reviewing materials and information<br />
• Selecting the audit team<br />
• Scheduling the audit<br />
• Issuing the audit plan/agenda<br />
18
Planning the Audit Objectives<br />
• Fulfill requirements of your established quality<br />
system<br />
• Opportunity for improvement<br />
• Assess specific non-conformance, data<br />
integrity it issue or cus<strong>to</strong>mer complaint<br />
• Prepare for the future<br />
• Find practical solutions (not <strong>to</strong> affix blame)<br />
• Compliance-based or system-based<br />
19
Planning the Audit Scope<br />
• Scope addresses focus of audit<br />
• Identify key or critical elements and activities<br />
• Are there any sensitive or confidential issues<br />
(proprietary, personnel, data integrity, etc.)<br />
20
Scope of the Audit?<br />
• Field sampling operations<br />
• Field testing operations<br />
• (e.g., pH, residual Cl)<br />
• Labora<strong>to</strong>ry operations<br />
• Entire labora<strong>to</strong>ry, or<br />
• Selected department(s)<br />
• Test methods SOPs (compliance with SOPs)<br />
• Review data (assess data quality)<br />
21
Audit Scope - what <strong>to</strong> include?<br />
• Standard operating procedures (SOPs)<br />
• PT Results/DMR QA<br />
• Document control<br />
• Reagent water<br />
• Control charting<br />
• Notebooks<br />
• Sample handling<br />
• Equipment maintenance<br />
• Standard d and reagent preparation<br />
22
Audit Scope - what <strong>to</strong> include?<br />
• Quality Manual<br />
• Data review<br />
• Holding times<br />
• Training<br />
• Records management<br />
• Quality control<br />
• Facilities<br />
• Subcontracting<br />
• MDLs<br />
23
Audit Scope – what type of audit?<br />
• Horizontal <strong>Audits</strong> follow a process from start<br />
<strong>to</strong> end. This type of audit would look at<br />
procedures as they support the process itself<br />
and is likely <strong>to</strong> span many different functions<br />
or departments.<br />
• Vertical <strong>Audits</strong> look, in depth, at a particular<br />
function or department. e t This type of audit<br />
would moni<strong>to</strong>r the use of all relevant<br />
procedures as they are used <strong>to</strong> support the<br />
function or activity.<br />
it<br />
24
Establishing the Requirements<br />
Against which <strong>to</strong> Audit<br />
• Is there a contract or client requirements?<br />
• Is there a NPDES or Pre-treatment permit?<br />
• Is there a standard (e.g., TNI)?<br />
• Is there a regula<strong>to</strong>ry requirement and/or<br />
mandated test methods (e.g., 40 CFR part 136)?<br />
• Is there a Quality Manual and/or QAPP?<br />
• Are there established policies and/or procedures<br />
(SOPs)?<br />
• Do any of these documents contain performance<br />
criteria?<br />
• No requirements, then it’s not an audit!<br />
25
Developing the Framework or<br />
Technical Approach<br />
• Planning audit data collection<br />
• Results-based information<br />
• Records<br />
• Reports<br />
• Data<br />
• <strong>Perform</strong>ance-based information<br />
• Real time observations<br />
• Interviews<br />
• <strong>Perform</strong>ance indica<strong>to</strong>rs (e.g., metrics)<br />
• Program-based information<br />
• Policies, procedures, plans (e.g., QAPP)<br />
26
Audit Materials and Resources<br />
• Get Prepared and allow time <strong>to</strong> prepare!<br />
• Obtain documents/records upfront and review<br />
prior <strong>to</strong> on-site<br />
• Quality manual, SOPs, QAPP, etc.<br />
• Permit conditions (required test methods,<br />
discharge limits)<br />
• PT and/or DMR QA performance<br />
• Major equipment list<br />
• Organization and staffing information<br />
• Previous audits and corrective actions<br />
27
Selecting the Audit Team<br />
• Senior management should decide who is in<br />
charge of the audit.<br />
• Preferably, it’s someone with QA<br />
responsibilities.<br />
• The subject matter experts may take the lead<br />
in asking questions during the audit, but the<br />
audit team leader is in charge overall.<br />
• Is more than one person needed?<br />
• <strong>How</strong> complex is the work being performed?<br />
• What disciplines are involved?<br />
• Who are the subject matter experts?<br />
28
Selecting the Audit Team (cont.)<br />
• Audi<strong>to</strong>rs need <strong>to</strong> understand the requirements<br />
and work being performed (i.e., knowledge of).<br />
• Others can participate in support roles.<br />
• Attributes of audi<strong>to</strong>rs:<br />
• Technical knowledge (whole team covers all<br />
aspects)<br />
• Auditing skills (particularly lead audi<strong>to</strong>r)<br />
• Independent and objective<br />
• Professionalism and sound judgment<br />
• Competence, integrity, fairness, use of due<br />
diligence, respect for confidentiality<br />
• Respectful and courteous <strong>to</strong>ward all<br />
29
Audi<strong>to</strong>rs Role<br />
• Understand quality system<br />
• Knowledge of expectations and requirements<br />
• Gather objective information<br />
• Report on observations<br />
• In some cases,<br />
• Provide recommendations<br />
• Conduct follow-up activities (verify)<br />
30
Scheduling <strong>Audits</strong><br />
• Routine audits generally can be more flexible in<br />
their scheduling.<br />
• Need <strong>to</strong> balance convenience of the organization<br />
being audited against the need <strong>to</strong> perform an<br />
audit.<br />
• Provide potential dates for the audit, but<br />
understand that not all staff may be present at<br />
that time, or that critical activities take place on<br />
schedules other than those convenient <strong>to</strong> the<br />
audi<strong>to</strong>rs (particularly an issue for field audits).<br />
31
Scheduling <strong>Audits</strong> (cont.)<br />
• <strong>Audits</strong> for cause should be scheduled quickly<br />
<strong>to</strong> avoid generating more questionable data.<br />
• Unannounced audits may be needed (e.g.,<br />
Section 308 of the Clean Water Act allows EPA<br />
<strong>to</strong> show up any time <strong>to</strong> audit a lab performing<br />
NPDES compliance moni<strong>to</strong>ring)<br />
32
Scheduling <strong>Audits</strong><br />
• When <strong>to</strong> audit: e.g., 200.8 by ICP-MS<br />
• Tk Take in<strong>to</strong> consideration:<br />
• Labora<strong>to</strong>ry workload<br />
• External audit schedule<br />
• Analyst(s) schedule<br />
• Audi<strong>to</strong>r’s schedule<br />
e<br />
• Other?<br />
33
Issuing an Audit Plan/Agenda<br />
• Depending on the purpose of the audit, this may be<br />
informal or formal.<br />
• Prepare and distribute appropriate documentation<br />
(communications, schedules, etc.).<br />
• Outline for formal audit plan:<br />
• Subject<br />
• Objective<br />
• Scope<br />
• Audit team members<br />
• Detailed schedule<br />
• Checklist(s)<br />
• Data collection responsibilities<br />
• Affected organizations and interfaces<br />
34
Quiz – An Audit Is:<br />
• A formal activity that requires forethought<br />
and planning<br />
• Quick and simple <strong>to</strong> perform<br />
• Citi Critical for every project<br />
• A good way <strong>to</strong> fix flawed environmental data<br />
• A good excuse for a road trip<br />
35
Answer #1 – An Audit Is:<br />
• A formal activity that requires forethought<br />
and planning<br />
Anything worth doing, is worth doing<br />
well, and that takes planning!<br />
36
Quiz #2 – To be a good audi<strong>to</strong>r<br />
you need <strong>to</strong> be:<br />
• An expert in all aspects of<br />
sampling and analysis<br />
• A QA expert<br />
• Sherlock Holmes<br />
• Observant, logical, able <strong>to</strong><br />
think on your feet, and<br />
impartial<br />
37
Answer #2 – To be a good<br />
audi<strong>to</strong>r, you need <strong>to</strong> be:<br />
• Observant, logical, able <strong>to</strong> think on your feet,<br />
and impartial<br />
Nobody knows everything and there is<br />
plenty of room for the “non-experts,”<br />
especially if they are willing <strong>to</strong> learn.<br />
38
Short Break<br />
See you back in 5 minutes! ☺<br />
39
<strong>Perform</strong>ing the Audit:<br />
On-site Steps<br />
• Conducting the opening meeting<br />
• Examining the overall system and processes<br />
• Documenting the various processes<br />
• Examining the data or products of the<br />
organization<br />
• On-site deliberations<br />
• Conducting the closing meeting<br />
40
Opening Meeting<br />
• Review objectives and scope<br />
• Review agenda and schedule<br />
• Review internal audit process<br />
• Process for identifying findings<br />
• Confirmation of information<br />
• Time for closing meeting<br />
41
Examining the Overall Process<br />
• Common approach is <strong>to</strong> follow the data<br />
generation process<br />
• “Treat me like a sample” in lab situations,<br />
ti<br />
starting with sample receipt.<br />
• <strong>Audits</strong> tend <strong>to</strong> follow the processes in a sample<br />
flow, data flow and/or chronological order.<br />
• Speak <strong>to</strong> everyone involved in each step, if you<br />
can.<br />
• For larger organizations, speak <strong>to</strong> someone at<br />
each step, and all of the supervisory staff.<br />
42
Documenting the Process<br />
• Checklists are a common approach, but not<br />
always ideal.<br />
• If there are clear-cut requirements, use those <strong>to</strong><br />
develop checklists during the planning phase.<br />
• Checklists can help assure completeness and<br />
logical observation format.<br />
• Don’t be bound by a checklist –ask yes/no and<br />
open-ended questions based on what you<br />
observe.<br />
• Don’t make checklists so detailed that you spend<br />
all your time trying <strong>to</strong> fill them out and rush<br />
through the process. They are a <strong>to</strong>ol.<br />
43
Tools: Checklists<br />
• In-house Management System Checklists<br />
• Method-specific Checklists<br />
• Regula<strong>to</strong>ry and/or Accreditation Checklists, for<br />
example:<br />
• http://www.deq.state.va.us/vpdes/checklist.html<br />
• http://www.cdph.ca.gov/services/boards/eltac/Pages/MethodAu<br />
ditChecklists.aspx<br />
• http://www.dnr.state.wi.us/org/es/science/lc/OUTREACH/Check<br />
lists.htm<br />
All are good for notes, recording observations,<br />
etc.<br />
44
Example Checklists<br />
45
Example Checklists (cont.)<br />
46
Example Checklists (cont.)<br />
47
Tools: Interviewing<br />
• Information gathering<br />
• Discussion<br />
• Build rapport<br />
• Interviewee does most of the talking<br />
• Attitude of interviewer<br />
48
Successful Interviews<br />
• Comfortable for Interviewee<br />
• Trust<br />
• Mutual Respect<br />
• Acceptance<br />
• Open atmosphere<br />
• Professionalism, and<br />
• Smile ☺<br />
49
Types of Questions<br />
• Yes or No<br />
• Short answer<br />
• Probing<br />
• Open-ended<br />
• What.., Why.., When..<br />
50
Documenting the Results<br />
• Complete the checklists or make notes as<br />
questions asked and if needed then dig<br />
deeper<br />
• Where responses are not clear or where<br />
observed procedures not in keeping with what<br />
the staff or supervisors say they do.<br />
• Take good notes and take time <strong>to</strong> record<br />
observations (don’t be rushed)<br />
• If a problem is found, s<strong>to</strong>p and try <strong>to</strong> find out<br />
why it occurs<br />
51
Examining Data or Products<br />
• Schedule time <strong>to</strong> review any data or records<br />
on-site (e.g., calibration, QC, data review)<br />
• Request space <strong>to</strong> spread out paper or products<br />
• Should be able <strong>to</strong> trace all results back <strong>to</strong> specific<br />
actions by the field or lab staff<br />
• Doing this on-site allows you <strong>to</strong> ask questions<br />
about the data, presentation of the results,<br />
exceptions you uncover, etc.<br />
• Take notes or ask <strong>to</strong> make copies of information<br />
for further review (off-site) or <strong>to</strong> support your<br />
findings<br />
52
Deliberations<br />
• Audit team should meet privately after reviewing<br />
all aspects of the process <strong>to</strong> discuss the situation.<br />
ti<br />
• On-site (e.g., conference room) or off-site<br />
• Even one-person audits require some time for<br />
deliberation<br />
• Decide who on the audit team will lead the closing<br />
meeting, and who will present any detailed<br />
findings, observations and any recommendations<br />
53
Closing Meeting<br />
• Review scope<br />
• Review follow-up schedule<br />
• Review internal audit process<br />
• Review findings<br />
• Positive (mention good and best<br />
practices observed)<br />
• Negative (non-conformances)<br />
• Address Questions and Concerns<br />
(but be prepared <strong>to</strong> say “let me<br />
think about that some more”)<br />
54
Closing Meeting (cont.)<br />
• Don’t forget <strong>to</strong>:<br />
• Listen <strong>to</strong> their responses and observe their<br />
reactions <strong>to</strong> any information you present.<br />
• Were they surprised?<br />
• Defensive?<br />
• Eager <strong>to</strong> address non-conformance?<br />
• Aware of the issues?<br />
• Do managers and staff react differently?<br />
• Do they appear <strong>to</strong> take your concerns seriously?<br />
• Do they disagree with a finding(s)?<br />
55
Evaluation and Documentation<br />
• As a team, finalize plans for whatever final<br />
report you need <strong>to</strong> provide.<br />
• Level of detail depends on the purpose of the<br />
audit and any subsequent use of the findings.<br />
• Compile everyone’s notes and checklists and<br />
any relevant information or data collected on-<br />
site.<br />
• Draft the report and circulate within the audit<br />
team and the auditing organization.<br />
56
Evaluation and Documentation<br />
• Minimum audit report content includes introduction<br />
(why the audit was conducted), list of audi<strong>to</strong>rs,<br />
findings requiring corrective action, requirement<br />
finding was made on, and any due date for completion<br />
of corrective actions.<br />
• Possibly include recommendations for improvement.<br />
• Address internal comments and prepare a draft for<br />
broader circulation.<br />
• Provide copy of the report <strong>to</strong> the audited organization.<br />
As appropriate, ask them for comments.<br />
• Goal is <strong>to</strong> allow them <strong>to</strong> correct any factual errors in<br />
the report.<br />
• Their comments are NOT part of any corrective<br />
actions.<br />
57
Quiz #3 – On-site <strong>Audits</strong><br />
• Follow a rigid standardized approach and<br />
schedule<br />
• Require that you develop a detailed<br />
checklist<br />
• Take months <strong>to</strong> plan and schedule, so why<br />
bother<br />
• Apply only <strong>to</strong> lab work<br />
58
Answer #3 – On-site <strong>Audits</strong><br />
None of these!!!<br />
• Planning is critical, but audits can happen in<br />
days or weeks when necessary<br />
• Audi<strong>to</strong>rs need <strong>to</strong> be flexible<br />
• Checklists are <strong>to</strong>ols, not the only way <strong>to</strong><br />
perform an audit<br />
59
Quiz #4 – The goal of an audit is <strong>to</strong>:<br />
• Find out what’s wrong and fix it<br />
• Make everyone do it the same way (preferably<br />
your way)<br />
• Examine the overall process of data<br />
generation, understand how it works,<br />
determine if it meets the needs and goals of<br />
the project, and perhaps identify areas for<br />
improvement<br />
60
Answer #4 – The goal of an audit is <strong>to</strong>:<br />
Examine the overall process of data<br />
generation, understand how it works,<br />
determine if it meets the needs of the<br />
client and goals of the labora<strong>to</strong>ry, and<br />
perhaps identify areas for<br />
improvement<br />
61
7 th Inning Stretch<br />
See you back in 5 minutes! ☺<br />
or<br />
62
Corrective Actions<br />
• Remember:<br />
• All audit findings and any corrective actions that<br />
arise from them shall be documented.<br />
• The internal audit does not s<strong>to</strong>p with the audit<br />
report!<br />
• Now the important work starts!<br />
63
TNI Standard – ISO 17025<br />
• V1M2 4.11.1 - The labora<strong>to</strong>ry shall<br />
establish a policy and a procedure and<br />
shall designate appropriate authorities for<br />
implementing corrective action when<br />
nonconforming work or departures from<br />
the policies and procedures in the<br />
management system or technical<br />
operations have been identified.<br />
64
TNI Standard – ISO 17025<br />
• V1M2 4.11.2 - The procedure for corrective<br />
action shall start with an investigation <strong>to</strong><br />
determine the root cause(s) of the problem.<br />
NOTE: Cause analysis is the key and sometimes most<br />
difficult part in the corrective action procedure. Often<br />
the root cause is not obvious and thus a careful<br />
analysis of all potential causes of the problem is<br />
required. Potential causes could include cus<strong>to</strong>mer<br />
requirements, the samples, sample specifications,<br />
methods and procedures, staff skill and training,<br />
consumables, or equipment and its calibration.<br />
65
TNI Standard – ISO 17025<br />
• V1M2 4.11.3 - Where corrective action is<br />
needed, the labora<strong>to</strong>ry shall identify potential<br />
corrective actions. It shall select and implement<br />
the action(s) most likely <strong>to</strong> eliminate the problem<br />
and <strong>to</strong> prevent recurrence.<br />
66
TNI Standard – ISO 17025<br />
• V1M2 4.11.3 - Corrective actions shall be <strong>to</strong> a<br />
degree appropriate <strong>to</strong> the magnitude and the risk<br />
of the problem.<br />
The labora<strong>to</strong>ry shall document and implement any<br />
required changes resulting from corrective action<br />
investigations.<br />
• V1M2 4.11.4 - The labora<strong>to</strong>ry shall moni<strong>to</strong>r the<br />
results <strong>to</strong> ensure that the corrective actions taken<br />
have been effective.<br />
67
Responsibility for Corrective Action<br />
• Establish responsibility on both sides for<br />
following up on the audit<br />
• Who in the audited organization is responsible<br />
for communicating and documenting the<br />
corrective actions <strong>to</strong> the audi<strong>to</strong>rs?<br />
• Who on the audit team is responsible for saying<br />
it’s fixed? (could be audit team leader or other<br />
team member)<br />
68
Corrective Action Steps<br />
• Identify problem, non-conformance, concern,<br />
etc.<br />
• Includes all internal audit findings<br />
• Investigate the problem<br />
• Determine the root cause(s)<br />
• Is it a systematic or random problem<br />
• Select and implement corrective action<br />
• Follow-up <strong>to</strong> ensure the problem is fixed<br />
• Evaluate process <strong>to</strong> ensure that the fix<br />
prevents recurrence of problem<br />
69
Corrective Action<br />
• Important 1 st Question: Were there any critical<br />
findings that affect the quality or validity of the<br />
data and/or test results reported <strong>to</strong> clients ?<br />
• If so, identify required corrective actions <strong>to</strong> be<br />
taken by the labora<strong>to</strong>ry.<br />
• If the corrective actions are based on specific<br />
requirements, identify those requirements.<br />
• If appropriate, call on the labora<strong>to</strong>ry <strong>to</strong> spell out the<br />
corrective actions that they will take.<br />
• Identify a timetable for taking corrective actions<br />
and the format for showing that they have taken<br />
place.<br />
70
Time Frame for Corrective Action<br />
• The labora<strong>to</strong>ry management shall ensure<br />
that these actions are discharged within a<br />
reasonable and agreed time frame as<br />
indicated in the quality manual and/or<br />
SOPs.<br />
• What is the agreed upon timeframe for taking<br />
actions?<br />
• Where is this found (QM, Corrective Action SOP<br />
and/or <strong>Internal</strong> Audit SOP)?<br />
71
Correction vs. Corrective Action<br />
• Correction - the quick fix (i.e., superficial fix)<br />
• Get it out the door<br />
• May cause other problems<br />
• Corrective Actions - the thoughtful fix<br />
• Correct the underlying cause<br />
• Do not cause other problems<br />
• Permanent fix <strong>to</strong> problem – does not reoccur<br />
72
Labora<strong>to</strong>ry Needs <strong>to</strong> Consider<br />
• Were there recommendations for improvements<br />
made by the assessor that should be<br />
implemented?<br />
• Be careful <strong>to</strong> distinguish these from required<br />
corrective actions.<br />
• “Do it better” does not mean “Do it my way.”<br />
• Both sides can learn from one another.<br />
73
Root Cause Analysis<br />
• Working Definition<br />
“Root Cause Analysis is determining what<br />
happened, how it happened and why it<br />
happened”<br />
74
Root Cause Analysis<br />
• Goal<br />
“The goal of root cause analysis is <strong>to</strong><br />
determine what can be done <strong>to</strong> prevent<br />
it from happening again”<br />
75
Characteristics of Root Causes<br />
• Root causes are specific underlying causes<br />
• Root causes are those that can reasonably be<br />
identified<br />
• Root causes are those management has control <strong>to</strong><br />
fix<br />
• Root causes are those for which effective<br />
recommendations for preventing occurrences can<br />
be generated<br />
Quoted from “Root Cause Analysis for Beginners”, Rooney and Vanden Heuvel, Quality Progress, July, 2004<br />
76
Process of Root Cause Analysis<br />
• Collect data<br />
• Determine causal fac<strong>to</strong>rs<br />
• Identify root causes<br />
• Generate recommendations and implement<br />
77
Root Cause Analysis<br />
• One approach <strong>to</strong> root cause analysis is <strong>to</strong>:<br />
“Asking Why Five Times”<br />
Example: ICP-MS instrument maintenance was not<br />
documented<br />
1. Why didn’t the analyst know the maintenance<br />
required documentation?<br />
Action – Read the SOP<br />
2. If the analyst didn’t have access <strong>to</strong> the SOP, why not?<br />
Action – Get access <strong>to</strong> SOP<br />
78
Root Cause Analysis (cont.)<br />
3. If the Metals Coordina<strong>to</strong>r was supposed <strong>to</strong><br />
update the SOP, why isn’t it complete?<br />
Action – Get the Coordina<strong>to</strong>r <strong>to</strong> finish the<br />
SOP update.<br />
4. If the Coordina<strong>to</strong>r is <strong>to</strong>o busy, Why is<br />
the Coordina<strong>to</strong>r <strong>to</strong>o busy?<br />
Action – Delegate duties<br />
79
Root Cause Analysis (cont.)<br />
5. If there is no one <strong>to</strong> delegate some of<br />
Coordina<strong>to</strong>r’s duties <strong>to</strong>, Why not?<br />
Action – Examine work load, review<br />
work requests.<br />
80
Benefits of Root Cause Analysis<br />
• Typical Results of Root Cause Analysis<br />
• Improved training procedure<br />
• Updated SOP<br />
• Evaluation of staffing levels and work load<br />
• Changes in procedures<br />
• Prevent a repeat deficiency<br />
81
Implementing Corrective Actions<br />
• Use your labora<strong>to</strong>ry’s current corrective action<br />
documentation procedures, or<br />
• Create a spreadsheet eet or other document <strong>to</strong> track<br />
progress<br />
• Finding/Deficiency<br />
• Assigned responsible person(s)/labora<strong>to</strong>ry<br />
department ideally mutually agreed <strong>to</strong>)<br />
• Corrective action <strong>to</strong> be taken<br />
• Implementation Date (ideally mutually agreed <strong>to</strong>)<br />
• Completion Date (ideally mutually agreed <strong>to</strong>)<br />
• Verification Date (i.e., moni<strong>to</strong>r the results)<br />
• Comments<br />
82
Tracking Corrective Actions<br />
83
Verifying Corrective Actions<br />
• Verify corrective actions for<br />
implementation and effectiveness.<br />
• This is one of the places that corrective<br />
action processes fail.<br />
• Verify implementation and effectiveness<br />
(suggest 30-45 days after completion date)<br />
• Moni<strong>to</strong>r over the next several months (old<br />
habits are hard <strong>to</strong> break)<br />
84
Closeout – “It ain’t over ‘til it’s over.”Y. Berra<br />
• The audit team is responsible for notifying all<br />
involved that:<br />
• An audit <strong>to</strong>ok place<br />
• The results or findings were reviewed<br />
• The resolution of those findings<br />
• The final conclusions:<br />
• It’s OK, or<br />
• Problems still remain that affect closure<br />
• The findings and report are filed with all other<br />
audit documentation, and maybe subject <strong>to</strong> an<br />
external audit.<br />
85
Remember<br />
<strong>Internal</strong> Audi<strong>to</strong>rs gather objective evidence<br />
of conformance <strong>to</strong> the quality system and<br />
other applicable requirements.<br />
86
Remember<br />
• Need <strong>to</strong> audit entire labora<strong>to</strong>ry annually<br />
• Audit must be organized and planned<br />
• <strong>Perform</strong>ed by trained and qualified personnel<br />
• Must remedy findings through the corrective<br />
action process<br />
• Notify clients if necessary due <strong>to</strong> any negative<br />
impact on reported data/results.<br />
87
Desired Outcomes<br />
of <strong>Internal</strong> <strong>Audits</strong><br />
• Management effectiveness<br />
• Sound scientific approach (good science)<br />
• Decision with documented rationale<br />
• Efficiency for labora<strong>to</strong>ry<br />
• Improvements <strong>to</strong> operations, organization<br />
and implementation<br />
• Reduce organizational risks and cost of noncompliance<br />
• Enhanced cus<strong>to</strong>mer satisfaction and loyalty<br />
88
Answer and Question Time<br />
Great opportunity <strong>to</strong> have<br />
an open discussion and<br />
learn from others in<br />
audience.<br />
First person <strong>to</strong> ask a<br />
question wins the door<br />
prize (i.e., they get <strong>to</strong> go out the door first for lunch)<br />
89
Thank you<br />
• Special thanks <strong>to</strong> the IWEA Labora<strong>to</strong>ry<br />
Committee for inviting me <strong>to</strong> speak.<br />
• Thank you <strong>to</strong> MWRD-GC for hosting us.<br />
• Wish <strong>to</strong> thank the following individuals for<br />
their contributions <strong>to</strong> this presentation:<br />
• Betsy Kent, RCID Environmental Services<br />
• Cathy Westerman, Virginia ELAP, and<br />
• Marlene Moore, Advanced Systems, Inc.<br />
• mmoore@advancedsys.com<br />
Contact me at: scott.siders@illinois.gov<br />
90