How to Perform Laboratory Internal Audits

How to Perform Laboratory Internal Audits 

Scott D. Siders 

Division of Laboratories 

Illinois EPA 

1

Training i Disclaimer 

i 

The material presented in this training session 

is for informational purposes p only. It is 

designed to promote understanding, 

consistency and clarification of internal 

auditing practices. It should not be 

considered a change or alteration of any 

accreditation standards, the published 

methods, a regulatory agency requirement or 

the position of the Illinois EPA. The opinions 

expressed by the speaker are his own. 

2

Training Goals 

• At the completion of this training presentation 

you should have a basic understanding of: 

• what an internal audit is, 

• why one might perform an audit, 

• how to plan and conduct an audit, 

• the end results of the audit process, and 

• corrective actions. 

3

Training Goals (cont.) 

• By the end, you should be familiar with the 

steps in the auditing i process, including: 

• Establishing requirements against which to audit, 

• Assembling an audit team, 

• Developing relevant audit process/procedures, 

• Documenting the audit, and indentifying 

corrective actions and follow-up activities. 

4

Course Agenda 

• How to perform and internal audit 

• Introduction to internal auditing, 

• Planning an audit 

• Performing the audit 

• Assessment and documentation 

• Corrective actions 

• Follow-up, and 

• Closure 

5

Definitions (EL-V1M2-ISO-2009) 

• Assessment: The evaluation process used to 

measure or establish the performance, 

effectiveness, and conformance of an 

organization and/or its systems to defined 

criteria (to the standards and requirements of 

laboratory accreditation). 

6

Definitions (EL-V1M2-ISO-2009) 

• Audit: A systematic and independent 

examination of facilities, equipment, 

personnel, training, procedures, recordkeeping, 

data validation, data management, 

and reporting aspects of a system to 

determine whether QA/QC and technical 

activities are being conducted as planned and 

whether these activities will effectively 

achieve quality objectives. 

7

TNI Standard – ISO 17025 

• V1M2 4.14.1 – The laboratory shall 

periodically, and in accordance with a 

predetermined schedule and procedure, 

conduct internal audits of its activities to 

verify its operations continue to comply with 

the requirements of the management system. 

• V1M2 4.14.1 – The internal audit program 

shall address all elements of the management 

system, including the testing and/or 

calibration activities. 

iti 

8


• V1M2 4.14.1 – It is the responsibility of the 

quality manager to plan and organize audits 

as required by the schedule and requested by 

management. 

• V1M2 4.14.1 – Such audits shall be carried 

out by trained and qualified personnel e who 

are, wherever resources permit, independent 

of the activity to be audited. 

9


• V1M2 4.14.2 – When audit findings cast 

doubt on the effectiveness of the operations 

or on the correctness or validity of the 

laboratory’s test or calibration results, the 

laboratory shall take timely corrective action, 

and shall notify customers in writing if 

investigations show that the laboratory results 

may have been affected. 

10


• V1M2 4.14.3 4143 – The area of activity audited, 

the audit findings and corrective actions that 

arise from them shall be recorded. 

• V1M2 4.14.4 – Follow-up audit activities 

shall verify and record the implementation 

and effectiveness of the corrective action 

taken. 

11


• V1M2 4.15.5 4155 a) – The laboratory shall have 

a policy that specifies the time frame for 

notifying a client of events that cast doubt on 

the validity of the results. 

• V1M2 4.15.5 b) – The laboratory 

management shall ensure that t these actions 

are discharged within the agreed time frame. 

12


• V1M2 4.15.5 c) – The internal audit 

schedule shall be completed annually. 

13

Oh the troubles I have seen… 

• Significant issue for some laboratories 

• Lack of senior management commitment 

• QA Manager wearing to many hats 

• Lack of internal audit process 

• Internal audit too limited to encompass 

quality system issues 

• Internal audit findings not followed by 

corrective action process 

• Corrective actions not implemented 

14

Benefits 

• Identify issues and resolve them more quickly 

than bi-annual external on-site assessment 

• Demonstrates pro-active or continual 

improvement quality system approach 

• Enhance quality of reported data 

• Correct non-conformances before they 

become the customers problem 

• Helps avoid loss of control (QA, QC or 

otherwise) 

• Reduces risks to organization 

15

Why Audit and When to Audit 

• Routine oversight as part of your established 

quality system 

• Prior to starting a project or “new work” 

• During the course of a project 

• High-visibility project that require extra 

attention 

• Audit for cause 

• Other activities have called practices into 

question 

• Overall failure to perform 

16

Basic Audit Concepts 

• Objective 

• Scope 

• Planning 

• On-site 

• Reporting 

• Corrective Action 

• Follow-up 

• Closure 

17

Planning an Audit - Steps 

• Planning the audit objectives 

• Planning the audit scope 

• Establishing the requirements against which 

to audit 

• Developing framework or technical approach 

• Reviewing materials and information 

• Selecting the audit team 

• Scheduling the audit 

• Issuing the audit plan/agenda 

18

Planning the Audit Objectives 

• Fulfill requirements of your established quality 

system 

• Opportunity for improvement 

• Assess specific non-conformance, data 

integrity it issue or customer complaint 

• Prepare for the future 

• Find practical solutions (not to affix blame) 

• Compliance-based or system-based 

19

Planning the Audit Scope 

• Scope addresses focus of audit 

• Identify key or critical elements and activities 

• Are there any sensitive or confidential issues 

(proprietary, personnel, data integrity, etc.) 

20

Scope of the Audit? 

• Field sampling operations 

• Field testing operations 

• (e.g., pH, residual Cl) 

• Laboratory operations 

• Entire laboratory, or 

• Selected department(s) 

• Test methods SOPs (compliance with SOPs) 

• Review data (assess data quality) 

21

Audit Scope - what to include? 

• Standard operating procedures (SOPs) 

• PT Results/DMR QA 

• Document control 

• Reagent water 

• Control charting 

• Notebooks 

• Sample handling 

• Equipment maintenance 

• Standard d and reagent preparation 

22

Audit Scope - what to include? 

• Quality Manual 

• Data review 

• Holding times 

• Training 

• Records management 

• Quality control 

• Facilities 

• Subcontracting 

• MDLs 

23

Audit Scope – what type of audit? 

• Horizontal Audits follow a process from start 

to end. This type of audit would look at 

procedures as they support the process itself 

and is likely to span many different functions 

or departments. 

• Vertical Audits look, in depth, at a particular 

function or department. e t This type of audit 

would monitor the use of all relevant 

procedures as they are used to support the 

function or activity. 

it 

24

Establishing the Requirements 

Against which to Audit 

• Is there a contract or client requirements? 

• Is there a NPDES or Pre-treatment permit? 

• Is there a standard (e.g., TNI)? 

• Is there a regulatory requirement and/or 

mandated test methods (e.g., 40 CFR part 136)? 

• Is there a Quality Manual and/or QAPP? 

• Are there established policies and/or procedures 

(SOPs)? 

• Do any of these documents contain performance 

criteria? 

• No requirements, then it’s not an audit! 

25

Developing the Framework or 

Technical Approach 

• Planning audit data collection 

• Results-based information 

• Records 

• Reports 

• Data 

• Performance-based information 

• Real time observations 

• Interviews 

• Performance indicators (e.g., metrics) 

• Program-based information 

• Policies, procedures, plans (e.g., QAPP) 

26

Audit Materials and Resources 

• Get Prepared and allow time to prepare! 

• Obtain documents/records upfront and review 

prior to on-site 

• Quality manual, SOPs, QAPP, etc. 

• Permit conditions (required test methods, 

discharge limits) 

• PT and/or DMR QA performance 

• Major equipment list 

• Organization and staffing information 

• Previous audits and corrective actions 

27

Selecting the Audit Team 

• Senior management should decide who is in 

charge of the audit. 

• Preferably, it’s someone with QA 

responsibilities. 

• The subject matter experts may take the lead 

in asking questions during the audit, but the 

audit team leader is in charge overall. 

• Is more than one person needed? 

• How complex is the work being performed? 

• What disciplines are involved? 

• Who are the subject matter experts? 

28

Selecting the Audit Team (cont.) 

• Auditors need to understand the requirements 

and work being performed (i.e., knowledge of). 

• Others can participate in support roles. 

• Attributes of auditors: 

• Technical knowledge (whole team covers all 

aspects) 

• Auditing skills (particularly lead auditor) 

• Independent and objective 

• Professionalism and sound judgment 

• Competence, integrity, fairness, use of due 

diligence, respect for confidentiality 

• Respectful and courteous toward all 

29

Auditors Role 

• Understand quality system 

• Knowledge of expectations and requirements 

• Gather objective information 

• Report on observations 

• In some cases, 

• Provide recommendations 

• Conduct follow-up activities (verify) 

30

Scheduling Audits 

• Routine audits generally can be more flexible in 

their scheduling. 

• Need to balance convenience of the organization 

being audited against the need to perform an 

audit. 

• Provide potential dates for the audit, but 

understand that not all staff may be present at 

that time, or that critical activities take place on 

schedules other than those convenient to the 

auditors (particularly an issue for field audits). 

31

Scheduling Audits (cont.) 

• Audits for cause should be scheduled quickly 

to avoid generating more questionable data. 

• Unannounced audits may be needed (e.g., 

Section 308 of the Clean Water Act allows EPA 

to show up any time to audit a lab performing 

NPDES compliance monitoring) 

32

Scheduling Audits 

• When to audit: e.g., 200.8 by ICP-MS 

• Tk Take into consideration: 

• Laboratory workload 

• External audit schedule 

• Analyst(s) schedule 

• Auditor’s schedule 

e 

• Other? 

33

Issuing an Audit Plan/Agenda 

• Depending on the purpose of the audit, this may be 

informal or formal. 

• Prepare and distribute appropriate documentation 

(communications, schedules, etc.). 

• Outline for formal audit plan: 

• Subject 

• Objective 

• Scope 

• Audit team members 

• Detailed schedule 

• Checklist(s) 

• Data collection responsibilities 

• Affected organizations and interfaces 

34

Quiz – An Audit Is: 

• A formal activity that requires forethought 

and planning 

• Quick and simple to perform 

• Citi Critical for every project 

• A good way to fix flawed environmental data 

• A good excuse for a road trip 

35

Answer #1 – An Audit Is: 

• A formal activity that requires forethought 

and planning 

Anything worth doing, is worth doing 

well, and that takes planning! 

36

Quiz #2 – To be a good auditor 

you need to be: 

• An expert in all aspects of 

sampling and analysis 

• A QA expert 

• Sherlock Holmes 

• Observant, logical, able to 

think on your feet, and 

impartial 

37

Answer #2 – To be a good 

auditor, you need to be: 

• Observant, logical, able to think on your feet, 

and impartial 

Nobody knows everything and there is 

plenty of room for the “non-experts,” 

especially if they are willing to learn. 

38

Short Break 

See you back in 5 minutes! ☺ 

39

Performing the Audit: 

On-site Steps 

• Conducting the opening meeting 

• Examining the overall system and processes 

• Documenting the various processes 

• Examining the data or products of the 

organization 

• On-site deliberations 

• Conducting the closing meeting 

40

Opening Meeting 

• Review objectives and scope 

• Review agenda and schedule 

• Review internal audit process 

• Process for identifying findings 

• Confirmation of information 

• Time for closing meeting 

41

Examining the Overall Process 

• Common approach is to follow the data 

generation process 

• “Treat me like a sample” in lab situations, 

ti 

starting with sample receipt. 

• Audits tend to follow the processes in a sample 

flow, data flow and/or chronological order. 

• Speak to everyone involved in each step, if you 

can. 

• For larger organizations, speak to someone at 

each step, and all of the supervisory staff. 

42

Documenting the Process 

• Checklists are a common approach, but not 

always ideal. 

• If there are clear-cut requirements, use those to 

develop checklists during the planning phase. 

• Checklists can help assure completeness and 

logical observation format. 

• Don’t be bound by a checklist –ask yes/no and 

open-ended questions based on what you 

observe. 

• Don’t make checklists so detailed that you spend 

all your time trying to fill them out and rush 

through the process. They are a tool. 

43

Tools: Checklists 

• In-house Management System Checklists 

• Method-specific Checklists 

• Regulatory and/or Accreditation Checklists, for 

example: 

• http://www.deq.state.va.us/vpdes/checklist.html 

• http://www.cdph.ca.gov/services/boards/eltac/Pages/MethodAu 

ditChecklists.aspx 

• http://www.dnr.state.wi.us/org/es/science/lc/OUTREACH/Check 

lists.htm 

All are good for notes, recording observations, 

etc. 

44

Example Checklists 

45

Example Checklists (cont.) 

46

Example Checklists (cont.) 

47

Tools: Interviewing 

• Information gathering 

• Discussion 

• Build rapport 

• Interviewee does most of the talking 

• Attitude of interviewer 

48

Successful Interviews 

• Comfortable for Interviewee 

• Trust 

• Mutual Respect 

• Acceptance 

• Open atmosphere 

• Professionalism, and 

• Smile ☺ 

49

Types of Questions 

• Yes or No 

• Short answer 

• Probing 

• Open-ended 

• What.., Why.., When.. 

50

Documenting the Results 

• Complete the checklists or make notes as 

questions asked and if needed then dig 

deeper 

• Where responses are not clear or where 

observed procedures not in keeping with what 

the staff or supervisors say they do. 

• Take good notes and take time to record 

observations (don’t be rushed) 

• If a problem is found, stop and try to find out 

why it occurs 

51

Examining Data or Products 

• Schedule time to review any data or records 

on-site (e.g., calibration, QC, data review) 

• Request space to spread out paper or products 

• Should be able to trace all results back to specific 

actions by the field or lab staff 

• Doing this on-site allows you to ask questions 

about the data, presentation of the results, 

exceptions you uncover, etc. 

• Take notes or ask to make copies of information 

for further review (off-site) or to support your 

findings 

52

Deliberations 

• Audit team should meet privately after reviewing 

all aspects of the process to discuss the situation. 

ti 

• On-site (e.g., conference room) or off-site 

• Even one-person audits require some time for 

deliberation 

• Decide who on the audit team will lead the closing 

meeting, and who will present any detailed 

findings, observations and any recommendations 

53

Closing Meeting 

• Review scope 

• Review follow-up schedule 

• Review internal audit process 

• Review findings 

• Positive (mention good and best 

practices observed) 

• Negative (non-conformances) 

• Address Questions and Concerns 

(but be prepared to say “let me 

think about that some more”) 

54

Closing Meeting (cont.) 

• Don’t forget to: 

• Listen to their responses and observe their 

reactions to any information you present. 

• Were they surprised? 

• Defensive? 

• Eager to address non-conformance? 

• Aware of the issues? 

• Do managers and staff react differently? 

• Do they appear to take your concerns seriously? 

• Do they disagree with a finding(s)? 

55

Evaluation and Documentation 

• As a team, finalize plans for whatever final 

report you need to provide. 

• Level of detail depends on the purpose of the 

audit and any subsequent use of the findings. 

• Compile everyone’s notes and checklists and 

any relevant information or data collected on- 

site. 

• Draft the report and circulate within the audit 

team and the auditing organization. 

56

Evaluation and Documentation 

• Minimum audit report content includes introduction 

(why the audit was conducted), list of auditors, 

findings requiring corrective action, requirement 

finding was made on, and any due date for completion 

of corrective actions. 

• Possibly include recommendations for improvement. 

• Address internal comments and prepare a draft for 

broader circulation. 

• Provide copy of the report to the audited organization. 

As appropriate, ask them for comments. 

• Goal is to allow them to correct any factual errors in 

the report. 

• Their comments are NOT part of any corrective 

actions. 

57

Quiz #3 – On-site Audits 

• Follow a rigid standardized approach and 

schedule 

• Require that you develop a detailed 

checklist 

• Take months to plan and schedule, so why 

bother 

• Apply only to lab work 

58

Answer #3 – On-site Audits 

None of these!!! 

• Planning is critical, but audits can happen in 

days or weeks when necessary 

• Auditors need to be flexible 

• Checklists are tools, not the only way to 

perform an audit 

59

Quiz #4 – The goal of an audit is to: 

• Find out what’s wrong and fix it 

• Make everyone do it the same way (preferably 

your way) 

• Examine the overall process of data 

generation, understand how it works, 

determine if it meets the needs and goals of 

the project, and perhaps identify areas for 

improvement 

60

Answer #4 – The goal of an audit is to: 

Examine the overall process of data 

generation, understand how it works, 

determine if it meets the needs of the 

client and goals of the laboratory, and 

perhaps identify areas for 

improvement 

61

7 th Inning Stretch 

See you back in 5 minutes! ☺ 

or 

62

Corrective Actions 

• Remember: 

• All audit findings and any corrective actions that 

arise from them shall be documented. 

• The internal audit does not stop with the audit 

report! 

• Now the important work starts! 

63


• V1M2 4.11.1 - The laboratory shall 

establish a policy and a procedure and 

shall designate appropriate authorities for 

implementing corrective action when 

nonconforming work or departures from 

the policies and procedures in the 

management system or technical 

operations have been identified. 

64


• V1M2 4.11.2 - The procedure for corrective 

action shall start with an investigation to 

determine the root cause(s) of the problem. 

NOTE: Cause analysis is the key and sometimes most 

difficult part in the corrective action procedure. Often 

the root cause is not obvious and thus a careful 

analysis of all potential causes of the problem is 

required. Potential causes could include customer 

requirements, the samples, sample specifications, 

methods and procedures, staff skill and training, 

consumables, or equipment and its calibration. 

65


• V1M2 4.11.3 - Where corrective action is 

needed, the laboratory shall identify potential 

corrective actions. It shall select and implement 

the action(s) most likely to eliminate the problem 

and to prevent recurrence. 

66


• V1M2 4.11.3 - Corrective actions shall be to a 

degree appropriate to the magnitude and the risk 

of the problem. 

The laboratory shall document and implement any 

required changes resulting from corrective action 

investigations. 

• V1M2 4.11.4 - The laboratory shall monitor the 

results to ensure that the corrective actions taken 

have been effective. 

67

Responsibility for Corrective Action 

• Establish responsibility on both sides for 

following up on the audit 

• Who in the audited organization is responsible 

for communicating and documenting the 

corrective actions to the auditors? 

• Who on the audit team is responsible for saying 

it’s fixed? (could be audit team leader or other 

team member) 

68

Corrective Action Steps 

• Identify problem, non-conformance, concern, 

etc. 

• Includes all internal audit findings 

• Investigate the problem 

• Determine the root cause(s) 

• Is it a systematic or random problem 

• Select and implement corrective action 

• Follow-up to ensure the problem is fixed 

• Evaluate process to ensure that the fix 

prevents recurrence of problem 

69

Corrective Action 

• Important 1 st Question: Were there any critical 

findings that affect the quality or validity of the 

data and/or test results reported to clients ? 

• If so, identify required corrective actions to be 

taken by the laboratory. 

• If the corrective actions are based on specific 

requirements, identify those requirements. 

• If appropriate, call on the laboratory to spell out the 

corrective actions that they will take. 

• Identify a timetable for taking corrective actions 

and the format for showing that they have taken 

place. 

70

Time Frame for Corrective Action 

• The laboratory management shall ensure 

that these actions are discharged within a 

reasonable and agreed time frame as 

indicated in the quality manual and/or 

SOPs. 

• What is the agreed upon timeframe for taking 

actions? 

• Where is this found (QM, Corrective Action SOP 

and/or Internal Audit SOP)? 

71

Correction vs. Corrective Action 

• Correction - the quick fix (i.e., superficial fix) 

• Get it out the door 

• May cause other problems 

• Corrective Actions - the thoughtful fix 

• Correct the underlying cause 

• Do not cause other problems 

• Permanent fix to problem – does not reoccur 

72

Laboratory Needs to Consider 

• Were there recommendations for improvements 

made by the assessor that should be 

implemented? 

• Be careful to distinguish these from required 

corrective actions. 

• “Do it better” does not mean “Do it my way.” 

• Both sides can learn from one another. 

73

Root Cause Analysis 

• Working Definition 

“Root Cause Analysis is determining what 

happened, how it happened and why it 

happened” 

74


• Goal 

“The goal of root cause analysis is to 

determine what can be done to prevent 

it from happening again” 

75

Characteristics of Root Causes 

• Root causes are specific underlying causes 

• Root causes are those that can reasonably be 

identified 

• Root causes are those management has control to 

fix 

• Root causes are those for which effective 

recommendations for preventing occurrences can 

be generated 

Quoted from “Root Cause Analysis for Beginners”, Rooney and Vanden Heuvel, Quality Progress, July, 2004 

76

Process of Root Cause Analysis 

• Collect data 

• Determine causal factors 

• Identify root causes 

• Generate recommendations and implement 

77


• One approach to root cause analysis is to: 

“Asking Why Five Times” 

Example: ICP-MS instrument maintenance was not 

documented 

1. Why didn’t the analyst know the maintenance 

required documentation? 

Action – Read the SOP 

2. If the analyst didn’t have access to the SOP, why not? 

Action – Get access to SOP 

78

Root Cause Analysis (cont.) 

3. If the Metals Coordinator was supposed to 

update the SOP, why isn’t it complete? 

Action – Get the Coordinator to finish the 

SOP update. 

4. If the Coordinator is too busy, Why is 

the Coordinator too busy? 

Action – Delegate duties 

79

Root Cause Analysis (cont.) 

5. If there is no one to delegate some of 

Coordinator’s duties to, Why not? 

Action – Examine work load, review 

work requests. 

80

Benefits of Root Cause Analysis 

• Typical Results of Root Cause Analysis 

• Improved training procedure 

• Updated SOP 

• Evaluation of staffing levels and work load 

• Changes in procedures 

• Prevent a repeat deficiency 

81

Implementing Corrective Actions 

• Use your laboratory’s current corrective action 

documentation procedures, or 

• Create a spreadsheet eet or other document to track 

progress 

• Finding/Deficiency 

• Assigned responsible person(s)/laboratory 

department ideally mutually agreed to) 

• Corrective action to be taken 

• Implementation Date (ideally mutually agreed to) 

• Completion Date (ideally mutually agreed to) 

• Verification Date (i.e., monitor the results) 

• Comments 

82

Tracking Corrective Actions 

83

Verifying Corrective Actions 

• Verify corrective actions for 

implementation and effectiveness. 

• This is one of the places that corrective 

action processes fail. 

• Verify implementation and effectiveness 

(suggest 30-45 days after completion date) 

• Monitor over the next several months (old 

habits are hard to break) 

84

Closeout – “It ain’t over ‘til it’s over.”Y. Berra 

• The audit team is responsible for notifying all 

involved that: 

• An audit took place 

• The results or findings were reviewed 

• The resolution of those findings 

• The final conclusions: 

• It’s OK, or 

• Problems still remain that affect closure 

• The findings and report are filed with all other 

audit documentation, and maybe subject to an 

external audit. 

85

Remember 

Internal Auditors gather objective evidence 

of conformance to the quality system and 

other applicable requirements. 

86

Remember 

• Need to audit entire laboratory annually 

• Audit must be organized and planned 

• Performed by trained and qualified personnel 

• Must remedy findings through the corrective 

action process 

• Notify clients if necessary due to any negative 

impact on reported data/results. 

87

Desired Outcomes 

of Internal Audits 

• Management effectiveness 

• Sound scientific approach (good science) 

• Decision with documented rationale 

• Efficiency for laboratory 

• Improvements to operations, organization 

and implementation 

• Reduce organizational risks and cost of noncompliance 

• Enhanced customer satisfaction and loyalty 

88

Answer and Question Time 

Great opportunity to have 

an open discussion and 

learn from others in 

audience. 

First person to ask a 

question wins the door 

prize (i.e., they get to go out the door first for lunch) 

89

Thank you 

• Special thanks to the IWEA Laboratory 

Committee for inviting me to speak. 

• Thank you to MWRD-GC for hosting us. 

• Wish to thank the following individuals for 

their contributions to this presentation: 

• Betsy Kent, RCID Environmental Services 

• Cathy Westerman, Virginia ELAP, and 

• Marlene Moore, Advanced Systems, Inc. 

• mmoore@advancedsys.com 

Contact me at: scott.siders@illinois.gov 

90

How to Perform Laboratory Internal Audits

Create successful ePaper yourself

Delete template?

Save as template?