WECC Audits of Registered Entities

John McGheeDirector of Compliance Audits andInvestigations360-567-4060jmcghee@wecc.bizCompliance with the NERC Reliability StandardsWECC Audits of Registered EntitiesJune 14 th , 2011John McGhee, WECC Director of Compliance Audits and InvestigationsPlease note: This paper is intended to provide an overview of the Compliance Auditprocess, beginning with the scheduling of an audit through the completion of the auditreport and does not include the Enforcement process.What is an audit?The Compliance Audit is one of the eight Compliance Monitoring Processes and is asystematic, objective review and examination of records and activities to determinewhether a Registered Entity meets the requirements of applicable Reliability Standards.It is important to keep in mind:• Full Compliance is only Full Compliance if it is documented• Documentation is the responsibility of the Registered Entity• The Registered Entity must be able to produce evidence at the audit thatproves complianceThe auditor will determine whether evidence presented is sufficient and appropriate toprovide a reasonable basis for the findings and conclusions, within the context of theaudit objectives. The auditor will also determine whether the evidence satisfies the auditobjectives by assessing the relevance, validity, and reliability of the documentationpresented.It is the responsibility of the entity to provide evidence of compliance with all applicablereliability standards. In most cases, the evidence will be written plans, programs orrecords of performance in accordance with the plans and procedures.Each audit team has an assigned Audit Team Lead (“ATL”), who essentially acts as theproject manager and is responsible for completion of the audit deliverables.W E S T E R N E L E C T R I C I T Y C O O R D I N A T I N G C O U N C I L • W W W . W E C C . B I Z1 5 5 N O R T H 4 0 0 W E S T • S U I T E 2 0 0 • S A L T L A K E C I T Y • U T A H • 8 4 1 0 3 - 1 1 1 4 • P H 8 0 1 . 5 8 2 . 0 3 5 3 • F X 8 0 1 . 5 8 2 . 3 9 1 8

Page - 2When and where will my audit take place?WECC will post the audit schedule for the next calendar year in July of the current year.For example, the audit schedule for 2012 will be posted by August 1, 2011. The WECCaudit schedule can be found: WECC Audit ScheduleAudits are conducted from the WECC offices in Vancouver, Washington and Salt LakeCity, Utah. Audits of Balancing Authorities (“BA”) and Transmission Operators (“TOP”)will always include an on-site component in which the auditors conduct part of the auditat the entity’s location. Generator Owners (“GO”) and Generator Operators (“GOP”) whohave Critical Cyber Assets (“CCA”) identified, per CIP-002 R3, will also be subject to anon-site component. An on-site visit by the audit team will not be required for entities nothaving the BA and/or TOP functions and/or identified CCAs. The audits that include anon-site component are typically referred to as “On-site Audits,” whereas those that areconducted solely from the WECC offices have been termed “Off-site Audits.”What is the frequency of the audits?All entities registered as a BA and/or a TOP will be audited on a three year cycle.Entities whose registration does not include the BA or TOP functions will be audited atleast every six years.When will I be officially notified of my audit?WECC will provide the Notice of Audit 90 days prior to the audit start date. This noticewill include the following information:Dates the Audit will begin and endNames of Auditors, Compliance Program Coordinator (“CPC”) responsible fordocumentation, Observers from WECC, NERC or FERCAudit ScopeIdentification of documents WECC utilizes to conduct the auditsOpening Presentation expectationsTable of Important Due Dateso Pre-Audit Survey (60 days prior to audit)o Pre-Audit Conference Call Date (30 days prior to audit)o All requested evidence, RSAWs, Program and Procedure Documents (20days prior to audit)o Objections to Audit Team Members (15 days prior to audit)W E S T E R N E L E C T R I C I T Y C O O R D I N A T I N G C O U N C I L • W W W . W E C C . B I Z1 5 5 N O R T H 4 0 0 W E S T • S U I T E 2 0 0 • S A L T L A K E C I T Y • U T A H • 8 4 1 0 3 - 1 1 1 4 • P H 8 0 1 . 5 8 2 . 0 3 5 3 • F X 8 0 1 . 5 8 2 . 3 9 1 8

Page - 4auditor’s participation is the responsibility of the Director of Compliance Audits andInvestigations.What do the observers do and why are they coming to the audit?Observers may include NERC staff, FERC staff and WECC staff. Observers aretypically in attendance to survey the WECC auditors’ work performance. There aretimes when an observer is attending as an on-the-job training type of exercise. In thesecases, they work closely with the audit team members. The observers do not make anydecisions on findings, as that is the responsibility of the audit team in conjunction withthe Audit Team Lead, who is always a WECC Compliance staff member. The ATL isresponsible for managing the participation of the observers.What is the Audit Scope?The audit will include, at a minimum, all of the reliability standards that are identified inthe NERC and WECC Compliance Monitoring and Enforcement ProgramImplementation Plan (“CMEP IP”) that are applicable to the registered functionsidentified in the Notice of Compliance Audit. The annual Actively Monitored ReliabilityStandards List identifies those standards and requirements that are mandatory to beaudited. See http://www.nerc.com/commondocs.php?cd=3 for a list of the ActivelyMonitored Standards. For the most current information regarding the applicability of theReliability Standards to registered functions, please refer to the NERC website NERCReliability Standards. WECC will also review the status of mitigation plans associatedwith any Open Enforcement Actions. The audit team may expand the scope of the auditto include additional reliability standards, as necessary.What is the purpose of the Pre-Audit Conference Call mentioned in the Notice of Audit?During the Pre-Audit Conference Call, the ATL will discuss the audit process, agenda,and answer any questions the entity may have regarding their approaching audit. Therewill be one call held for Off-site Audits that are scheduled to occur within the sameweek. This call typically takes about 15 to 20 minutes and is general in nature. Pre-Auditcalls for On-site Audits will include only the entity scheduled for that audit. This call willalso address the audit process, agenda, and questions, as well as logistics and anyOpen Enforcement Actions.W E S T E R N E L E C T R I C I T Y C O O R D I N A T I N G C O U N C I L • W W W . W E C C . B I Z1 5 5 N O R T H 4 0 0 W E S T • S U I T E 2 0 0 • S A L T L A K E C I T Y • U T A H • 8 4 1 0 3 - 1 1 1 4 • P H 8 0 1 . 5 8 2 . 0 3 5 3 • F X 8 0 1 . 5 8 2 . 3 9 1 8

Page - 5What if I have specific questions about the audit?The ATL will address any questions or concerns you have at any time following yourreceipt of the Notice of Audit. You do not need to wait for the Pre-Audit Conference Callto ask technical questions. We do encourage you to ask any process questions, such aslogistics, during the Pre-Audit Call.How many auditors and support staff can I expect at the audit?That depends on the scope of the audit.If you are registered as a BA and/or a TOP, there will be a minimum of 14 auditors andone CPC participating in the audit. Currently, all of the auditors will come to the entity’slocation to conduct part of the audit.If you are not registered as a BA and/or a TOP, your audit will not typically include anon-site visit by the audit team and, depending on the number and type of registrationsthat are applicable to your entity, you may expect two to four auditors and one CPC tobe assigned to the audit.How long will the audit take?Audits of large entities registered for several functions, including the BA and TOPfunctions, will be scheduled to take place for eight to ten working days over the courseof two consecutive weeks. During the first week, the auditors will work at the WECCoffices. The second week will be at the entity location. We typically travel to the auditsite on Monday morning and begin the audit Monday afternoon, with an expectation tocomplete the audit by noon on Friday of that week.Audits of smaller entities not registered for the BA and TOP functions are scheduled forone to four days, depending on the type and number of functional registrationsapplicable to the entity. For example, the audit of a Generator Owner can beaccomplished in one day while the audit of an entity registered as aGO/TO/GOP/DP/LSE/TO would take four days.Entities having Critical Assets and identified Critical Cyber Assets per CIP-002 willalways have an on-site visit by a team of auditors who will examine a sample set of theirCritical Assets. This will be identified in the Notice of Compliance Audit.What can I expect logistically for the on-site portion of an audit?W E S T E R N E L E C T R I C I T Y C O O R D I N A T I N G C O U N C I L • W W W . W E C C . B I Z1 5 5 N O R T H 4 0 0 W E S T • S U I T E 2 0 0 • S A L T L A K E C I T Y • U T A H • 8 4 1 0 3 - 1 1 1 4 • P H 8 0 1 . 5 8 2 . 0 3 5 3 • F X 8 0 1 . 5 8 2 . 3 9 1 8

Page - 6The audit team will arrive at your location at approximately 1:30 pm on Monday. TheATL and CPC will work with you prior to arriving to determine the size and number ofconference rooms needed by the audit team. Typically, one room will be used as anaudit room so it should be of sufficient size to accommodate at least 15 people, but mayneed to be large enough for up to 20, depending on the number of observers attending.The following are some logistical issues that are common:Lack of sufficient power receptacles. Each auditor and observer will need tworeceptacle spaces, which increases the need for plenty of power strips.Chairs designed for interrogation rooms; please supply chairs that you would becomfortable in for long hours.Insufficient table top space; each auditor should have at least thirty inches oftable top space.Please respect the privacy of the auditors by knocking before entering the roomwhere the auditors are working. There will be signs posted requesting thiscourtesy.The ATL will inform you of any additional requests/requirements. The CPC will workwith you on the logistics for lunch and other details.The auditors will work while eating lunch and they may work longer than eight hoursduring the day. They will usually begin work at 8:00 am and end at 5:00 pm, but mayneed to start earlier and work later. The ATL will inform the compliance contact if theaudit team plans to alter the normal 8-5 schedule.I’m a smaller entity, registered as a DP/LSE, so what are the logistics for my audit?Your audit will be scheduled for the WECC offices in either Vancouver, Washington orSalt Lake City, Utah. You are invited to attend your audit and, while it isn’t required, wedo encourage you to attend. While you are at the WECC offices, you will be providedwith a private space and internet access so you can work. The ATL will keep youinformed of progress during each day. Whether you choose to attend or not, you willneed to ensure that your Subject Matter Experts (“SMEs”) are available at all timesduring the audit. The auditors will contact the Compliance Contact to arrange for phoneinterviews of the SMEs.How should I submit my evidence?W E S T E R N E L E C T R I C I T Y C O O R D I N A T I N G C O U N C I L • W W W . W E C C . B I Z1 5 5 N O R T H 4 0 0 W E S T • S U I T E 2 0 0 • S A L T L A K E C I T Y • U T A H • 8 4 1 0 3 - 1 1 1 4 • P H 8 0 1 . 5 8 2 . 0 3 5 3 • F X 8 0 1 . 5 8 2 . 3 9 1 8

Page - 7All evidence should be submitted in electronic format and uploaded on the WECCCompliance EFT Server.What if the auditors have questions about my evidence relating to compliance?The auditors will review the RSAWs and accompanying evidence you submit. If theycannot determine that they have sufficient confirmation that there is No Finding, or theybelieve there is a Possible Violation, they will send you a Data Request for additionalinformation. A Data Request (“DR”) is a formal, documented request for additionalinformation. You will be given a reasonable amount of time to respond to the DR, butkeep in mind that during Off-site Audits, the response to the DR will likely be due thesame day. If you have any concern with meeting the deadline, you should immediatelycontact the CPC or the ATL to discuss other arrangements.Will there always be Data Requests?Yes, because some Requirements are audited by sampling rather than looking at theentire population, DRs need to be sent after the initial review of the evidence. Anexample of this is the PRC-005 protection system. We do not look at the maintenancerecords for every single relay, communication system, battery, current/voltage sensingdevice, or DC circuit associated with the protection system. Instead, we use NERCSampling Methodology Guidelines and Criteria to determine an appropriate samplepopulation. The auditors most often use a program called RAT-STATS, which is astatistical tool used by the U.S. Department of Health and Human Services Office ofAudit Services and developed by the Regional Advanced Techniques Staff (“RATS”) inSan Francisco. The statistical software tool assists the auditors in selecting randomsamples. A primary population of substations or generating stations would be selectedand, from those, a dependent population of elements, such as relays will be selected. Asingle substation may contain over 100 relays from which 23 or more samples will beselected randomly using RAT-STATS. Once the sample set is selected, the auditor willsubmit a DR for the maintenance records for those applicable elements.When will I be advised of Audit Findings?The ATL will keep you informed of the status of the audit each day. Any potential issueswill be communicated to you during the day. This will give you an opportunity to workwith your SMEs to determine if there is evidence that wasn’t originally presented or ifthere may be an alternate method of proving compliance. The ATL will inform thecompliance contact of any Possible Violations discovered during the audit. Any PossibleW E S T E R N E L E C T R I C I T Y C O O R D I N A T I N G C O U N C I L • W W W . W E C C . B I Z1 5 5 N O R T H 4 0 0 W E S T • S U I T E 2 0 0 • S A L T L A K E C I T Y • U T A H • 8 4 1 0 3 - 1 1 1 4 • P H 8 0 1 . 5 8 2 . 0 3 5 3 • F X 8 0 1 . 5 8 2 . 3 9 1 8

Page - 8Violations that will be sent to WECC Enforcement will be identified in the closingpresentation scheduled for the last day of the audit. Compliance and Enforcement aretwo separate processes.Compliance:• The burden is on the Registered Entity to provide sufficient evidence so that theRegional Entity (“RE”) can be reasonably assured of compliance.Enforcement:• A possible violation found during an audit will not be deemed a confirmedviolation until it has been reviewed and acted upon by WECC Enforcementpersonnel.Will I get a report from WECC regarding my audit results?WECC will prepare a draft audit report within 15 days of the end of the audit. You willhave an opportunity to comment on the draft audit report. Comments should be limitedto corrections in spelling of names, company profile information and errors in the findingsummary, but arguments as to the findings are not accepted. You will have 10 calendardays to provide comments on the form provided with the report draft. If no commentsare received, the audit report will be sent to WECC management for final review andacceptance. If you provide comments, they will be considered and either accepted(corrections to names, titles and company profile will be automatically accepted) orrejected. Please be aware that NERC provides the template for the audit report and,consequently, there is template language that WECC cannot change.What can we do to make our audit go more smoothly?DocumentationKeep current and retain past versions of all relevant documents and be able toproduce them on requestMake sure retention policies align with the obligation to produce evidence for anaudit. Be familiar with bookend requirementsKeep track of potential document retention obligation changesOperationso Retain logs and communications of significant eventsProtection Systems, Vegetation Managemento Retain all maintenance and testing recordsPlanningo Retain studies and relevant planning communicationsW E S T E R N E L E C T R I C I T Y C O O R D I N A T I N G C O U N C I L • W W W . W E C C . B I Z1 5 5 N O R T H 4 0 0 W E S T • S U I T E 2 0 0 • S A L T L A K E C I T Y • U T A H • 8 4 1 0 3 - 1 1 1 4 • P H 8 0 1 . 5 8 2 . 0 3 5 3 • F X 8 0 1 . 5 8 2 . 3 9 1 8

Page - 9SMEs should be familiar with their documents in advance of the audit so thatthey can identify provisions that meet the requirements quickly and efficientlyBookmarking or highlighting relevant provisions benefits the entity and audit staffIdentify and inform SMEs well in advance of the audit. Engineers are oftenunaccustomed to regulatory interviews and also need to be familiar andcomfortable with their roles.Involve SMEs in RSAW preparation. They know the material and this aids theirunderstanding of the organization of the evidence and helps them build acoherent story.Reference Documents:20110420_CAN-0010_Definition_of_Annual.pdf20110420_CAN-000 Quality Evidence for Off-site audits with a8_PRC-005-1_R2_Pre-June_18th_Evidence.pdfAudits.pdf CIP Component January 13, 2011.pptRelay_Loadability_Reference_09Jan07.pdfhttp://www.nwppa.org/web/presentations/09_EandO_Conf/Risk_Based_Assessment_Methodology1.pdfhttp://www.nerc.com/docs/pc/spctf/Relay_Maintenance_Tech_Ref_approved_by_PC.pdfW E S T E R N E L E C T R I C I T Y C O O R D I N A T I N G C O U N C I L • W W W . W E C C . B I Z1 5 5 N O R T H 4 0 0 W E S T • S U I T E 2 0 0 • S A L T L A K E C I T Y • U T A H • 8 4 1 0 3 - 1 1 1 4 • P H 8 0 1 . 5 8 2 . 0 3 5 3 • F X 8 0 1 . 5 8 2 . 3 9 1 8