guide to continuous improvement review (cir) - WDA

SINGAPORE WORKFORCE SKILLS QUALIFICATIONS SYSTEM (WSQ)
GUIDE TO CONTINUOUS IMPROVEMENT REVIEW (CIR)
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 1

Contents at a Glance
Introduction
Content
Pg
Part I: Overview of Quality Assurance Framework 4
Part II: Continuous Improvement Review (CIR)
8
– Essential Elements of Review
Part III: Roles and Responsibilities of Approved Training Organisation 17
(ATO)
Annex A : Steps-by-Steps Process for ATO 23
Annex B : Self-assessment Checklist 28
Annex C : 2 Year Plan 35
Annex D: CIR Presentation by ATO 37
Annex E : CIR Outcome Measures 49
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 2

Introduction
The field of adult training is about providing quality courses to cater to the learning
needs of the adult. A quality course would be able to enhance the learning
experience of the adult and contribute to the adult in more ways than we can ever
imagine. Some examples of the contribution would be 1) increase employability, 2)
improve skills set, and 3) innovate work processes.
However, as an Approved Training Organisation (ATO), providing quality courses
are just one of the many responsibilities you are expected to do. It is really a case of
simple logic. To provide quality output i.e. quality courses, you need to have quality
input.
What is quality input you might ask?
To answer the question, we strongly recommend you to explore this guide. We
assure you that it will be a meaningful learning journey, and you will be able to take
away more than you can imagine.
The bottom-line is that this guide serves more than just preparing your organisation
for the Continuous Improvement Review (CIR). It aims to guide your organisation in
establishing a strong quality Continuous Education and Training (CET) infrastructure
system within your organisation.
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 3

Part I
Overview of
Quality Assurance Framework
In this part …
Part 1 deals with the WSQ quality assurance framework and the components
existing within the framework. On the application side, this part introduces the
actions required by any organisation embarking on the journey of becoming a WSQ
Approved Training Organisation (ATO). It is a brief introduction of the different
processes involved. The objective of Part 1 is to allow you to understand the journey
you have taken to becoming an ATO and examine the processes that you would
have gone through before the CIR.
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 4

Part I
Overview of
Quality Assurance Framework
In This Part
‣ Introduction to the Quality Assurance Framework – What it means to you
‣ The Process within the Framework
The Concept of WSQ That You Just Have
to Know
If you are a WSQ ATO, the term WSQ would not be unfamiliar to you. From the
moment you have chosen to be a WSQ ATO, you should have a basic
understanding of WSQ. However, if it happens to slip your mind at the point you are
reading this guide, we are providing you with a quick note to refresh your memory.
The Singapore Workforce Skills Qualifications (WSQ) is a robust and integrated
continuing education and training system. Designed to build industry competencies,
it is founded on international best practices and validated by industries and
employers. WSQ caters to adult workers who have widely diverse training needs; it
offers a wide range of certification and qualification ranging from Certificate to
Graduate Diploma.
Quality assurance provides the backbone for the WSQ. A strong quality assurance
framework is important to build up public confidence in the WSQ.
So as a provider under the WSQ framework, you are recognised by the public to be
providing courses that are based on a robust quality CET system.
It is thus a joint responsibility between yourself and WDA to continuously contribute
and improve on the robustness and quality of the CET system.
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 5

Mechanism to contribute and improve
WSQ quality
WDA has developed the quality assurance framework as a mechanism to uphold the
quality of WSQ. In addition, this framework serves to continuously improve the
quality of WSQ.
The quality assurance framework is made up of two processes:
i) Pre-delivery Approval – There are 2 stages of accreditation under the
pre-delivery approval, 1) the organisation approval accreditation, and 2)
the course accreditation. The objectives of both accreditations are
similar; that is to ensure that organisation has the capability to delivery
courses that satisfy the competency requirements of the WSQ and is
accessible to workers through appropriate delivery modes and
resources. The process also ensures that the course will be delivered
by a quality training organisation using qualified adult educators. Upon
approval, the following status will be conferred:-
(a)
An organisation approved to deliver stipulated training
and/or assessment services under the WSQ is known as an
Approved Training Organisation (ATO);
Henceforth the term Approved Training Organisations or ATOs will
be used to refer to WSQ accredited organisations.
At this stage, we would assume that you have gone through the above mentioned
process and the 2 stages within the process. If you have not, we recommend you to
visit this link http://www.wda.gov.sg/content/wdawebsite/L225-
ForTrainingProviders/L325A-001TP-ATO.html, so as to better understand the details
behind the accreditation.
However, do not stop your reading of this guide. It is still relevant for you to
understand the types of mechanism WDA is adopting to ensure the quality of the
WSQ frameworks.
Moving on, the second process under the quality framework is none other than the
theme of this guide – Continuous Improvement Review (CIR).
ii)
Continuous Improvement Review (CIR) – Upon approval (organisation
and course), every ATO is to maintain and upkeep the standard of
delivery in training and assessment. In addition, every ATO is to pursue
continuous improvement to uplift their quality of design and delivery.
WDA will periodically validate the internal quality assurance system and
capability of each ATO under CIR.
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 6

It is important to note you are not taking this journey alone. Throughout the journey,
WDA will assist ATO to develop their capabilities in designing and delivering of
quality training and assessment under WSQ.
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 7

Part II
Continuous Improvement Review (CIR)
– Essential Elements of Review
In this part …
Here is where you learn about the essential criteria of review under the Continuous
Improvement Review (CIR) framework. We will begin by explain the purpose of the
CIR. From there, we will look in the areas which we will review during the CIR. Along
the way, you will find out about the specific components that we are evaluating under
each of the essential criteria.
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 8

Part II
Continuous Improvement Review (CIR)
– Essential Elements of Review
In This Part
‣ The Purpose of CIR
‣ The Essential Elements of Review
Continuous Improvement Review
Continuous Improvement Review is one of the extremely important process in the
quality assurance framework that WDA puts in place to maintain and improve the
quality of WSQ. CIR enables WDA to discern the gaps in quality pertaining to WSQ.
As an ATO, you could use CIR as a quality check tool to improve your internal CET
structure. It is thus valuable to understand the criteria in the CIR system so as to
maximise the usage of it to improve your organisation’s capability and quality as an
ATO.
Why CIR?
The main objectives of CIR are:
i) Inculcate the practice of seeking continuous improvement in the delivery of
training and assessment;
ii) Identify best practices and role models amongst the ATO; and
iii) Nurture ATO to be WDA’s partners in providing quality continuing education
and training.
Therefore, CIR is really a process for both WDA and the ATO to critically reflect on
the practices adopted by the ATO. Such reflections provide opportunities for the ATO
to refine its CET system, and thus contributing to the overall improvement of WSQ.
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 9

Essential Criteria of Review
All ATO is subjected to audit on the following essential criteria for CIR:
1. Course Quality Assurance System
i) System to ensure courseware and assessment are aligned to WSQ,
pedagogically sound and contextualised
ii) System to manage profiling of learners to ensure that the course(s) is fit for
purpose
iii) System to manage training activities
iv) System to manage assessment activities
2. Plan-Do-Check-Act (PDCA) System
i) System to ensure PDCA is implemented regularly for the
comprehensiveness of training, assessment and management system
3. Outcome Evaluation System
i) System to monitor and analyse L1 outcomes
ii) System to monitor and analyse L2 outcomes
iii) System to monitor and analyse L3 outcomes
iv) System to monitor and analyse L4 outcomes
4. Adult Educator Management System
i) HR system to ensure most qualified adult educators are recruited and
inducted
ii) HR system to monitor, appraise and manage performance of adult
educators
iii) HR system to train and develop adult educators
5. WSQ Related Administrative System
i) Learner support system
ii) System to ensure accurate and timely submission of assessment
results
iii) WSQ certificate printing and distribution system
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 10

iv) Maintain up to date profile in Skillsconnect
6. Viability of Organisation
i) Governance and management of organisation
ii) Financial health of organisation
7. Free from Breach of Terms and Conditions
i) Records of organisation free breach of WDA terms and conditions.
System-Evidence-Outcome Approach
WDA is adopting the system-evidence-outcome (SEO) approach in the evaluation of
the CIR criteria. The focus on the SEO approach is because there is strong
correlation between sound systems and the quality of outcomes. It has to be reiterated
here that the continuation of your approval status/es is/are subjected to the
outcome of WDA’s scheduled CIR audit which would be graded, based on the
scoring matrices for S-E-O.
Definition of S-E-O
1. Systems: a set of interacting or interdependent entities forming an integrated
whole, giving a macro perspective. Components within the system generally have
interconnectivity i.e. functional and structural relationships.
2. Evidences: a thing or things helpful in forming a conclusion or judgment:, for
instance, the broken window was evidence that a burglary had taken place.
3. Outcomes: refers to the way something turns out; result; consequence.
To ensure that you are on the same page as us, we have provided an example of S-
E-O institute for courseware development. (Please see pg. 13 for the given example)
You have to note that in the given example, the system is represented by a process
flow chart. You can represent your system in many forms e.g. a standard operating
manual, a policy guide etc.
The key in developing a comprehensive system is how concise and systematic the
information is presented. To achieve this, you first have to set your organisation
SMART (Specific, Measurable, Agreed upon, Realistic and Time-based) targets
when drafting any system at the outset.
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 11

Succinctly, each system devised by you should address the following 5 questions:
i) When would the system be devised and implemented; i.e., effective
start date of system, frequency of system implementation.
ii) Who would be involved in the system; i.e., the personnel who is
directly involved in either the overseeing or implementing of the system
iii) What approach/tools would the system used; i.e., the specific
approach/tools that the system would be using in its course of
implementation.
iv) Why did ATO have this system; i.e., the rationale/objective of the
system.
v) How would each process within the system be evaluated and follow
up on; i.e., any necessary follow up actions/evaluation are to be
addressed accordingly in the system.
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 12

Example of S-E-O
System: Process flow for Courseware Development
Courseware
Development Team *
Conduct research and survey the market and
customers’ needs
Industry Information, Market
Research, Course Evaluation
Forms, Companies’ Feedback
Perform thorough study of the course objectives and assess
capabilities such as trainer’s skills, facilities, equipment
(Refer Note 1)
Perform simple costing to evaluate if the course is
profitable
Costing
Profitable
No
Case Closed
Yes
Courseware Developer
(Refer Note 2)
Develop the Course Programme Design and Overview,
Trainers’ Guide, Lesson Plan and Course Materials as
per WSQ standards (1 st draft to be submitted within 1
month to the Courseware Development Team)
WSQ Programme Design and
Overview, Trainers’ Guide,
Lesson Plan,
Course Materials
Courseware
Development Team
Submit to Courseware Development Team for
approval
* Team consists of Principal, Training
Manager and Trainers
Approved by Team
(See Note 3)?
No
Course Developer to rework
based on Team’s directions
Yes
A
Continue on pg 2 of 2
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 13

Effective Date: 1 Jul 10
Pg 1 of 2
XYZ Pte Ltd
Title : Course Development Process Effective Date: 1 Jul 10
Doc No: XYZ-QA-01
Page Version 1 of 2 : of 1.0 Doc
A
Training Manager
Brief the appropriate trainers on the training concept
before trainers proceed to conduct the new course.
Trainers
Proceed to conduct the course
Perform design validation by analysing course
evaluation forms / interviewing trainees and conducting
post-course profiling for 3 sessions to gather
participants’ profiles/feedback on the new course
Course Evaluation Form
(Trainee) – Form 1
Course Evaluation Form
(Trainer) – Form 2
Post-Course Profiling – Form 3
Changes in course
design?
Yes
Follow up by Courseware Developer to
amend and update the relevant
documents and affected staff
accordingly
No
Proceed with next course
Note 1
For any deficiencies of trainer’s
skills, suitable trainers shall be
sent for training or training could
be outsourced to qualified parttime
trainers.
Training facilities shall be properly
planned. Any new training
equipment/products required can
be bought, pending the costing
calculation.
Note 2
New course developers will have
to attend pre-development briefing
conducted by the Course
Development Team. The minutes
of the briefing and the attendance
must be recorded and compiled for
future reference.
Note 3
For WSQ programmes, any new
courses or revisions done to
training materials must be
submitted to WDA for approval
XYZ Pte Ltd
Title : Course Development Process
Effective Date: 1 Jul 10
Effective Date: 1 Jul 10
Doc No: XYZ-QA-01
Pg 2 of 2
Page Version 1 of 2 : of 1.0 Doc
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 14

It is always useful to enhance each flowchart and all the supporting form/s with
document names, document numbers, effective dates and version control tracking
mechanisms.
As the scoring for the CIR criteria is based on 3 components – system, evidence and
outcome, having a comprehensive flowchart means that the ATO would score for
system.
The Evidence in this specific example would be the following:
- Industry Information, Market Research, Course Evaluation Forms, Companies’
Feedback, etc.
- Course Evaluation Form (Trainee) – Form 1, Course Evaluation Form (Trainer) –
Form 2, Post-Course Profiling – Form 3, etc.
The Outcome in this specific example would be the following:
- Good L1 score as trainees find the course relevant;
- Repeat corporate clients being served (as the organizations know what they are
getting if they are requesting for the same training the second time).
Please note that the under this specific example, the ATO has to devise Forms 1, 2
and 3 as well as document the Industry Information, Market Research, Companies’
Feedback, etc as evidence of deployment of the system so as to score points under
the evidence matrix.
Likewise, outcome/s would have to be presented as well-collated L1 scores by
course or trainer on a periodic (e.g., quarterly, half-yearly, etc) basis. While the list of
clients served, especially established corporate clients or walk-ins through words of
mouth, etc if available would be useful in securing points in the outcome matrix.
In addition, it is important to note that there should be
clear and concise documentation for all the criteria under
the S-E-O. Any S-E-O without documentation will affect the
CIR evaluation outcome.
You have come to the end of Part II of this guide. At this point, you should have a
general idea of the seven essential criteria of review in the CIR:
i) Course Quality Assurance System
ii) Plan-Do-Check-Act (PDCA) System
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 15

iii) Outcome Evaluation System
iv) Adult Educator Management System
v) WSQ Related Administrative System
vi) Viability of Organisation
vii) Free from Breach of Terms and Conditions
You should also have a general understanding of the S-E-0 approach adopted by
WDA in the evaluation of the essential criteria. However, if you still have doubts in
your mind, you can always email or drop a call to the officer-in-charge of your
organisation to seek their advice.
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 16

Part III
Roles and Responsibilities
of ATO
In this part …
In the third part of the guide, we will be talking about the preparation requires by an
ATO/ATC/AAC when being notified of a CIR by WDA. We will talk about the things
that you are required to prepare after you have received the CIR notification from
WDA, and the things that you are required to do consistently as an ATO.
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 17

Part III
Roles and Responsibilities
of an ATO
In This Part
‣ The Roles and Responsibilities of an ATO
‣ Scope of CIR
‣ ATO Continuous Improvement Efforts
‣ Outcomes of CIR
‣ Fees and Feedback
The Roles and Responsibilities –
Together We Play a Part
It is important to note that you do not journey alone when you are an ATO. WDA will
provide the necessary assistance to aid you along the way. Thus, before the CIR,
during the CIR, and even after the CIR, WDA seeks to work closely with you to
improve your overall quality as an ATO. Some of the aids that WDA would provide to
you in building you quality are:
i) Validate the internal quality assurance of ATOs
ii) Build up the competencies of the ATOs through feedback
iii) Issue CIR audit report for CIR audit conducted
iv) Highlight ATO’s strengths and areas for improvement
v) Facilitate ATO’s participation in capability development initiatives and
workshops
However, as an ATO, you need to have the correct attitude in building up your own
quality. As the saying goes, “Your Attitude Determines Your Altitude”, you would
have to put in effort to improve your own ability and quality as an ATO. Some of the
things you are recommended to do constantly are:
i) Practice internal quality assurance
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 18

ii) Ensure Plan-Do-Check-Action (PDCA) system is in place and is
diligently making continuous improvements to meet the required quality
or for better quality, especially in areas where required by WDA from
previous CIR audits conducted
iii) Ensure quality of accredited courseware and adult educators are
consistent with actual training and/or assessment
iv) Ensure evidences of all systems implementation, and dates of training
and assessment are available for audit when requested
You have learned about the criteria of review and the S-E-0 that WDA used in CIR
evaluation. It is time to look into the activities that you are required to do on the day
of the CIR.
Scope of CIR
In part II of the guide, we have gone through the essential criteria of review, and the
expected responsibilities of an ATO. We will now move on to look into the specific
things you will be required to do when you received the CIR notification from WDA
and on the actual day of the CIR.
When you received the CIR notification email from WDA (one month before the CIR
date), you are required to response to the notification email within 5 workings days
providing the following:
i) A series of available dates and time for the CIR
ii) A training and/or assessment schedules
iii) A self assessment checklist (see Annex B)*
iv) A 2 year plan (see Annex C) (Applicable to only ATO offering public
courses)*
*This is applicable only for public ATOs and is to be provided within 2 weeks
from the date of the notification
Upon receiving your response with the attached documents, the office-in-charged of
your CIR will then get in touch with you to finalise on the CIR date and time based on
the series of dates and time provided by you. The officer may also request more
information based on the 2 attached documents that you are required to submit.
Once the date and time of the CIR is finalised, you can look forward to the actual
CIR date and prepare the necessary presentation and materials for the CIR.
According to the stated essential criteria in part II, you will be required to make a
presentation based on each criteria. The presentation is recommended to be kept
under 15mins (see Annex D for the breakdown of the presentation process). You
need to have the supporting evidences ready for verification during the CIR audit.
The review covers the following:
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 19

i) Showcase of all the implemented systems based on the essential
criteria of review e.g. relevant SOP manual
ii) Specific evidence of each implemented system, e.g. training
observation checklist with requirements clearly communicated to all
adult educators, or learners’ profiling checklist in determining
contextualisation of courseware, etc.
iii) Outcome of each implemented system, e.g. systematic documentation
of all CVs and qualifications of qualified adult educators (against
declaration submitted at accreditation), or assessment results with
analysis to reflect L1 and L2 outcomes, etc.
It is important that you prepare the relevant evidences and supporting documents in
advance before your scheduled CIR audit by WDA. The reason of the preparation is
really quite simple - you do not want to be running around getting the documents and
indirectly reflecting the possibility of an inadequacy in your administrative system.
Your Continuous Improvement Efforts
Specifically, ATO is required to practice continuous improvement in its ongoing
delivery of training and assessment under WSQ. Evidence of these efforts should be
available or demonstrated in the form of PDCA through any of the following:
i) Quality of design and delivery of courseware, e.g. regular reviews to
ensure currency, for continuous improvement, contextualisation to be
relevant to a particular industry or target audience etc;
ii) Quality of trainers, assessors and curriculum developers, e.g. training
and development plan and implementation, performance management
system etc.;
iii) Assessment Management System, e.g. regular reviews and
improvements made to internal quality assurance system particularly
for the management and conduct of assessments, ranging from how
assessments are designed and conducted to ensure the four principles
of assessments – reliability, fairness, validity and flexibility as well as
positive and negative features experienced in conducting the
assessments, and refinements made, assessment validation etc.;
iv) Administrative System (supporting training and assessments for
accurate and timely records submission to WDA and certificate
issuance to trainees);
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 20

v) Tracking of outcomes, e.g. end-of-course evaluation, attrition rate,
passing rate, surveys conducted etc.
You can refer to Annex A for the step-by-step of CIR for ATO to have an overview of
the CIR processes.
Outcomes of CIR
Possible Capability Development Initiatives
From the CIR pilots conducted, WDA has collated some common areas of
improvement arising from findings of the ATOs piloted. These are taken into
consideration for capability development initiatives that are being planned. Some of
the initiatives in the pipeline include the conduct of regular seminars to equip ATOs
with the knowledge to professionalise internal practices, which can be in the form of
monthly briefings, one-to-one consultations, or sharing of best practices and
outcomes of CIR in groups. ATOs can also look forward to the development of a
guidebook to assist them to set up a training system and implement continuous
improvement practices.
The CIR will further provide information and evidence that will enable ATOs and
WDA to identify areas of strength as well as areas in need of focused attention for
planned improvement in the short, medium and long term. With these needs
identified, WDA can then design suitable interventions to assist the ATOs.
Continuation of Approval Statuses
In addition to capability development, the CIR also replaces the need for application
by ATO for renewal or extension of WSQ statuses, i.e. the CIR determines if the
approval status of the ATO should be continued.
The continuation of approval statuses underpinned by the results of CIR is guided by
the CIR Outcome Measures. Other considerations determining the continuation of
approval statuses besides the periodic CIR are also guided by the CIR Outcome
Measures.
In the event that the ATO's WSQ status is not renewed arising from the results of
CIR, the ATO can appeal to WDA on the outcome of the CIR. The ATO need to
submit an appeal letter to WDA, Director (Quality Assurance Division) stating the
background and nature of the appeal together with the reasons and supporting
evidence as to why the original outcome decision should be reversed or with a non-
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 21

efundable fee of $350 within 2 weeks from the revoke/termination notification letter
issued by WDA.
The complete CIR Outcome Measures is represented in Annex E.
Fees
An annual fee of $321 (inclusive of GST) is payable per ATO for CIR. CIR fees are
payable from 1 Apr 2007 via e-payment by ATO upon receipt of WDA’s payment
advice.
Feedback
WDA welcomes any feedback to seek continuous improvement and fine tune the CIR
system. Please send your comments to Ms Chiang Han Meng at
chiang_han_meng@wda.gov.sg.
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 22

Annex A: Step-by-Step Process of CIR for ATO
Step Action Needed Remarks Documentary Evidence Timeline
Step 1:
Receipt
CIR Notice
of
ATOs are scheduled for CIR
and notified by WDA through
email 1 month before the CIR
According to CIR
schedule
Upon receipt of WDA’s CIR
notification, ATO is to respond
to WDA with the completed
self-assessment checklist
(Annex B) and a 2 year plan
(Annex C) (Applicable to only
ATO offering public courses)
ATO is also to indicate a series
of available dates and time in
the response to the CIR
notification.
For areas which ATO does not
have a system or not
implemented yet, ATO is to
submit an action plan providing
details on:
WDA Officer will evaluate the
ATO’s readiness for the CIR
WDA Officer advises on
allocated documentation
submission schedule and/or
range of duration for possible
site audit and possible
training/assessment
observation sessions.
WDA Officer will finalise the
CIR date and time with the
ATO based on the provided
series of dates and time from
the ATO
Training and assessment
schedules
ATO is to
responding to WDA
within 5 working
days upon receipt of
CIR notification
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 23

Step Action Needed Remarks Documentary Evidence Timeline
a) Gaps
b) Follow-up actions to
address gaps
c) Time needed to
implement changes.
ATO is to provide
training/assessment schedules
and anticipate a period (and not
a stipulated date and time)
when the site visit would
possibly be conducted
Step 2:
Preparing for
CIR Audit
ATO to prepare a 15-min
presentation outline and
coverage advised in Annex D
ATO to get ready required
evidences and supporting
documents and when informed
about the site audit, ATO to
avail facilities for site audit.
WDA-QAD will go down to
ATO’s training site to conduct
CIR site audit.
Standard CIR checks to
include :
1. Course Quality
Assurance System
2. Plan-Do-Check-Act
(PDCA) System
3. Outcome Evaluation
System
4. Adult Educator
Management System
1. SOP manual/flow chart
etc, of the various
systems
2. CVs and evidence
showing qualifications
of trainers, assessors
and curriculum
developers declared
3. Process of review by
ATO to ensure
currency of
courseware, relevance
to industry, target
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 24

Step Action Needed Remarks Documentary Evidence Timeline
5. WSQ Administrative
Related System
6. Viability of
Organisation
7. Free from Breach of
Terms and Conditions
8. Evidence-of-
Implementation of the
Various System
9. Outcome(s) of the
Various System
audience and
contextualisation
4. Administrative System
– Show evidences of
accurate and timely
assessment
submission and
certification issuance
processes
5. Outcomes tracking,
e.g. attrition rate,
passing rate, surveys
conducted etc
6. All evidence of
implementation of the
various systems
7. Documentation of
outcomes of all the
various systems
Supplementary evidence
sources or supporting
documents which may be
requested for the CIR
audit are:
1. Course files (course
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 25

Step 3:
Step Action Needed Remarks Documentary Evidence Timeline
During site
audit
15 minutes presentation by
ATO
Site audit and MR interview
Conduct audit of training
premises (if applicable)
The management
representative should be
present during the site audit
materials submitted at
point of course
application, includes
Assessment Plan,
Slides, Participants’
Handouts and Trainer
Guide)
2. Course attendance
records
3. Training and
assessment records
ATO’s slides and
handouts for presentation.
All SOPs related to the
implemented systems
The site audit would
normally not take
more than half a day
Step 4 :
Training/
assessment
Observation
Step 5 :
After the audit
ATO to avail
training/assessment sessions
for WDA Officer’s observation
Go through areas (strengths
and weaknesses) highlighted
WDA Officer will
brief provider and
discuss follow-up
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 26

Step Action Needed Remarks Documentary Evidence Timeline
by WDA Officer
actions if necessary
WDA Officer to
prepare CIR report
Step 6 :
Receive CIR
audit report
and outcome
letter
Study findings in the report.
Take note of rectifications and
timeline in CIR outcome letter
and take corrective actions
CIR outcome
measures (shown in
Annex E) will apply
– where ATO has
cleared, taken action
or failed to take
corrective actions
within stipulated
time/does not
respond
* Timeline for CIR cases is 50 working days from the date of the last CIR activity.
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 27

Annex B: Self-assessment Checklist
Name of ATO:
Name of Management Representative (MR):
Contact Details of MR:
(Email)
Proposed
CIR Date:
(Tel No) (O) (Hp)
Section A
Are you able to show evidence for all the items in the checklist? If your answers are all ‘Evident’ in Section A, your organisation is likely to be ready for a CIR
audit. Please note that the list of evidence suggested for each area is NOT EXHAUSTIVE.
Area of Review 1 – Course Quality Assurance System
Criteria Interpretation of Criteria Implemented System Evidence of System
Implementation
Outcome of System
Implementation
Evident (E) / Not Evident (NE)
1.1) System to ensure
courseware and assessment are
aligned to WSQ, pedagogically
sound and contextualised
System in place to manage developers - E
Process in place to conduct internal
evaluation and validation of
courseware to ensure alignment to
WSQ standards
- NE
- E
- NE
- E
- NE
- E
- NE
- E
- NE
- E
- NE
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 28

System in place to facilitate
contextualisation of courseware to
ensure its currency and relevancy, etc
- E
- NE
- E
- NE
- E
- NE
1.2) System to manage profiling
of learners to ensure that the
course(s) is fit for purpose
Process in place to ensure systematic
screening/profiling of learners, e.g. did
the learner meet the prerequisite of
the course:
1. literacy level
2. numeracy level
3. required assumed
knowledge/skills, etc
Process in place ensure to only learners
with the appropriate aptitude/attitude
are enrolled into the course
- E
- NE
- E
- NE
- E
- NE
- E
- NE
- E
- NE
- E
- NE
1.3) System to manage training
activities
1.4) System to manage
assessment activities
System in place to continuously
monitor and manage the training
processes to ensure:
1. it is in accordance to industry
practices i.e. ensuring quality and
currency of training
System in place to continuously
monitor and validate the assessment
processes to ensure it is in accordance
to principles of assessment i.e., valid,
reliable, fair, and flexible, and rules of
evidence
- E
- NE
- E
- NE
- E
- NE
- E
- NE
- E
- NE
- E
- NE
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 29

Area of Review 2 - Plan-Do-Check-Act (PDCA) System
Criteria
Interpretation of Criteria Implemented System Evidence of System
Implementation
Evident (E) / Not Evident (NE)
Outcome of System
Implementation
2.1) System to ensure PDCA is
implemented regularly for the
comprehensiveness of training,
assessment and management
system
Demonstration of establishment in
PDCA. This may include:
1. Organization management
2. Training management
3. Assessment management
- E
- NE
- E
- NE
- E
- NE
Area of Review 3 – Outcome Evaluation System
Criteria Interpretation of Criteria Implemented System Evidence of System
Implementation
Outcome of System
Implementation
Evident (E) / Not Evident (NE)
3.1) System to monitor and
analyse L1 outcomes
Clear articulation of indicators, this is to
include trainee satisfaction rate.
Trainee satisfaction rate minimally on
these 4 areas:
1. Training Programme
2. Course Material
3. Trainers
4. Facilities and Logistical support
Analysis of obtained data
Follow up action(s) on analyzed data
- E
- NE
- E
- NE
- E
- NE
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 30

3.2) System to monitor and
analyse L2 outcomes
3.3) System to monitor and
analyse L3 outcomes
Clear articulation of indicators, this is to
include
1. Passing rate
2. Attrition rate
Analysis of obtained data
Follow up action(s) on analyzed data
Clear articulation of indicators on
impact of transfer of learning, this is to
include:
1. Extent of candidate’s behaviour
and/ or capability improvement
as indicated by candidate
2. Extent of candidate’s behaviour
and/or capability improvement
as indicated by employer
Analysis of obtained data
- E
- NE
- E
- NE
- E
- NE
- E
- NE
- E
- NE
- E
- NE
Follow up action(s)on analyzed data
3.4) System to monitor and
analyse L4 outcomes
Clear articulation of indicators on
impact of training on business, which
may include:
1. volumes,
2. values,
3. percentages,
4. timescales,
5. return on investment,
6. other quantifiable aspects of
organisational performance, for
instance; numbers of complaints,
staff turnover, wastage, noncompliance,
quality ratings,
accident ratings, zero-defect
ratings, growth, retention, etc.
- E
- NE
- E
- NE
- E
- NE
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 31

Analysis of obtained data
Follow up action(s)on analyzed data
Area of Review 4 – Adult Educator Management System
Criteria Interpretation of Criteria Implemented System Evidence of System
Implementation
Outcome of System
Implementation
Evident (E) / Not Evident (NE)
4.1) HR system to ensure most
qualified adult educators are
recruited and inducted
System in place to recruit and induct
qualified adult educators
- E
- NE
- E
- NE
- E
- NE
4.2) HR system to monitor,
appraise and manage
performance of adult educators
System in place to manage the
performance of adult educators
- E
- NE
- E
- NE
- E
- NE
4.3) HR system to train and
develop adult educators
System in place to facilitate adult
educators ongoing professional
development
- E
- NE
- E
- NE
- E
- NE
Area of Review 5 – WSQ Related Administrative System
Criteria Interpretation of Criteria Implemented System Evidence of System
Implementation
Outcome of System
Implementation
Evident (E) / Not Evident (NE)
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 32

5.1) Learner Support System in
place
5.2) System to ensure accurate
and timely submission of
assessment results
5.3) WSQ certificate printing and
distribution system
5.4) Maintain up to date profile
in Skillsconnect
Learner support system that covers:
1. Course advisory
2. Progression pathway
3. Course schedule
4. Fees and refund policy
5. Retest/Appeal procedures
6. Career advisory (if applicable)
7. Placement support (If applicable)
Assessment data are submitted in
accordance to prescribed timeline
within 2 weeks from the date of the
assessment/within 1 month from the
date of assessment*, unless otherwise
approved
Submission of results must be accurate
*applicable only to In-house ATO
WSQ cert is printed and distributed
according to agreed service timeline:
1. Cert must be printed and
distributed to trainees within 3
months from date of assessment,
unless otherwise approved
2. Distribution of original certificate
must be accurate
Profile information in Skillsconnect
must maintain the most up-to-date
information. This include:
1. Organization profile
2. Course profile
3. Adult educators profile
- E
- NE
- E
- NE
- NA
- E
- NE
- NA
- E
- NE
- E
- NE
- E
- NE
- NA
- E
- NE
- NA
- E
- NE
- E
- NE
- E
- NE
- NA
- E
- NE
- NA
- E
- NE
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 33

Section B
If answer to any question below is ‘Yes’, please elaborate under Remarks column.
S/N Questions Yes No Remarks
1 Are there changes in the management level personnel for the past one year?
2 Is there any major instability in the organisation’s financial health for the past
one year?
3 Has your organisation violated any WSQ terms and conditions?
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 34

Annex C: 2 Year Plan
(Applicable to only ATO offering public courses)
ATO is required to submit proposal on the training and assessment delivery on a 2-year
timeline, this is to ensure that the ATO have a clear mission and vision on their involvement
in WSQ. The proposal must indicate the following:
i) Identified one specific WSQ framework, e.g., Security WSQ;
ii) Intended scope of delivery within the framework, e.g. bundling of SOAs for
the 2 licensing modules at the Certificate in Security Operations (CSO) level,
full qualification of the CSO, etc;
iii) Target market and proposed clientele mixed, eg., training only Malaysian
learners;
iv) Existing clientele and new clientele over the past year who dropped out, eg.,
Malaysian trainees who had attained at least 3 GCE ‘O’ Levels including
English Language or its equivalent; and Malaysian trainees with language
difficulties and cultural adjustment issues (ascertained through screening);
v) Proposed marketing and outreach strategies, eg. advertising in the Malaysian
print media and newspapers, course previews and career fairs at Malaysian
recruitment centres, etc;
vi) Intended annual headcount to be trained, eg. at least 25 trainees for the 2
licensing modules at the CSO level, (or 50 Statements of Attainments [SOAs])
per month; and
vii) Proposed innovative learning and assessment strategies, eg. Compact Disks
(CD) would be deployed for intensive trainee revisions on the practical
performances before assessments at the centralized assessment centre,
Polwel Co-operative Society Limited.
viii)Risk assessments by the ATO. These include:
Financial Risk
Operations Risk
Strategic Risk
To facilitate the submission of the proposal by the ATO, ATO can use the provided template
in the following page. ATO is advised to provide between 1000 and 1250 words for the
proposal.
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 35

Template for 2-year plan
Applicants are advised to provide between 1000 and 1250 words for the proposal.
i. Specific WSQ framework
Identified existing WSQ framework, and/or new WSQ framework(s) with intention of embarking
into.
ii. Scope of delivery
Intended scope of delivery within the framework (existing and/or new). Indicate the WSQ modules
e.g. interact with guest, deliver service excellence, workplace safety and health.
iii. Target market and proposed new clientele-mix
The trainees and the intended proportion Eg., training only Singaporean and local PRs.
Target Corporate Clients (indicate clienteles’ names).
iv. Existing clientele
The existing ratio of Singaporeans/PRs trainees in relation to foreign trainees.
Existing Corporate Clients.
v. Proposed outreach strategies
Proposed marketing and outreach strategies, eg. advertising in the print media and newspapers,
course previews and career fairs, etc.
vi. Estimated annual headcount to be trained and projected number of SOAs
Eg. at least 25 trainees for the 2 licensing modules at the Customer Service Officer level, (or 40
Statements of Attainments [SOAs]) per month.
vii. Proposed learning and assessment strategies
Eg. Small group tutorials cum buddy scheme for learners with low learning retention ability and
compact disks (CD) would be deployed for intensive trainee revisions on the practical
performances before assessments at the centralised assessment centre.
viii. Risk assessments by the applicant
Eg. Security ATOs would face a declining no. of trainees should their clients convert to in-house
training or install the following IT infrastructure:
- Bio-metric access and manpower scheduling system;
- Remote CCTV surveillance system; and
- Visitor management system.
The ATO would hence face a challenge to meet the initial proposed training target/s. ATO to
suggest and elaborate how to mitigate this risk.
ix. Remark/Other Comments
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 36

Annex D: CIR Presentation by ATO
The Continuous Improvement Review (CIR) will commence with a
presentation by the ATO outlined as follows. The presentation can be in the
form of a powerpoint presentation or briefing notes. The Management
Representative (MR) must be present to conduct the presentation or provide
clarifications. The presentation or brief notes will be part of the documentary
evidences of the CIR. Evidences can include the following:
1. Course Quality Assurance
i) Course files (course materials submitted at point of course application,
includes Assessment Plan, Slides, Participants’ Handouts and Trainer
Guide)
ii) Summary page of courses with updated changes and what
were the specific affected materials.
iii) E.g. 1 “Interact with Guests” - Lesson plan has been
contextualised.
iv) E.g.2 “Prepare for Service” – Assessment plan has been
amended.
v) Tracking mechanism used by ATO
2. Plan-Do-Check-Act (PDCA)
i) Demonstration of establishment in PDCA.
3. Outcome Evaluation
i) Update on actions taken based on organisation audit or previous CIR
ii) Continuous review and feedback mechanisms in place and
implementation
iii) System to monitor and analyse L1 to L4 outcomes with detailed
analyzed of collected data
4. Adult Educator Management
i) Performance management of trainers/assessors
ii) Training and development of trainers/assessors
iii) Description of supervision mechanism for trainers and
assessors under supervision
5. WSQ Related Administrative System
i) Learner support system that covers course advisory, progression
pathway, course schedule, fees and refund policy, retest/appeal
procedures, career advisory, placement support (if applicable)
ii) System in place to ensure accurate and timely submission of
assessment results
iii) WSQ certificate printing (if applicable) and distribution system
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 37

iv) Course training and assessment records
6. Viability of Organisation
i) Update on governance and management of ATO
ii) ATO’s financial health for the past one year
7. Free from Breach of Terms and Conditions
i) Records of ATO for complaints or compliments received by ATO or on
ATO
*Presentation should be kept to 15 mins.
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 38

Annex E: CIR Outcome Measures and
Grading
Result of CIR
(can be either A or B)
A) Cleared CIR with no
action required
Not Addressed Within
Advised Timeline
Outcome
Not Addressed After
Reminder
Continuation of approval status with new validity
period, with or without new applications
B) CIR with action required Reminder letter with
rectification within 5 working
days
Revoke of WSQ ATO
status.
Ad-hoc CIR
(outside of CIR
schedule)
Not Addressed Within
Advised Timeline
Not Addressed After
Reminder
C) Triggered by Other
Factors
i) Complaints with
action required
ii) Misrepresentation
iii) Inaccurate or late
report submission
iv) Breach of WDA
Terms and
Conditions
Reminder letter with
rectification within 5 working
days
Revoke of WSQ ATO
status.
Issued by: Quality Assurance Division
Singapore Workforce Development Agency
Version: 1.9
Effective: 13 Sep 2012
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 39

Version Control
No Date Changes Made Version No.
1 31 May 07 Fee revised to $315 (pg. 6) 1.2
2 21 Sep 07 Fee revised to $321 (pg. 6) 1.3
3 5 Jan 09 Revision of areas of review 1.4
4 1 Aug 09
5 1 July 10
6 1 Sep 10
7 4 Nov 11
Added Essential Elements of Review
(pg.12)
Added System-Evidence -Outcome
Approach (pg. 13)
Added Annex B: Self-assessment
Checklist (pg. 25) 1.5
Removed NSRS
Added Example of S-E-O (pg.13)
Added Annex C: 2 year plan (pg.35)
Added CIR Grade (pg.40) 1.6
Revised submission timeline for self
assessment checklist and 2 year plan
(p.19)
Added criterion 1.2 into “Course
Quality assurance system”
Revised Outcome Measures, i.e.
“Revoke of WSQ ATO status” under
column “Not Addressed After
Reminder” (p.39) 1.7
Revised timeline for issuance of
certificates
Added Appeal Process (p.21) 1.8
8 13 Sep 12 Revised webpage link (p.6) 1.9
© Singapore Workforce Development Agency
WDA-WSQ-CIR-GUIDE Effective date: 13 Sep 2012 version 1.9 Page 40