VET Assessment Report - National Skills Standards Council

Investigation into industry expectations ofVocational Education and Training AssessmentFinal Report - June 2008ContentsExecutive summary 4Background 8Project objective and outcomes 8Research focus 81. Introduction 8Project methodology 9Limitations to the research 92. Literature review 10Employer perceptions 153. Industry perspectives 15Key Themes 194. Industry engagement inassessment processes 235. Strategies to increase industryconfidence in assessment 24Linking assessment to enterprise needs 266. Best practice – Case studies 26RPL processes 28Customisable assessment tools0Use of self assessment0Appendix I – Submissions Received 31Appendix II – Data 31Appendix III – Survey respondents 32Appendix IV – Interviewees 33Appendix V – Resources 34Contact DetailsPrecision ConsultancySuite 2/Level 5167 – 169 Queen StreetMelbourne VIC 000Telephone: (0) 9606 0118Fax: (0) 9606 0119Email: info@precisionconsultancy.com.auWebsite: www.precisionconsultancy.com.auABN: 0 056 617 728DisclaimerThis work has been produced on behalf of the NationalQuality Council with funding provided through the AustralianGovernment Department of Education Employment andWorkplace Relations and state and territory governments.The views expressed herein are not necessarily those of theAustralian Government or state and territory governments.Recommendations are under consideration by the NQCbut it should not be assumed that they will necessarily beimplemented in the future.NQC VET ASSESSMENT REPORT 2008 -

Executive summaryAssessment in the vocational education and training(VET) system is defined most recently in Australian QualityTraining Framework (AQTF) 2007 as the judgement ofevidence against the standard of performance expected inthe workplace. It is therefore reasonable that industry hasexpectations of the assessment process and its outcomes.The research design included the collection of data froma range of sources, including a literature review, a callfor submissions focussed on peak bodies and industryassociations, an online survey of enterprises and telephoneinterviews with employers, and other informants such asindustry associations and group training organisations.This research was carried out on behalf of the National The data in this report is the outcome of more than 60Quality Council (NQC) Training Packages Standinginterviews, the responses of approximately 80 people toCommittee with the prime purpose of examining thean online survey and submissions from peak bodies andperceived mismatch between assessment outcomes and industry associations. Key findings are listed below, and areindustry standards. The NQC are seeking to establish a strong accompanied by issues for consideration.evidence base for future work to build industry confidence inthe outcomes of VET assessment.Level of involvement with assessmentThe AQTF 2007 uses the following definition of competency: 92% of respondents regarded themselves as having agood or excellent understanding of assessment. The levelThe broad concept of industry competency concernsof involvement with assessment processes and decisionsthe ability to perform particular tasks and duties to thevaried widely amongst respondents, and individualstandard of performance expected in the workplace.respondents often commented in interviews that they wereCompetency requires the application of specified skillshighly involved in some areas, and less involved in others.and knowledge relevant to effective participation in anEnterprises were more likely to want to be involved directlyindustry sector or enterprise.in the assessment of their current workforce, rather thanWhether an experienced employee being trained andtheir potential or future workforce. They felt that they oughtassessed in the enterprise, or a full time VET in schoolto be more involved in the assessment of apprentices andstudent seeking skills and qualifications before entering the trainees rather than assessment related to totally off-the-jobworkforce, the definition of competency, the definition of or institutionally based training. Some case studies of goodassessment and the ‘rules’ of training packages and AQTF practice in industry engagement in assessment processes are2007 apply equally. In essence, the VET system makes the provided in the body of the report. Strong communicationsame claim as to the ability of the person who completes between the provider and the enterprise are a key aspect ofa VET qualification – that they can perform particular tasks good practice. International experiences with programs suchand duties to the standard of performance expected in the as Unionlearn in Britain suggest possible mechanisms whichworkplace – regardless of the context in which they gained may aid communication between enterprises and trainingthose skills. Employers disagree with this claim, and have providers.different expectations depending on where the skills weregained and how the candidates were assessed.Tensions about where and how to assess whether acandidate meets the expectations of the workplace havebeen ongoing. For some of the more traditional trades,where apprentices typically work with skilled tradespeoplein order to learn their craft and attain their qualifications,engagement with industry and the identification ofworkplace requirements are more straightforward. Howeverfor institutional pathways, where students undertake theirlearning and often their assessment in classroom settings,and where there is no direct link to a workplace, it is moredifficult to ascertain whether students can meet therequirements expected in industry.This research arose out of reports which raised concernson the part of industry and regulators about the qualityand integrity of registered training organisation (RTO)assessment, particularly perceptions that learners are beingdeemed competent inappropriately. Because of other workfocussing on licensed occupations, this project only collectedinformation about non licensed areas.Issues for considerationWhat further work would be required to expand andenhance the positive case studies of industry engagementin assessment presented here, and to promote thosewidely? Could this be part of the NQC’s role in supportingthe implementation of AQTF2007?Could industry expectations of assessment be quantifiedand measured over time by the addition of a specificquestion to the NCVER Employer satisfaction with VETsurvey?Could relationships between enterprises and providers beimproved through developing a model similar to the BritishUnionlearn model (in Britain unions work with employersto improve enterprise engagement with learning andskill development and to match learners’ developmentprograms with current work requirements and individualaspirations)? - NQC VET ASSESSMENT REPORT 2008

Different expectations in differentcircumstancesThere are different expectations of assessment that isundertaken within the workplace, or in partnership with theworkplace, and assessment that takes place at the RTO or ina simulated environment. Industry was much more likely tobe satisfied with assessment processes and outcomes whenthere was a strong partnership between itself and the RTOor where it had developed its own enterprise RTO. The closerthe industry person interviewed was to the decision makingprocess of assessment, the more satisfied they were. Somesubmissions by industry associations focussed particularlyon institutionally based training for specific occupationsand dissatisfaction with the judgement of competencyin those circumstances. Other research, mentioned in theliterature review found that 90% of employers surveyed“valued qualifications in at least one area in managing theirbusiness”. Qualifications were considered more importantfor professional, technical/trades and managerial employeesthan for people in operational type positions. The reportconcludes with a challenge to the VET system to ensure thatit provides “an appropriate match in the array of trainingproducts and services” to meet the complexity of individualenterprise needs.Issues for considerationHow can these different expectations be met?Can the NQC develop a strategy which recognizesthe different purposes and contexts of training andassessment? In particular industry expectations of broadpreparatory programs delivered in institutional settings,and more narrowly focused, occupational on-the-jobtraining are vastly different.Can a one size fits all approach to the definition ofassessment and expectations of outcomes be sustained, orwill it lead to an overall loss of integrity in the system?What difference have recent COAG RPL projects made to thepractice or update of RPL?ConsistencyIssues related to consistency and reliability were mentionedby many respondents. These ranged from criticisms abouta lack of consistency (perceived or real) between assessorsconducting assessments to variations in assessmenttechniques and tools. There is a perception that thereis inconsistency in assessment practices between RTOs,some of which may be eliminated by the use of commonassessment tools and or by moderation and validationacross providers. Many respondents claim to have builtstrong trusting relationships with particular RTOs, yet arestill sceptical about the capacity of all RTOs to deliver thesame level of quality in assessment practice. Although therewas general support for increased consistency, there weredifferent opinions about how best to achieve it, and to whatextent it can be achieved while still permitting flexibility atenterprise level.The issue of quality around the TAA0 Training Package hasbeen raised in a number of forums.Issues for considerationWhat, if any, changes are required to the content, structureand delivery of the TAA04 Certificate IV in Training andAssessment are required to increase industry confidenceand to widen the skill development options available toworkplace assessors? Should a national strategic audit ofTAA04 be conducted to provide more data?The AQTF 2007 (and its predecessors) requires RTO assessorsto ‘have relevant vocational competencies at least to the levelbeing delivered or assessed’How can the VET sector more clearly specify the vocationalindustry competencies requirement for assessors in RTOs tohave current vocational competencies?Should training and assessment undertaken with a TrainingContract in place require ‘dual sign off’ by the RTO assessorand a workplace supervisor?How could the NQC investigate further internationalmodels of assessment moderation (particularly those in usein the UK and NZ) where the agency that sets the standards(the equivalent of the industry skills council) is involved inmoderating assessments of the standard in appropriateindustries?Risk management in assessmentIssues arose in relation to the relative risk of different areasof competency, or groups of skills. For example, some unitsor elements, or combinations of units are regarded byindustry as critical, yet current requirements dictate that allwill be treated equally. Industry would like to see a way ofdifferentiating between crucial competencies that must havethe highest level of rigour about “assessing to the standard ofperformance expected in the workplace” and those that areeither less important, or where the expectation is that the skillwill be developed by experience in the workforce over timeand is not expected in new employees. This perhaps linksto the consistency area above, but in some circumstancesenterprises wish to highlight particular competencies ascritical and desire a higher level of engagement in theassessment process for those areas.NQC VET ASSESSMENT REPORT 2008 - 5

Issues for considerationIs it desirable for the VET system to take a ‘riskmanagement approach’ to the development of TrainingPackages to ensure that where units or elements, orcombinations of units, are regarded by industry as “critical”,they are clearly identified and accompanied by prescribedassessment tools?Strategies to increase confidence inassessmentWhen the process of assessment is examined in the broadsense there are many possible points of intervention whichmay lead to increased confidence.Although assessment is not entirely a linear process, it ispossible to set out these potential points of intervention asa continuum. They begin with the defining of the unit ofcompetency and the assessment guidelines in the originaltraining package, and go on to assessors as interpretersand users of those standards and the ways in which thoseindividuals are trained and assessed themselves. Other pointsof intervention include the interactions and communicationbetween RTOs and industry, and the work context withinthe enterprise. As assessment is the judgement of evidenceagainst a recognised standard or benchmark; the firstpotential point of intervention is in the setting of thatstandard. Following a review of the literature, a suite ofpotential strategies to increase industry confidence inassessment were developed. These strategies were testedwith participants who were asked to comment on theirsupport for each approach. There was general support thateach of the following potential interventions would increaseconfidence in the outcomes of assessment.Potential points of interventionPotential Point of interventiontraining package developmenttraining and ongoing professionaldevelopment of assessorsvalidation and/or moderationrelationship between enterprisesand rtos (for current employeesundertaking training)Other suggestionsaction required• Clear conditions for assessments written into evidence guides of competencies• Identification of priority units of competency where consistency would be enhanced by theaddition of related, standardised national assessment tools.• Delivery of training/short courses for assessors in the workplace to assist them in supportingor working with RTO assessors in the identification of relevant evidence and agreeing onenterprise contextualized units of competency• Assessors regularly attending professional development activities with industry people• More quality in the Certificate IV in Training and Assessment.• Assessors meeting with each other and industry representatives to discuss approaches, agreecommon assessment tools and check consistency• Endorsement or registration of assessors by an appropriate body in some industry areas.• Joint sign off on assessments (enterprise and RTO)• More detailed AQTF audits of assessment processes, involving industry representatives andincluding an examination of the assessment tools used• Better communication between the enterprise and the RTO about processes, expectationsand information provision.• Strategies for assessors to update their industry knowledge and ongoing monitoring of theeffectiveness of this• Better monitoring of the requirement to assess ‘over a period of time’ and more clarificationabout these time requirements• Better use of workplace evidence by taking workplace tasks and mapping them tocompetencies, rather than asking the enterprise to find evidence in the workplace to matchthe competency. This could include processes to recognise enterprise based, non accreditedtraining so that the outcomes of such training could lead to statements of attainment and orqualifications. Work which is being undertaken in Britain could help here• Undertaking more holistic assessment that encourages the collection of ‘workplace evidence’• Communication of the benefits of RPL to the enterprise, particularly in terms of return oninvestment.6 - NQC VET ASSESSMENT REPORT 2008

By collating all the data from the research we canconceptualise an ‘ideal’ assessment system, atleast in terms of meeting industry expectations,which comprises:1. A very well written training package with clearlyexpressed standards and stronger assessment advice,including clarity about the required conditions ofassessment2. Prioritisation of critical units of competency in terms ofthe level of specificity applied by developers, and thelevel of scrutiny applied by assessors and by auditors. Competent assessors with current industry knowledgeand experience preferably working in partnership withthe enterprise. Assessors working with the national competencystandards to contextualise the standards for individualenterprises or workplaces5. Moderation of assessments including input from industry,assessors and Industry Skills Councils as developers of theTraining Packages.6. Assessment processes to be rigorous and to add value tothe enterprise as well as to the individual learner7. Communication between the RTO and the enterprise tocontribute to a partnership approach8. A system which encourages consistency within andacross RTOs in assessment decisions9. In some cases, a way of differentiating betweeninstitutional pathways preparing individuals for theworkplace (where it is expected they will developcompetency over time), and assessment of currentemployees10. An auditing system which can remove poor performingRTOs from the market place.Issues for considerationHow best can this ‘ideal’ be achieved?NQC VET ASSESSMENT REPORT 2008 - 7

1. IntroductionBACkgRounDPRojECt oBjECtIVE AnDoutComESDebate about how best to ensure quality in assessment, andparticularly how to build employer confidence in assessment,The project objective was:has been alive for many years in Australia. Yet there arevarious assumptions about what is commonly understoodThrough research, consultation and analysis, to establishto constitute quality in assessment, and whether the issuesa strong evidence base for future NQC work to buildabout quality in assessment are the same across differentconfidence in the outcomes of VET assessment.industries and work contexts.Competency-based assessment was described in Australia in Project outcomes1992 by the National Training Board in this way: • Provide indicative information about industry (includingboth employer/employee) perceptions and expectations,The concept of competency focuses on what is expected of across non-licensed industry sectors, around VETan employee in the workplace rather than on the learning assessment processes and outcomesprocess; and embodies the ability to transfer and applyskills and knowledge to new situations and environments. • Identify examples of best practice around industryThis is a broad concept of competency in that all aspects engagement in VET assessment processesof work performance, and not only narrow task skills, areincluded.…The definition has since been reworked, yet the emphasis onworkplace expectations has been maintained. The AustralianQuality Training Framework (AQTF) 2007 uses the followingdefinition of competency:The broad concept of industry competency concernsthe ability to perform particular tasks and duties to thestandard of performance expected in the workplace.Competency requires the application of specified skillsand knowledge relevant to effective participation in anindustry sector or enterprise.• Identify strategies for increasing industry confidence in VETassessment; and• Identify the level of industry support for various strategiesfor increasing industry confidence in VET assessment.RESEARCh foCuSThe research focus was clearly on industry perceptions andexpectations. Key research questions were:• What does industry expect of individuals with VETcredentials in terms of workplace performance?The tensions about how to assess whether a candidate meetsthe expectations of the workplace have been ongoing. Forsome of the more traditional trades, where apprenticestypically work with skilled trades people in order to learn theircraft and attain their qualifications, the expectations of theworkplace are more identifiable. However for institutionalpathways, where students undertake their learning, and oftentheir assessment in classroom settings, it is more difficultto ascertain whether students can meet the requirementsexpected in the contemporary workplace.Whether a current employee, or a full time student seekingskills and qualifications before entering the workforce, thedefinition of competency, the definition of assessment andthe ‘rules’ of training packages and AQTF 2007 apply equally.The VET system’s aim is to produce graduates with skillsthat meet current workplace requirements. This projectinvestigated whether employers believe that the VET systemis producing graduates who are deemed to be ‘competent’according to the above definition as a result of assessmentprocesses, and if not, why not.• Are these expectations reasonable?• Are these expectations being met?• What issues does industry see as affecting their confidencein the outcomes of VET assessment processes?• What evidence is available to support industry perceptionsof VET assessment processes and outcomes?• To what extent do industry perceptions about theoutcomes of VET assessment relate to issues outside theassessment process itself (e.g. training package contentand/or advice and development processes, assessorqualifications/experience)?• To what extent and in what ways is industry currentlyinvolved in VET assessment processes?• How might industry be more effectively engaged toimprove the validity and reliability of VET assessments?• To what extent would industry support engagementstrategies designed to increase industry confidence in RTOassessments?8 - NQC VET ASSESSMENT REPORT 2008

PRojECt mEthoDoLogyThe research design included the collection of data from arange of sources.literature reviewThe project commenced with a review of current literatureon assessment and the factors affecting confidence inassessment outcomes. It also looked at other research aboutemployer perceptions of VET. The key trends in the literatureare summarised in the report.call for submissionsA Call for Submissions was disseminated throughvarious channels including NQC members. A total of 12submissions were received. The list of organisations thatprovided submissions is provided in Appendix I. Most of thesubmissions came from peak bodies or industry associations.online surveyThe original intention of the project brief was to conducta national, cross industry, online survey of approximately2,000-2500 employers/employees with the intention ofgetting 500 respondents. Two surveys were developed, oneaimed at employers or their representatives, another aimed atemployees.An online survey was developed and publicised through arange of communication channels. 82 representatives from atotal of 52 organisations participated in providing responsesto the survey. There were only a handful of respondents tothe employee survey. The list of respondents’ organisations isprovided in Appendix II.interviewsA standardised structured interview tool was developedto gather quantitative and qualitative data. Industry SkillsCouncils were requested to nominate up to ten industrybased people who could be interviewed. Industry peakbodies provided the names of other informants andothers were suggested by those interviewed. All ofthose interviewed were based in Australia. The researchmethodology did not extend to those RTOs deliveringtraining overseas. Sixty-two one-on-one interviews wereconducted with representatives from different organisations.Most interviews were conducted by telephone with a smallnumber of face to face interviews and meetings.LImItAtIonS to thE RESEARChReaders should be wary of comparing outcomes of thisresearch with larger surveys, in particular the biennial surveyof employers undertaken by NCVER. Since 1995 NCVER hasconducted surveys aimed at employers with a focus on theVET system. In 2005 and 2007 the Survey of Employer Use andViews of the VET system surveyed the ‘manager responsiblefor staff training’ using computer assisted telephoneinterviews. Approximately 700 interviews were conductedin 2007 using the Australian Business Register as thesampling frame. Such a large sample allows for differentiationof results across states industry area and other variables.By contrast this was a relatively short project which focussedon Australian employers’ opinions of assessment. Thisreport provides quantitative and qualitative data, togetherwith some examples of best practice. It does not provide acomparative analysis of the efficacy of assessment practicesin RTOs.Other factors to be acknowledged in this project follow.informed respondentsAlthough efforts were made to contact all types of businessesand people at different levels in the organisation, in manycases the people who availed themselves for interviews andcompleted the surveys were often in training managementor human resource management roles. This meant that theywere typically more knowledgeable than their businesscolleagues about the training system, and could provideinformed comment about the system’s strengths andweaknesses from their own business perspective. It alsomeans that the data about perceptions is not wholly typicalof people across all job-roles in Australian business.response rateWhen it become apparent that only a low number ofenterprises were responding to the online survey, themethodology was changed to increase the number oftelephone interviews, thus enabling more qualitative datato be collected and in particular more examples of thecharacteristics of assessments likely to increase industryconfidence in assessment. It provides data from a crosssectionof industries, and key informants with extensivenetworks have provided input through submissions andinterviews.General perceptions versus individualexperienceRespondents were asked to comment on their perceptionsof assessment practices. In many cases respondents advisedthat there were vast differences between their perceptionsof assessment process in the VET system and their ownexperiences, because they had located good RTOs andensured that assessment practices addressed their businessneeds. Many respondents and interviewees claimed thatthey had found ways to ensure that assessment practicesmet with the high standards of their business, but that theydid not believe that all RTOs and assessment practices wereequally thorough.NQC VET ASSESSMENT REPORT 2008 - 9

2. Literature reviewThe purpose of this literature review is to explore relevant The 2007 survey conducted telephone interviews withnational and international literature on industry perceptions ,701 employers. The report on the 2007 survey resultsand expectations around VET assessment processes and supports the AiGroup results about skills shortages, revealingoutcomes.some skill shortages in Australia, with .% of employersexperiencing difficulty in recruitment, particularly largeThe literature on industry expectations of the Australianemployers and mining employers.VET system, and also international literature, is relativelybroad and does not often separate assessment from the This difficulty in recruitment increased from 0.6% in 2005.wider training issues, particularly when discussing industry Employers cited recruitment difficulties as being a result of aengagement. Contemporary VET strategies are focusing shortage of skilled people in the industry, as well as limitedon the dual challenges of providing on-the-job trainingapplicants.and the customisation of this training to workplace needs.While most of the discussion has been around delivery, the This skill shortage is not confined to Australia. The Leitchchallenge extends to assessment strategies. An assumption Review, Prosperity for all in the global economy – world classhas to be made that discussions about the expectations and skills (2006), considered the United Kingdom’s “long-termsatisfaction with the VET system often include both deliveryskills needs”. An ageing population and low “economicallyand assessment.valuable” skills were some of the issues highlighted in thisProviders of VET services have multiple responsibilities in report.meeting their stakeholders’ needs. The flexibility to meetworkplace needs may be critical to employers; the credibilityThe New Zealand Department of Labour, through itsof the VET system necessitates that there is quality assurance quarterly Survey of Business Opinion conducted by thein the delivery and assessment of VET services that lead to New Zealand Institute of Economic Research also reports anqualifications.increase in skill shortages, with 6% of firms having difficultyfinding skilled staff in the December 2007 quarter, up fromThis review begins with an examination of the literature1% in the September 2007 quarter. The biggest increase inaround skill shortages and employer and employeeskill shortage was reported in the building sector, althoughexpectations of VET, followed by an examination of industry it was high in all sectors, and it is expected that labourand government involvement in potential solutions toconditions “will remain tight over 2008”, with “skill and labouridentified issues.shortages … set to remain a major issue in the New Zealandeconomy”.Skill shortagesWhen examining industry expectations of VET, research industry and employee expectations of vetshows some issues driving these expectations, both inDespite the expected skill shortages, NCVER research intoAustralia and internationally. A critical issue is that of skill employers’ use and views of the VET system reveals thatshortages and the need for skill development.the proportion of employers in Australia, particularly smallemployers, using the VET system declined from 57.1% in 2005The Australian Industry Group (AiGroup) represents 10,000 to 5% in 2007, although the percentage of employers withemployers in a range of industries including manufacturing, apprentices and trainees increased, from 28.2% in 2005 toconstruction, automotive and transport, and aims to29.1% in 2007. The NCVER research found that over the nexthelp them become more competitive domestically andthree years, substantially more employers believe the skillinternationally. Its report World Class Skills for World Class shortage will increase compared to those who believe it willIndustries (2006) surveyed 526 employers, both large (60%), decrease (NCVER, 2008).and small to medium enterprises (0%). A large proportion(6%) of respondents came from manufacturing, withThe NCVER research revealed that about half (9%) of2% from service industries. This research shows that most employers use “unaccredited training”, outside the VETemployers reported difficulty in recruiting people with the system (AiGroup, 2006). Even when nationally recognisedskills needed, particularly middle level and trade skills, as well training was available, a large proportion of employers didas soft skills. Inadequate skills in existing employees were also not use it because of cost or flexibility issues, whilst othersreported.did not know accredited training was available. One of thedifferences between nationally recognised training andThe National Centre for Vocational Education Researchunaccredited training is that nationally recognised training(NCVER), funded jointly by the Commonwealth and state and always includes assessment. However, this NCVER surveyterritory governments, regularly collects information about does not explore attitudes towards assessment, so it is notthe various ways employers meet their skill needs, how they possible to draw any conclusions directly from this dataengage with the VET system, and their level of satisfaction about assessment (or the lack of quality in assessment) beingwith the products and services of this system in the previous a factor in the choice to use unaccredited training.12 months. Surveys have been conducted every two yearssince 1995, although the focus of the surveys has changedsomewhat over the years.10 - NQC VET ASSESSMENT REPORT 2008

The 2007 NCVER survey also found that despite the slightdecline in use of the VET system, since 2005 there hasbeen an increase in employers’ satisfaction with all formsof training, and satisfaction is high. The satisfaction ratefor employers with jobs requiring vocational qualificationsis 80.8%; for employers with apprentices and trainees thesatisfaction rate is 8.%, and for employers using othernationally recognised training, 80.5 % are satisfied. However,the increase in satisfaction with unaccredited training wassmaller, the percentage of satisfaction of employers using thistraining was highest, 92.5%.Other in-depth industry specific studies have beenconducted recently, commissioned by the National TrainingQuality Council (NTQC – now the NQC). The first in 200was the National Strategic Industry Audit of Training whichfocused on the transport industry, and the second in 2005investigated the hospitality industry.The first audit into the transport industry (DEST, 200) wasundertaken to allow the NTQC to obtain a national pictureof the operation of the National Training Framework in oneindustry. It was determined that the audit should concentrateon the key risk areas of fully on the job training andapprenticeships and traineeships. Sixty-six RTOs were auditedand 216 employers and 1,900 trainees surveyed.that there is a disparity between the skill level of apprenticesand trainees provided by the training system and industryrequirements”.The audits of RTOs found “significant non complianceagainst AQTF Standard 8.2 – RTO Assessments”, includinglack of holistic assessment, assessments not always beingconducted in a realistic workplace, and lack of sufficientevidence, unreliability of assessment across assessors, andlack of validity of assessment tools and outcomes.One area that employers were asked about was related toassessment strategies. 71.% of employers surveyed ratedthe effectiveness of the assessment process as either “veryeffective” or “effective”. However, 16.7% found the process“ineffective” for a number of reasons including that therehad been no assessment, they had not seen any assessors,the apprentice/trainee was not skilled/qualified whensigned off, and employers received no information on thetrainee’s progress. Satisfaction with the skills and capabilitiesof the trainers and assessors was 75.%, and satisfactionabout when apprentices or trainees were assessed ascompetent, they could actually do the job was 7.7%. So,although employers rated reasonable to moderate levels ofsatisfaction, the report concludes that there are some areasof concern.The audits identified serious compliance issues with AFTF Apprentices/trainees reported a high level of satisfactionStandard 8 – Assessment and Standard 9 – Learning and (88%) with the quality of training provided by the RTO; withassessment strategies, across the majority of the registered 90% being satisfied with the fairness of the assessmenttraining organisations audited. This audit also showed a low process, and 92% of respondents believed that they hadrate of Recognition of Prior Learning (RPL).gained the skills they need to do their job. However, therewas concern expressed about the level of informationHowever, overall employee satisfaction with the quality of provided about assistance with literacy and numeracy, howtraining and assessment provided by registered training to obtain RPL, the expectations of the employer, the role oforganisations was high (78%) and 87% of employersthe trainer, and methods of assessment.considered that when trainees were assessed as competent,they could do the job. Some reasons for dissatisfactionThis research identified some good practice examples inincluded comments about the trainer, for example thefour registered training organisations that negotiated aorganisation did all the training with the trainer onlydirect relationship between workplace skills requirementscompleting the forms for assessment. A small percentage and the subsequent training and assessment. The process(5%) was dissatisfied with the skills and capabilities of trainers of determining what the “employer needed” was seenand assessors.as superior to a process based on “what the trainingThe trainees were generally satisfied with the quality oforganisation could deliver”.training and assessment provided by the registered trainingorganisations, with 60% being “very satisfied” with the fairness flexibilityof the assessment process, and a further 29% being “fairly The AiGroup research (2006) shows that there has been asatisfied”.lack of growth in training investment by enterprises, leadingto skill shortages and gaps, with barriers including theThe second audit, into the hospitality industry (DEST, 2005), constraints of accommodating training around business,has some similar findings about the skills of the workforce but also a lack of flexibility and relevance of the trainingwhich could have an impact on the VET system. An objective system. The report argues that the training system needs toof this audit was to gain employer and student opinion about have greater emphasis on technical, employability and softthe quality of delivery and assessment in the hospitalityskills; that it needs to respond more to industry needs, e.g.industry. Three hundred and sixty-five employers withinclusion of adult apprenticeships, and training for non-entryapprentices and trainees were surveyed by telephone, and level participants; that there needs to be flexibility at a systema student survey was made available online, with 7 out of level, e.g. more regular updating of training packages, and9,765 students responding. One of the training challenges provision of more adequate RPL; and that training providers,for the hospitality industry listed in the report is a “perception in particular TAFEs, need to be more flexible and proactive.NQC VET ASSESSMENT REPORT 2008 - 11

In research conducted in 2005 into issues in teachingand assessment in VET, some critical issues in relation toassessment arose. These issues, which remain “the subject ofdebate within the sector” include whether the focus of theassessment process should be the outcome or the process(Mitchell, 2006). This report also identifies other unresolvedissues such as how best to provide services for recognitionof current competence (RCC), grading of assessment andsupport for assessors. These issues all impact on the flexibilityand quality of assessment.In research into TAFE practitioners work with privateenterprise (Harris 2005), it was found that in the broadcontext, in order to meet industry’s call for greater workplace 2007).specialisation of skills and customisation of training andassessment, VET practitioners need to work in differentindicated that there were some issues about the quality ofteaching and support.One criticism was the lack of industry experience of thetrainers, and students felt that more VET staff should besourced from industry. They also expressed dissatisfactionwith the “competent and not competent assessmentmethod”, as it did not recognise higher levels of performance.Many students expected to gain a job after completing aVET qualification, but had some problems with the levelof careers information, remarking that they expected VETinstitutions to “foster relationships with employers and act asa ‘go between’ for students, employers and industries” (DESTqualityand more complex ways. VET practitioners need to be ableto develop the collaborative linkages with key industry or The introduction of the AQTF in 2001, which evolved fromemployer personnel. As well, there is greater demand on the the original Australian Recognition Framework, placed anprofessional capacity of VET personnel to be able not only increased emphasis on quality in the national VET system.tocontextualise the learning in a meaningful way but also An updated version was put in place in 2007, to continueensure the assessment processes and outcomes support the to provide “nationally consistent, high-quality training andlearning.assessment services for the clients of Australia’s vocationaleducation and training system” (AQTF 2007). A changequalificationsin focus of these standards is the focus on the “quality ofservices and outcomes being achieved for clients ratherAlthough some research does not discuss attitudes tothan inputs used to get there”. Assessment has a criticalassessment in particular, it does examine the importance of emphasis in these revised standards, with one of the threequalifications, which intrinsically include assessment. A recent standards ensuring that RTOs provide “quality trainingstudy into the relevance of higher-level VET qualifications and assessment across all of its operations” (Standard 1).for enterprises and employment outcomes for students, This includes continuous improvement of training andto address skill needs, found that “a relevant qualification assessment, ensuring that training and assessment meetstogether with workplace experience is valued by employers” the requirements of the training package or course and(Foster, 2007). This supports an earlier study (Ridoutt et al, that industry is involved in the development of assessment2005) about the value employers give to qualifications, which strategies. Standard 1 also emphasises the training/found that 90% of employers surveyed “valued qualifications assessment and vocational competencies of trainers andin at least one area in managing their business”. Qualifications assessors, and the provision of RPL.were considered more important for professional, technical/trades and managerial employees than for people inRather than being focussed on audit processes, the Newoperational type positions. The report concludes with aZealand approach to ensuring quality assessment ischallenge to the VET system to ensure that it provides “an underpinned by the Principles of Best Practice Moderation.appropriate match in the array of training products andThese principles “assert that best practice moderation occursservices” to meet the complexity of individual enterprise when it is based on partnerships between assessors andneeds.other assessors, and also assessors and moderators” (NewZealand Qualifications Authority in Clayton et al, 200). Withinthis system the “standard-setting bodies” that produce theemployee expectationsEmployees and potential employees of industry also wantthe VET system to meet industry needs. Graduate destinationsurveys consistently find that around 75% of studentsundertake VET qualifications for employment purposes.Overall satisfaction with VET training is high, with 82% of TAFEgraduates rating the quality of their course as “above average”or “excellent” (NCVER, 2001).However, some dissatisfaction of the VET system is apparentin the research. A series of focus groups, Student ExperienceRoundtable, conducted by the National VET Equity AdvisoryTaskforce in 2007 and attended by 55 participants, mainlystudents across the five target equity demographics,unit standards also produce unit standards for assessmentand moderation, and they meet together with assessors tomoderate or validate the standards. Together with trainingproviders, the standard-setting bodies “are responsible forensuring that assessors participate in the designated processfor ensuring validity and consistence in assessment”. Industryis also engaged in this process through involvement in thestandards development.rating training and assessmentThe Institute for Trade Skills Excellence was established in2007 to advance teaching and learning in trades education.A key strategy of the Institute is the introduction of a Star12 - NQC VET ASSESSMENT REPORT 2008

Rating scheme of RTOs, based on a set of evaluation criteria. Contemporary Apprenticeships for the Twenty First CenturyThe criteria apply to teaching departments within the RTO, (2005), they state that the “credibility of a new, competencyrather than the RTO as a whole. Some of the criteria relevant progression apprenticeship model will be derived fromto this study include showing evidence of conductingthe quality and rigour of assessment.” They recommendassessments “in conjunction with the enterprise and learner industry endorsed “assessment-centres” which specialise into reflect holistic work activities”, and employment of trainers recognition for a range of clients including adult apprentices,with “up-to-date knowledge”, “recent industry experience …” the skills of existing workers and catering also for entry leveland “a good understanding of the skill and workforce issues apprentices.confronting … industry” (www.tradeskills.com.au).The Australian Chamber of Commerce and Industry (ACCI),in ACCI Policy Review (February 2008), highlights therecognising non-accredited trainingimportance of assessment for employees to maintain andIn the UK, enterprises are being encouraged to recognise improve quality standards within an industry. If a person isthe value of their own training. The UK report Implementing assessed and issued with a qualification without having thethe Leitch Review of Skills in England (2007), acknowledges satisfactory skills, employers will need to retrain the personthat some companies “offer outstanding training”. By having in the workplace, which is costly. It would also “drive downenterprises recognise what they do well, individuals could standards of industry”, and give the training industry a “badgain credit for “what they achieve through their company’s name”. ACCI argues that the necessary relevant vocationaltraining schemes, and put that towards a nationally-competencies of VET assessors be maintained through therecognised qualification”. This promotes the development of RTO “supporting them (assessors) in meaningful engagementtransferable skills that have currency across a range of jobs. with industry and relevant professional bodies”. ACCIsuggests that in order to encourage the workplace to beTo encourage employers and employees to commit to the actively involved, the RTO approaches industry directly fordevelopment of workforce skills, the UK Government isinvolvement in the assessment process. They recommendpromoting the take up of the Skills Pledge. The Skills Pledge, a set of principles, including up-skilling and training ofgives employers “an opportunity to demonstrate theirworkplace supervisors to contribute to the assessmentcommitment to investing in the skills of their employees. process, and sign-off of competency by employers. ThisBy making the Skills Pledge, employers are investing in their would encourage assessment to be conducted on-theemployeesand in their business; supporting their employees job and “in a more meaningful way that suits businessto increase their skills will also support increased productivity requirements”. ACCI also encourages industry “to be moreand profitability for their business” (DIUS 2007). Employers assertive” about a collaborative approach to the assessmentwho take the pledge, can access funding to training.process and argues that the audit process is critical toengender employer confidence in assessment.encouraging relationships betweenenterprises and the training systemfuture trendsTrade unions have also played a role in raising the profile The Organisation for Economic Co-operation andof quality training and assessment in the UK. In 1998 the Development’s (OECD), publication Going for Growth (2007)Government funded Union Learning Fund (ULF) wasprovides an overview of structural policy developmentsestablished to promote activity by trade unions in support in OECD countries from a comparative perspective. Basedof creating a learning society. One of the key objectives of on a broad set of indicators of structural policies andUnionlearn, funded by the ULF, is to work with employers performance, this publication examines recent progressto improve their engagement with learning and skills,made in implementing policy reforms for each OECDpromoting learning to match learners’ starting points and country. The Australian VET system is seen as a mechanismcurrent needs and aspirations (www.unionlearningfund.org. that can provide not only a work-related qualification but alsouk). Within enterprises, learning representatives (Enterprise access to direct experience in the modern workplace andLearning Representatives – ELR) are recruited to encourage an understanding of the world of work - key ingredients forthe take up of learning opportunities. In turn they encourage ongoing employability.and promote good relationships between the company andtraining providers. This scheme has not only encouraged Most recently, the Australian Government has increasedworkers into learning programs and raised the skill base of investment in skills training to meet identified skills shortagesthe UK, but it has also set up a productive management through its Skilling Australia for the Future initiative. Thisand union dialogue on learning and made advances ininitiative includes an expanded role for Industry Skillsestablishing a culture of learning at work. A similar scheme Councils (ISCs) to contribute to workforce development. ISCsoperates in New Zealand.will be funded to provide industry intelligence and adviceto Skills Australia, government and enterprises on workforceindustry involvement in assessmentdevelopment and skills needs. Under the expanded role, ISCswill conduct an environmental scan to provide an overviewThe AiGroup considers assessment to be a criticalof the workforce issues in the industries covered by theircomponent of the apprenticeship system. In their reportjurisdiction.NQC VET ASSESSMENT REPORT 2008 - 1

They will also be required to assist enterprises undertakinga training needs analysis, to closely examine the skills ofcurrent workers, the current work of the enterprise andareas for potential expansion. The ISC will then be able tomake recommendations for how enterprises may improveproductivity by identifying the training that employeesrequire to upskill. This role will include assisting theenterprise to make informed choices about qualifications,skills recognition and gap training, and about which RTOsare best placed to provide what is required (DEEWR, 2008).Information gathered through training needs analyses willinform updated environmental scans and training packagedevelopment. This change to the role of ISCs does notappear to include the NZ approach of having the standardsdevelopers included in moderation of assessments.SummaryA search of the literature shows that the major surveysof employers’ opinions about VET focus only broadly onthe outcomes of assessment, and the characteristics ofassessment processes that provide confidence are not easilyidentifiable.Mechanisms in other countries such as the recognitionof unaccredited training in the UK and moderation ofassessment in NZ by industry bodies equivalent to our ISCs,offer potential for further investigation as strategies that mayincrease industry confidence in assessment outcomes.1 - NQC VET ASSESSMENT REPORT 2008

. Industry perspectivesThis section details both quantitative and qualitativedata provided through an online survey, interviews andsubmissions.Participation in apprenticeshipprograms, traineeships or hired VEtgraduates• 11 people provided quantitative data• 8.5 % of respondents were from organisations that had• 62 people provided data through interviews participated in an apprenticeship program in the past 12months• 79 people provided data through the online survey.• 66% of respondents were from organisations that had• 12 organisations provided submissionsparticipated in a traineeship in the past 12 months• Qualitative data was gathered through one-to-one andtelephone interviews.Appendix I lists the organisations that provided submissions,Appendix II lists the organisations of staff who participated inthe online survey, and Appendix III lists the organisations ofstaff who participated in interviews. Data from the telephoneinterviews and the online survey is included in Appendix V.Industry areasIn both online surveys and interviews, respondents wereasked to nominate their industry, using Australian Bureau ofStatistics (ABS) categories. The data covered all industry areas.• Property and Business Services had the highest number ofrespondents (9.9%)• Wholesale Trade had the lowest number of respondents(2.0%)States and territoriesInterviewees were asked to advise the states and territories inwhich their business operates (rather than in which state orterritory they were located).Note: many businesses operated in more than one state orterritorynumbers of staff employed in Australia• 51% of respondents and interviewees were frombusinesses employing more than 200 people. 22% werefrom businesses employing 20-200 people and 27% werefrom businesses employing less than 20 people. N=1Note that a number of respondents were from industrybodies and while they had small numbers of employees,they were representing the views of organisations ofvarying size.• 77% of respondents were from organisations that had hiredsomeone with a VET qualification in the past 12 months.Level of understanding ofassessment• 92% of respondents regarded themselves as having anexcellent or good understanding of how VET qualifiedemployees are assessed.EmPLoyER PERCEPtIonSIncreased participation inassessmentRespondents were asked whether they would like to be moreinvolved in the assessment process.• 50% of respondents stated that would like to be moreactively involved in the assessment process.• The 50% who did not wish to be more involved in theassessment process mostly stated that this was because:• They were already very involved in assessment and so didnot need to increase their participation in assessment• They felt that pressure of time would not permit them tobe more involved in assessment.• They were industry bodies and so this was not part of theirrole.Most of the interviewees however believe that industry oremployer involvement is critical to an effective assessmentprocess.Levels of satisfactionRespondents were asked how satisfied they were withthe assessment of people in their organisation who haveobtained, or are working towards, a vocational educationqualification.• 25% of respondents were very satisfied• 1% were satisfied• 19% were neither satisfied nor dissatisfied• 15% were dissatisfied.NQC VET ASSESSMENT REPORT 2008 - 15

in what State or States/territories does your organisation operate?answer options response Percent response countVIC 55.0% NSW 60.0% 6SA 5.0% 27TAS .% 20WA 6.7% 28NT 0.0% 18QLD 5.0% 27ACT .% 20Many interviewees who were satisfied or very satisfiedadvised that this was because they had worked hard tolocate an RTO which was responsive to their business needs.They stressed that their comments about satisfaction did notapply to the system as a whole.Satisfaction was seen to rely on:• The calibre of the RTO (particularly in hospitality)• The currency of the assessors industry knowledge• Whether the assessor took a ‘tick and flick’ approach orused a variety of evidence gathering techniques.Many interviewees advised that their levels of satisfactionvaried with different qualifications. For example, a HumanResources Manager from the manufacturing sector advisedthat he was very confident of the assessment processes forthe people they employed from the traditional trades, butthat he was not at all satisfied of the assessment of peoplewith a Certificate IV in Training and Assessment.Additionally many interviewees commented that while theywere satisfied with assessment generally, this was not thecase with RPL, which was often regarded as burdensome andproblematic.Confidence in outcomes16 - NQC VET ASSESSMENT REPORT 2008Additionally, it is seen that some units of competence andsome qualification structures do not reflect workplacerequirements. Where this is the case the individual may havebeen appropriately assessed against the unit – and assessedas competent – but still not be competent in the workplace,according to the employer.Work readinessRespondents were asked how long they expected it wouldtake someone who had graduated with a VET qualificationto be competent in a job role in their organisation, followingappropriate induction and support.• 16% said immediately• 21% said within 1-2 months• 0% said within -6 months• 26% said between 6-12 months.Generally employers felt that it was part of their role tosupport people to become competent in their workplaceand that it was unreasonable to expect that VET graduateswould be immediately competent in their workplacefollowing induction. Certain job-roles were regarded asneeding more support.Apprentices from the traditional trades, who had moreRespondents were asked how confident they are that an workplace experience, were regarded as typically requiringassessment of competence reflects workplace requirements less time to be able to perform effectively in their work role,for competence.however some interviewees were critical of the skills of ‘fasttracked’• 25% were very confidentapprentices. Variables to consider when ascertaininghow long it takes someone to become workplace competent• % were confidentwere:• 25% could not say whether they were confident or not • The individual and their learning style• 17% were not confident.Again, interviewees explain that their levels of confidencevery much depended on their opinion of the awarding RTOand which qualification. Interviewees stressed that there aregreat variations in assessment practices across the system,and between qualifications.• The extent of specialisation in the workplace / job role• The requirement to work with complex and high-endtechnology that is not available in training institutions

• The amount of previous workplace experience of theincumbentMost respondents do not believe that over-assessment iscurrently a problem for their employees.• The time it takes to get the qualification• The amount and quality of support from the employer.Employers expect to provide orientation and support for newemployees.how often are expectations ofworkplace competence met?• 15% of respondents stated that their expectations ofworkplace competence of people with a VET qualificationwere consistently met• % of respondents agreed or strongly agreed with thestatement that employees are under-assessed• 2% were undecided about whether employees are underassessed• 2% of respondents disagreed or strongly disagreed withthe statement that employees are under-assessed.In summary there are more employer concerns about underassessmentthan over-assessment.Appropriateness of assessment• 50% of respondents stated that their expectations ofRespondents were asked whether they thought assessorsworkplace competence of people with a VET qualification conduct appropriate assessment to determine workplacewere mostly metcompetence.• 6% of respondents stated that their expectations of • 59% of respondents agreed or strongly agreed with theworkplace competence of people with a VET qualification statement that assessors conduct appropriate assessmentwere sometimes metto determine workplace competence• % respondents stated that their expectations of • 21%of respondents were undecided about the statementworkplace competence of people with a VET qualification that assessors conduct appropriate assessment towere almost never met.determine workplace competenceAgain, interviewees remarked that they knew that certain • 2% disagreed or strongly disagreed with the statementRTOs were more likely to produce graduates who met their that assessors conduct appropriate assessment toexpectations. Some interviewees were more critical ofdetermine workplace competence.graduates who had taken an institutional pathway in order toAgain interviewees commented that there were wideobtain their qualification.variations between RTOs and that many of them had workedThere are concerns about the variations across the system. with good RTOs to relevant assessment practices. Problemswere seen to exist when:The quality of recruitment and selection processes was seento be a factor. Larger businesses with more sophisticated • Assessment occurs within the classroom and/or does notselection processes claim that their expectations are more include employer feedbacklikely to be met because they have systems and processes to• Assessors do not have current industry knowledgereview potential employees’ skill levels. That is, they place lessreliance on the qualification as a marker of skills and more • The assessment context does not reflect the reality of theemphasis on their own assessment.workplace, such as being required to work under timepressure and multi-taskAmount of assessmentRespondents were asked whether they believed that theiremployees undertaking VET qualifications were overassessed.• 20% agreed or strongly agreed with the statement thattheir employees were over-assessed• 20% were undecided about the statement that theiremployees were over-assessed• 60% of respondents strongly disagreed or disagreed withthe statement that their employees were over-assessed.• The competency standards and/or training do not reflectcurrent industry practices.Relevance of assessment to currentindustry practices• 61% of respondents agreed or strongly agreed with thestatement that assessment tasks and processes used byassessors are relevant to current industry practices.• 2% of respondents were undecided about whetherassessment tasks and processes used by assessors arerelevant to current industry practices.NQC VET ASSESSMENT REPORT 2008 - 17

• 16% of respondents disagreed or strongly disagreed withthe statement that assessment tasks and processes used byassessors are relevant to current industry practices.Interviewees belonging to organisations that were enterpriseRTOs were more convinced of the relevance of assessmentpractices. Again wide variations between RTOs werereported.Perceptions of flexibility in assessment• 66% agree or strongly agree that assessors are flexible inthe way they conduct assessments.• 17% neither agreed nor disagreed that assessors areflexible in their assessment practices.• 17% disagreed or strongly disagreed that assessors areflexible in the way they conduct assessments.Consistency in assessment• 20% of respondents agreed or strongly agreed withthe statement that RTOs are consistent in the way theyconduct assessment.• 21% of respondents neither agreed nor disagreed withthe statement that RTOs are consistent in the way theyconduct assessment.Perceptions of currency of assessors’knowledge and experience• 7% of survey respondents believed that, in theirexperience, assessors had current industry knowledge andskillsInterviewees were asked a slightly different question. Theirresponses showed that:• 1% believed that, based on their experience assessorsalways had current industry knowledge and experience• 52% believed that, based on their experience assessorssometimes had current industry knowledge andexperience• 7% believed that, based on their experience assessorsrarely had current industry knowledge and experience.Some interviewees believed that the lack of currency was aproblem resulting from the training package rather than theassessors’ practices. Other issues raised were that:• RTOs could not necessarily afford to provide access to thelatest equipment and technology• In industries such as health and community services, it isseen that RTOs do not keep pace with changes in servicemodels, government policy and legislation• 59% of respondents disagreed or strongly disagreed with • Assessors who had been in the training system for athe statement that RTOs are consistent in the way theylong time, and who had permanent jobs would almostconduct assessment.invariably become out of touch with industry unless theyWide variations were reported in approaches to assessmentactively went and worked in contemporary workplaces.– in some cases it was seen that this is merited. Industry There was conflicting data about whether the problem ofneeds assessors to be flexible and responsive to their needs, currency of assessor knowledge was more likely in publichowever there is also a perception that there are wideor private RTOs. Some interviewees regarded private RTOsvariations in the quality of how people are assessed and with a specific focus on a given industry or job role as morehence the level of confidence in the outcomes. There is a likely to have assessors with current and relevant industryparticular concern about ‘tick and flick’ assessment.knowledge; others saw the public system as offeringadvantages (stronger systems, less driven by the dollar).Preference for formal assessment• 75% of respondents agreed or strongly agreed with thestatement ‘I think training outcomes are better if formalassessment of training occurs”• 15% of respondents neither agreed nor disagreed with thestatement ‘I think training outcomes are better if formalassessment of training occurs”• 10% of respondents disagreed or strongly disagreed withthe statement ‘I think training outcomes are better if formalassessment of training occurs”.Despite concerns raised about assessment practices, theserespondents demonstrated a widespread preference forrecognised training which includes assessment.18 - NQC VET ASSESSMENT REPORT 2008

kEy thEmESThe research identified that employers and employerrepresentatives are often concerned about the skill levelsof graduates from VET qualifications. A number of factorscontributing to this perception were identified.training packagesSometimes the problem appeared to be with the trainingpackage or qualification structure, where skill levels are notclearly specified and industry standards are not clear. Inmany cases, assessors assess to the levels specified in thetraining package, but this standard is not representative ofwhat industry requires (R&CA submission and telephoneinterviews). There were suggestions from several sourcesthat training packages needed to be more specific whenreferring to performance levels and industry standards (SSAsubmission, R&CA submission).Units of competency need to reflect what’s required in theworkplace… in Aged Care, people can hold a Certificate IIIin Aged Care but never have showered a client, showering isa key part of the work.Assessors may conduct valid and reliable assessment againstthe documented competency standard/s, but if thosecompetency standards or qualification structures do notreflect the required work outcomes for their respective jobroles, it is inevitable that employers will have little faith intraining and assessment.Many units of competency specify that “where possible”assessment should take place in industry contexts. But often,assessments are undertaken in institutionalised contexts thatare not representative of industry settings (SSA submission,GTA submission, R&CA submission).Assessment of competence must be in the workplace orunder workplace conditions. Even simulation is a problem.There is no doubt that the maximum industry confidence inassessment is achieved when the assessment judgement ismade on the basis of evidence from the workplace.Large numbers of students obtain VET qualifications withoutany exposure to a real workplace.assessors’ skillsSkills of assessors vary. Some businesses noted that assessorshad current industry experience, others said assessors hadvague experience, others none at all. “People think becausethey have a Cert IV they can assess anything.” There isgeneral concern in the hospitality and tourism industriesthat assessors often do not have industry experience, or ifthey do, the experience is often not specific to the particularunit of competency, or it is not current (SSA submission,R&CA submission). From the hospitality industry, thereis also concern that the intent and outcomes of trainingpackages “are not being translated well on the ground” (R&CAsubmission).Training package assessment guidelines usually specify thatassessors must have industry experience, but the guidelinesoften do not specify that the experience must be currentor up-to-date. When they do make this specifciation, theymay not provide advice about how this currency should bejudged.It was suggested that assessor requirements outlinedin training packages be more specific about vocationalcompetence, and describe exactly what is meant by recentindustry experience (SSA submission).Many interviewees advised that they had little faith in theCertificate IV in Training and Assessment (TAA).Many of the current problems arise from the currentCertificate IV in TAA – the package is weak on assessment.You can’t teach someone to be a trainer in a week!Some interviewees claimed that there were RTOs awardinga Certificate IV in TAA on completion of a two day program.This qualification is seen as the VET system’s lynchpin. Ifassessors perceive that there is no rigour involved in thequalification for their own job, it is unlikely that they will inturn model rigour in the assessments they conduct. It wasbeyond the scope of this research to examine the validityof those claims but there is at least a perception by industrythat there is too much flexibility in how that particularqualification is being assessed.industry participation in assessmentWhen asked if they would like to be more involved inthe assessment process, many employers and employerrepresentatives said that they “couldn’t be more involvedthan they already are”. Many others said they would like tobe more involved, and those who lodged submissions wereclear that they wanted to see industry more involved in theassessment process to ensure that industry standards weremet (R&CA submission, ACCI submission, MLA submission,SSA submission, RWWA submission).In the fitness, sport and recreation industry, there arelong held views about the quality and consistency ofVET assessment. Calls are increasing for greater industryinvolvement and a much stronger emphasis on workplacebasedassessment.I like the idea of closer working relationships betweenindustry, RTOs and assessors – that seems essential and itrarely seems to happen.Respondents generally said they would like to see moreflexibility in assessments. This call is related to a desirefor assessments to be applicable to particular workplacecontexts, where assessments tasks match up with tasks ofparticular job roles in organisations and enterprises, and thestandards required by industry.NQC VET ASSESSMENT REPORT 2008 - 19

It was great when an RTO contextualised the learner guidesand assessment tools to link directly to our workplace andused real examples of the job they were employed in.Increasing flexibility leads to difficulty in ensuring thatstandards of training packages are being upheld.timeWithin the submissions, there was concern expressed abouthuge variations in the time taken to obtain qualifications (ATAsubmission).In some industries rapid change in industry standards orThere’s huge variation in standards between providers.legislation means that it is difficult for assessors to keepabreast of change. Sometimes rapid change leads to a lot ofThere’s extreme variation between the states.pressure on RTOs to quickly train and assess for the requiredqualification. In such situations quality outcomes can beThere are differing levels of consistency between RTOs – it’s overlooked. There were criticisms of this approach in someall over the place, really.areas of disability services, and as mentioned earlier, with theMany suggested that a more consistent approach couldCertificate IV in TAA.be developed through moderation. Moderation sessionsSome RTOs are unscrupulous and give out easy Certificateare conducted across various industry sectors, serving as aIVs – they simply have no integrity.forum to discuss and compare assessment tools, completedassessments and required standards.Pressures also exist around the assessment of studentsundertaking qualifications in skills shortage areas, often whenThis process works well in some instances, but on the whole there are influxes of students from overseas.it seems that industry involvement in moderation is low. Inboth the UK and in New Zealand, the standard setting bodies Some RTOs are offering AUR30405 – delivered and assessed(the equivalent to our ISCs) are involved in moderation of in 12 months! The problem is that this sits side by side a 4assessments.year apprenticeship program. Students end up with thesame qualification.Low industry participation rates are often due to cost factors.The cost of releasing an employee from his or her work toattend moderation or collaboration sessions with RTOs istoo great for many businesses to afford, particularly smallbusinesses. This is problematic because in several instances,industry stated they would like to be more involved withtraining and assessment processes, but the cost of doing sowas prohibitive.[There’s a need to fund] workplace supervisors oftrainees/apprentices to come together with RTOs to builda strong relationship between the RTO and those in thecompany working day to day to build capability. Needscommunication and ongoing relationship building.Some RTOs put forward a similar approach.Rigourous assessment is expensive. Engaging withemployers takes time and costs money.…strongly believe [communication] is the essenceof the whole thing. Issue is how? ISCs should developcommunication tools in consultation with RTOs andemployers that can be used.Enterprises that incorporated an RTO are more confidentthat assessment levels meet requirements of the workplace,and representatives from enterprise RTOs stated that theywere content with their level of involvement in assessmentprocess.A number of interviewees believe that assessors need tobuild stronger and more dynamic links with industry bodies.The Cert III in Hospitality – too easy to get – quality issue– issued with lack of industry experience in 12-15 weeks.(from SSA submission)And there is concern from employers about apprenticesbeing “moved through when they shouldn’t be”, when theyare not competent (AiGroup submission).Although there is general agreement that Competency BasedTraining (CBT) means there is not a requirement for a ‘timeserved’ approach there are very strongly voiced concernsabout what are seen as the ‘extremes’ – the ultra short versionof a program. In general terms, respondents seems to beraising the issue of sufficient practice, namely that a skillmay be learned in a short time but competence comes aftersufficient practice. Assessment should only be undertakenafter that practice period.assessment strategiesAssessment strategies are often developed in isolation fromindustry, without any industry input (WAHTITC submission,AiGroup submission).One employer reported that he had approached [anRTO] and asked if assessment could be completed in theworkplace so his apprentices could be assessed on themost current equipment used in the industry. While [theRTO] were ‘all for it’ they have not been able to follow upon the initial enthusiasm which the employer puts down tolack of staff and some internal disorganisation. (AiGroupsubmission)20 - NQC VET ASSESSMENT REPORT 2008

Industry seems to often miss out on the opportunityto contribute to training and assessment plans to helpdetermine assessment requirements.Far too often the training plan is predetermined andassessment is not discussed. There can be variations acrosscompany size which influence the extent of involvementa company will be interested in – it is sometimes the casethat small companies just don’t have time for extensiveinvolvement in assessments and just want the RTO to sortit out for them. While RTOs need to exercise their ownjudgement in these situations, they should also makeit clear to all employers that it is their right to have acustomised training plan, and that the RTO’s assessmentswill affect the duration of an apprenticeship and anapprentice’s wage progression. (AiGroup submission)This theme also resonated through the telephone interviews.aqtfThe research has identified that in many cases, employersand employer representatives believe that assessment is fartoo concerned with pre-2007 AQTF compliance (WAHTITCsubmission), ignoring a focus on industry standards. Thefocus has promoted a ‘tick and flick’ approach to satisfyAQTF auditors but fails to ensure that assessment processesaddress required standards.Several telephone interviewees particularly noted frustrationwith the AQTF.RTOs need to be more focussed on student outcomes andemployability and less on audit results. They are top heavywith admin because of the onerous and almost uselessrequirements of the AQTF.Some believe that the audit requirements of the AQTF haveput an end to holistic assessment.Holistic assessment of final competence at the completionof a VET qualification is something that has been lostdue to competency based training, modular delivery andassessment.The new AQTF does not directly address this issue. It requiresthat:Training and assessment are conducted by trainers andassessors who:- have the relevant vocational competencies at least to thelevel being delivered or assessed- continue developing their vocational and trainingand assessment competencies to support continuousimprovements in delivery of the RTO’s services. (AQTF 2007)Auditing “relevant vocational” competence is troublesomebecause it is open to different interpretations. Does it meanthe assessor must have the specific competencies beingassessed? Or is it competency that is somehow related to thecompetency being assessed? How in practical terms can it beaudited?The ACCI submission suggested that AQTF audits shouldoccur in conjunction with “a person deemed by industryto have the knowledge and industry experience to make ajudgement”.rPlBarriers to assessment such as inappropriate levels ofengagement from industry apply equally to RPL as they dofor other aspects of assessment (ACCI). Employers regardRPL as a high priority (ACCI submission, R&CA submission)and expect that employees will be engaged in the RPLprocess. The research revealed a lot of dissatisfaction aboutRPL processes. The survey responses and submissionsnoted that RPL is a very cumbersome, time-consuming,expensive process (R&CA submission, ACCI submission,AiGroup submission, RWWA submission, OWA submission),exacerbated by the “burden of proof” (ACCI submission).The RPL process relies heavily on supporting evidencesupplied by the candidate, or on evidence or reportssupplied by an employer, or someone who has first-handknowledge of the applicant’s capabilities. The task ofcollecting together vast quantities of evidence to satisfy therequirements of all the competencies listed in a qualificationis often very daunting for applicants: it can also be dauntingtask for assessors.The theme of holistic assessment reoccurs around RPL.Generally, employers would like to see RPL take on a holisticapproach, not an approach that has such a heavy emphasison evidence to suit a ‘tick and flick’ approach (WAHTITCsubmission).Holistic assessment must become the norm.It would help if assessors could take evidence fromworkplace tasks and map that evidence to competencies.Employers don’t want to be bogged down with trying tofind evidence to match up with the competency.It does seem that there is a trend toward tick and flickprocesses, and a lack of confidence about makingprofessional judgments.Stop making the whole process so full of red tape!But is the emphasis on recording minute details andevidence about every performance criteria in a qualificationreally what is expected by the AQTF? Or is this a distortedinterpretation that gets upheld by RTOs keen to utilise readymade,or easily constructed AQTF compliance templates?(Docking, in McKenna & Mitchell, 2006)NQC VET ASSESSMENT REPORT 2008 - 21

The Skills First RPL model being developed in Queenslandseeks to remove the heavy onus on paper-basedevidence and puts more emphasis on the professionaljudgement of the assessor. How assessors (who use anatomistic approach driven by perceptions about auditingrequirements) adjust to this style of assessment remains tobe seen.There are examples of RPL systems working well, whereorganisations have developed or located assessmentresources to guide the collection of evidence for particularjob roles, where performance is observed, and whereprocesses are streamlined to be cost effective (see Metalsmanufacturing and CITT/TITAB case study in Section 6), butthese examples tend to be the exception, rather than thenorm.Two submissions reported on the benefits of the StarRating Scheme implemented by the Institute for TradeSkills Excellence (ACCI submission, AiGroup submission).The scheme, which covers the skills shortage trade areas(automotive, manufacturing, building and construction,metals and engineering, electrical, mineral resources,hospitality and personal services, rural and farming),recognises high performing departments or schools withinRTOs through a range of evaluation criteria relating toenterprise and learner need, for example, whether the RTOcan conduct assessments to reflect holistic work activities,whether the RTO can use training package standards to meetlocal needs, or whether an RTO can deliver training at a timeand place that meets business and learner needs.The Star Rating Scheme provides a mechanism for therecognition of industry standards and expectations. Italso recognises the employment of “excellent” trainersand assessors. According to the Institute for Trade SkillsExcellence, excellent trainers and assessors have: up-to-date knowledge; recent industry experience that mirrorsindustry best practice; a good understanding of the skill andworkforce issues confronting local, regional and nationalindustry; and a leadership role amongst their peers. They alsohave a role in maintaining industry networks, maintainingtraining networks, and use state-of-the-art equipment andfacilities.Practitioners are also recognised for excellence in tradeteaching, through the Trade Teacher of the Year process.The R&CA submission also made reference to the ACCESSassessment system developed by the Australian HospitalityReview Panel, used in the late eighties and early nineties.The ACCESS system involved licensing of assessors with ascope equal to their industry experience, training assessorsin specific aspects of industry standards, professionaldevelopment to maintain status and the maintenance ofappropriate assessment standards and tools. According tothe submission, the ACCESS system wound up when StateTraining Authorities took control of training and assessment.vet in schoolsThere appears to be a serious industry lack of confidencein assessments in VET in schools programs in particularindustry areas. Several submissions note that graduates ofthe Certificate II and III level qualifications from this area,often had nowhere near the level of competence requiredfor targeted job roles (AiGroup submission, R&CA submission,SSA submission, GTA submission), and students had little orno exposure to industry contexts (GTA submission).The SSA submission also noted concern with the VET inschools programs that equipment and teaching methodswere often outdated, and there is real concern from thehospitality industry that standards are being eroded (SSAsubmission, GTA submission, R&CA submission).rto ratingsSeveral sources suggested that RTOs who meet industryrequired levels of training and assessment, should berecognised somehow.There needs to be a register of really good RTOs - notenough just to have qualification on scope, and audited– there are too many bad RTOs out there.I have found some RTOs to provide a better service thanothers – I immediately change RTO if I feel they are notdoing my staff justice.22 - NQC VET ASSESSMENT REPORT 2008

. Industry engagement inassessment processesThe closer an enterprise is to the decision making processesin assessment, the greater the confidence of the enterprise inthe assessment outcome.Involvement in assessment processes8% of respondents reported that they had participatedwith an RTO in the assessment of their employees. Follow upquestions with the interviewees indicated that the level andtype of participation varied across the enterprise’s range ofemployees and across RTOs with which the enterprise wasinvolved in some cases.• 75% were involved in collecting evidence such asobserving the employee, keeping examples of work• 61% were involved in discussing the performance of anemployee with an assessor• 9% were involved in providing technical advice to theassessor on workplace standards• 50% were involved in providing trained workplaceassessors.Interviewees were asked whether they would like to bemore actively involved in the assessment process. 55% of theinterviewees replied in the affirmative. Many of the remaininginterviewees answered no because:• they were enterprise RTOs and therefore felt they could notbe any more involved• they were already very involved in assessment• they were industry representative bodies and thereforewere not involved in assessment• they felt that their business commitments would not allowthem to allocate more time to involvement in assessment.The enterprises which took part in the research and hadtheir own RTO, either as an enterprise RTO or in a contractedrelationship, appeared to be most satisfied with the level ofengagement.Amongst the suggestions for increased involvement inassessment were• Employer input into final competency assessments• More scope for supervisor feedback and observedcompetence• Maintaining open and effective lines of communication• RTOs informing employers regularly of the progress of itsemployees• RTOs to use existing workplace measures to collectevidence of assessment rather than RTO developed orprovided proformas• Direct skill assessments in the workplace.NQC VET ASSESSMENT REPORT 2008 - 2

5. Strategies to increase industryconfidence in assessmentBoth the online survey and the interviewees canvassedstrategies to increase industry confidence in assessment. Inaddition, some of the submissions received documentedparticular strategies to improve the quality of assessmentoutcomes and in turn, increase industry confidence.When asked to respond to a number of statements aboutwhat would make the respondent feel more confident inthe assessment outcomes from VET training online surveyrespondents responded in the following way.• 62% either agreed or strongly agreed that they wouldbe more confident in assessment outcomes if there wasbetter communication between their organisationand the rto managing the assessment process, 12% eitherdisagreed or strongly disagreed.Greater communication and understanding within industryand particularly at line manager level on what assessmentsare about and the variety of ways they can be conductedboth on site and off site.In my experience the critical point is the relationship betweenthe employer and the RTO.• 87% either agreed or strongly agreed that they would bemore confident in assessment outcomes if assessors metwith each other and industry representatives to discussapproaches, agree common assessment tools and checkconsistency, they would be more confident. 12% eitherdisagreed or strongly disagreed. 5% either disagreed orstrongly disagreed.• 89% either agreed or strongly agreed that they wouldbe more confident in assessment outcomes if assessorsregularly attended professional developmentactivities with industry people to discuss workplaceexpectations. 5% either disagreed or strongly disagreed.Assessors must take ongoing professional development toensure they are up with technology eg. chemical application– need a process to regularly update: - should be trained inthe skill to work with another professional from industry– must have access to relevant funds and allow assessors toasses on site.Interviewees were asked a related set of questions with moreopportunities to expand on their responses.• 72% either agreed or strongly agreed that they wouldbe more confident in assessment outcomes if therewas endorsement or registration of assessors by anappropriate body (in addition to the requirements fora Certificate IV in Training and Assessment). 19% eitherdisagreed or strongly disagreed.So long as its transparent and clear about in what capacitythey have shown excellence and this can be communicated.The list could be endorsed by an industry association but therisk is that it may reduce access for new players who may beoffering innovation. It can be difficult to maintain integrityand quality of these sorts of lists over time.Good idea, but admin and support costly.Depends on which body and how it’s done.• 66% either agreed or strongly agreed that they wouldbe more confident in assessment outcomes if clearconditions for assessments are written into evidenceguides of competencies eg number of times a person isassessed, where assessed, who they are assessed by etc (egyou must be assessed in a commercial kitchen). 1% eitherdisagreed or strongly disagreed.Better sets of performance criteria. The more explicit thetraining package is the better. Critical aspects and conditionsof assessment also in user guideMay help but not in isolation as training packages are verycomplex technical documents and this may make themmore complex. Won’t make a difference on the ground unlessthere is more communication.• 62% either agreed or strongly agreed that they would bemore confident in assessment outcomes if standardisednational assessment tools were developed, (eg checklistsfor observations, questions, project descriptions etc) andapproved by an appropriate body in critical areas. 2% eitherdisagreed or strongly disagreed.There is a danger of becoming too prescriptive at RTO level- they need to be able to build in for the workplaces that theyare dealing with.‘One size fits all’ is a problem, fixed national program notflexible enough but still need national minimum standard.Difficult because the industry different. Template would beuseful.Against flexibility (OK if offered as exemplars say to setstandards and levels).Yes, but -Need to include unique aspects for assessment foreach enterprise.• 70% either agreed or strongly agreed they would bemore confident in assessment outcomes if funding fordelivery of training/short courses for assessors in theworkplace to assist with assessment (eg record keeping,explaining workplace requirements to external assessors,observing employees at work) was made available. 11%either disagreed or strongly disagreed.Should have a particular focus for SMEs and be on a day today support basis to show them how processes work, provideadvice etc.2 units at Cert III. Skill set of TAA.Needs to not be generic and needs to be flexible to industryneeds.• 84% either agreed or strongly agreed that they would bemore confident in assessment outcomes if there was jointsign off on assessments by both employer or companyand the rto. 11% either disagreed or strongly disagreed.2 - NQC VET ASSESSMENT REPORT 2008

Assessors should visit industry workplaces to guage skillrequirements and provide additional workplace assessmentThis can only work if there is good communication. Tensionarises when communication is not clear• 63% either agreed or strongly agreed that they wouldbe more confident in assessment outcomes if therewere more detailed audits of assessment processes,including assessment tools, outcomes (eg as required byAQTF). 20% either disagreed or strongly disagreed.Networks are a good idea. Should include both RTOs andcompanies and be open to supervisors and others who arenot formally ‘assessors’. People need a reason to keep comingback so need to be driven by good inputs and there needsto be communication around expectations of coming into anetwork.Need clear goal of what they want to achieve.Other suggestions included:Less focus on tools and administration and more focus onoutcomes eg. expressed in business terms - demonstratedbehaviours identified. Effort to develop auditors removedfrom business. Employers should be asked Did they (the RTO)add value to the business?I want to see someone get in trouble for running a ‘p*** poor’assessmentNeeds to be an industry validation – annuallyConcern is that some RTOs are unscrupulous and whippeople through and assess people just to make it quick andcheap - this isn’t picked up in audits• Improving information provided to the employerOften I receive statements of attainment or certificates thatare little more than line items and completion dates. It wouldbe good to include a longer description of what knowledgeand skills the trainee now possesses.• Assessor updates of industry knowledgeAssessors updating their own industry knowledge byscheduled returns to the workplace for regular blocks of workre-experience. Perhaps 2-3 blocks of 1-2 weeks per year...• Assessment over a period of timeVETAB (NSW) undertake joint audits with regulators.Recording of on-job evidence over time. RTO’s shouldnot sign anyone off as competent if the person has notPatchy auditing. Resourcing required to undertake whoconsistently applied their Skills and Knowledge on job to thepays the auditors. Strategic industry audits. Market drivenstandard required over a range of conditions. Hence on job- market will decide. profiling/log book system/ or some other means of evidencegathering is important.It won’t work if they are just paper based. If they are morefocussed on collecting feedback from the employer/Follow up and further checks with those assessed –- theemployee that the assessment process is OK or on how itfeeling seems to be, get your certificate and go! While peoplecould be improved then yes.forget and a lot of the training is not assessed properly.Auditors should randomly talk to students to discuss qualityin assessment.Spot audits on assessment should be the norm; RTOs knowwhen its happening and make processes look good. Thereshould be separate audits for assessment with a focus oncontinuous improvement. Auditors should be more proactivein helping to improve, not so much compliance driven.• 80% either agreed or strongly agreed that they wouldbe more confident in assessment outcomes if there wasbetter communication between employers or thecompany and the rto, so that employers are betterinformed about and involved in assessment processes. %either disagreed or strongly disagreed.Strongly believe this is the essence of the whole thing.Issue is how? ISCs should develop communication tools inconsultation with RTOs and employers that can be used.• 76% either agreed or strongly agreed that they wouldbe more confident in assessment outcomes if assessornetworks for moderation and ongoing support werecreated. 1% either disagreed or strongly disagreed.• Using workplace evidenceIndustry knowledge by the assessors would mean thatthey could take evidence from workplace tasks and map tocompetencies. Do not bog down the employer with trying tofind evidence to match the competency.Contextualise assessment processes and products and utilisethe resources made available by the enterpriseEnsure there is probity in what is occurring – demonstrationof work readiness• RPLFund workplace supervisors of trainees/apprentices tocome together with RTOs to build a strong relationshipbetween RTO and those in the company working day today to build capability. Needs communication and ongoingrelationship building. RPL is a major sleeper. Problems withperceptions of RPL are because employers have a poorunderstanding of how it can provide opportunities forthem. Communication of the benefits needs to be outcomesand solutions focussed, building on case studies to provokepeer interest.NQC VET ASSESSMENT REPORT 2008 - 25

6. Best practice – Case studiesThrough this research, a number of examples of goodassessment practice were described by the respondents.Some of these case studies have been summarized below,but it is important to note that these are examples only,and the researchers came across many good assessmentpractices which are working well for a particular group. Thecase studies describe the assessment process, the strengthsof the approach, and resource implications and transferabilityto other contexts.The case studies have been summarised under fourgroupings:• Linking assessment to enterprise needs• RPL processes• Customisable assessment tools• Use of self assessment in the assessment processLInkIng ASSESSmEnt toEntERPRISE nEEDSThere were many excellent examples of linking assessment toenterprise needs reported as part of the research. This sectionwill describe three examples, one by an enterprise based RTO,one by an RTO working in partnership with an enterprise, andone by an RTO which organises assessment around real worktasks for an industry.CASE StuDy 1use of performance indicators inassessment in an enterprise RtoBusiness:Insurance Australia GroupThey use a combined ‘Development and Accreditation’Model, which outlines steps. Each step is focussed on theopportunities for the learner and the organisation; and theassessment of these is mapped back to units of competency.During this mapping a distinction is made between thetasks required to meet performance criteria, and other‘business critical’ tasks. The assessment aspects are mappedand defined before training commences, and these areembedded in the performance indicators of the job function.Strengths of the approach:• Assessment is conducted at various stages, not just once• Employees are provided with performance measuresbefore commencement of training• Assessment is embedded in the performance appraisalprocesses of the organisation• Assessment is holistic and not atomistic to eachcompetency.resource implications:• Minimal additional resources are required as assessmentbuilds on resources already in the organisation.transferability to other contexts:This model has transferability to other organisations andindustries. Organisations that have mature performancedevelopment and management systems are more fertileplaces for this model to establish and be amalgamated intobusiness processes. However, the essence of mapping jobrole performance to competency units could be appliedmore broadly.contact:Chris Butler - Accreditation Manager – 02 9292 80– Christopher.Butler@iag.com.auassessment process:IAG has a commitment to offer staff qualifications and hasdeveloped an approach to assessment practices that isstrongly aligned with the needs of the organisation. As anenterprise RTO they have established assessment outcomesfor learners that link directly with the performance indicatorsalready used in the organisation. Learners’ managers, who arerecognized by the business as ‘subject matter experts’, assesstheir staff as part of their normal role. The RTO acts in anadvisory capacity, ensuring quality and consistency but notimposing any additional requirements other than those thebusiness already has built in.26 - NQC VET ASSESSMENT REPORT 2008

CASE StuDy 2 CASE StuDy 3the Westin hotel, melbourneassessment process:The RTO working with The Westin Hotel is flexible in the wayit conducts training and assessment, and has been chosenspecifically by the hotel because of this flexibility. When theRTO started working with The Westin Hotel they were givenan office in the hotel so they could organise the training andassessment to meet the needs of the hotel. This includedexamining job descriptions and organisation documentation,and having open forums with key personnel, so that thetraining and assessment was linked to the hotel’s tasks. Bylooking at the hotel requirements, they were able to linkthese back to the units of competency, rather than the otherway around, meaning they could deliver and assess realoutcomes for the hotel.The flexibility of approach also related to the timing oftraining and assessment in a 2 hour a day, 7 day a weekbusiness, and the chosen RTO was able to be flexible to meetthe delivery timetable of the hotel.Strengths of the approach:This approach means that the training and assessment isspecifically tailored to the hotel’s requirements, and theflexibility means that the RTO follows the hotel’s 7 day aweek, 2 hour a day timetable for training and assessment.The assessment is focussed on the hotel’s requirements, notthe RTO’s requirements.resource implications:The main resource implication for this approach is theallocation of a working space in the organisation for the RTO.However, now that they system is set up, the RTO maintains asmaller presence in the hotel training area.Another resource is the time taken by the RTO trainers/assessors, and the hotel’s personnel to discuss the realrequirements of the hotel in relation to training, and to setup the system. However, the Director of HR feels that thisinvestment in time is well worth the result.transferability to other contexts:This flexible approach could be used with any RT0/employerrelationship in most industry settings.tAfE nSW, Western Instituteassessment process:ShearingAt TAFE NSW Western Institute, a holistic assessment processhas been set up for the assessment of shearing, which canoccur in the ‘novice’ shearing shed which provides a workingshed training environment, and also on the job. The HeadTeacher, Shearing/Agribusiness reported that in the past, theyassessed by listing all the performance criteria in the units ofcompetency in a ‘lock step’ manner. However, the applicationof this approach for on the job training in working shearingsheds was less than ideal. They realised it was not a good wayto assess and it led assessors toward a ‘tick and flick’ process.To overcome this problem, a group of trainers togetherunpacked the units of competency, to see what criticalevidence they needed to demonstrate real work tasks, eitherin a notice training environment or working shed. Theythen mapped this back to the units of competency, to makesure everything was covered. The team then thought of thedifferent ways to assess, mainly observation with observationchecklists, some oral questioning with suggested questionsand information for assessors about what they needed tocover. They also used casual and guided discussions, egduring ‘smoko’ and lunchtime, with the outcomes of thisdiscussion being recorded and forming part of the evidence.The assessment process also included joint sign off, by theassessor and a workplace representative, eg classer, shedmanager, contractor.Wool handlingFor wool handling, they took this methodology further andobtained funding from Australian Wool Innovation (AWI)for a similar process. This project based assessment on workroles within the wool handling team and also included someresource development, trainer workshops and best practiceexamples from a range of RTOs nationally. One examplefor learners with low literacy was a series of slides showinga range of potential wool contamination situations, whichpeople being assessed could choose from, rather than relyingon words. AWI then disseminated this information nationallythrough a series of consistency workshops for trainers in arange of RTOs to help national consistency.Strengths of the approach:The holistic assessment process is based around theworkplace and workplace tasks (or working shed in a traininginstitution) and the assessment tasks are then mapped backto a unit of competency, rather than the other way around.NQC VET ASSESSMENT REPORT 2008 - 27

Using a group of assessors for this development processCASE StuDy 4leads to consistency of assessment, and assessment is valid asthey are assessing real work tasks. The process also allows formoderation/validation sessions three times a year.resource implications:There was a lot of resourcing to set up the system, in termsof assessor time, e.g. unpacking the units of competency,writing the assessment tools and other resources. Theresourcing issues are greater in this situation where assessorswork in different locations.metals, manufacturing & Service ItC, WAassessment Process:The Metals, Manufacturing and Services ITC, WA has astreamlined Skills Recognition Process which concentrateson support for the applicant. This is a fee for service process,mainly in the mining, engineering and resources industry forpeople who have been working for at least six years in theindustry, and who need trade papers.Once this was set up, the resourcing time was for:• travel to site and spending 10 hours in the shedconducting assessments• maintenance of the assessment tools – trainers meet threetimes a year for validation and refining of the tools.transferability to other contexts:This process could be used in any context, for any trainingpackage, if there is a body of assessors prepared to putthe time into the initial development of the assessmentresources.RPL PRocessesDuring the research, several industry training councils inWestern Australia talked about their RPL processes, anddiscussed how they had set up streamlined yet rigorousprocesses. One process described below, focuses on supportfor applicants. The other process works with the enterprisesto map the competency against enterprise training.Support for the applicant occurs throughout the process.• During an initial telephone conversation, the process isexplained to the applicant.• If an applicant decides to go ahead with the process, anapplication form which lists the units of competencyis sent. This enables the applicant to do an initial selfassessment.• After payment of an initial non refundable amount, theapplicant has an interview with the ITC officer, who goesover the evidence and talks the applicant through thewhole process, including advice on types of evidence.The officer make suggestions about additional evidence ifrequired.• The ITC officer then assists the applicant to make contactwith an RTO (usually TAFE) for the assessment.• The ITC sends a copy of the portfolio of evidence tothe assessor, who goes through it with the candidate. Ifassessment can occur in the workplace, that is preferable,and the assessors are encouraged to use professionaljudgment rather than a micro- approach.• At the TAFE, the applicant is enrolled in the whole course,so if the need to pick up some gap training they can dothis in a number of ways, including joining classes, offcampus or by completing activities.• The paper work is handled by the ITC.Strengths of the approach:The Executive Officer of the ITC believed this was a fair andrigorous process. The applicant gets a lot of personal supportthrough the process, eg explanation of process, examplesof typical evidence, application forms, initial meeting withMetals ITC before the skills assessment.However, the Executive Officer also reported someweaknesses of the approach, including difficulty in gettingsuitably experienced assessors, and the unfamiliar evidencegathering processes for most applicants. The weaknessesprovide a rational for support.28 - NQC VET ASSESSMENT REPORT 2008

esource implications: CASE StuDy 5The primary resources needed for this process is making staffavailable for support, and for the assessment, particularly if CItt and tItABassessment is to be conducted on the job.transferability to other contexts:The Metals, Manufacturing and Services process could beused by other industries if an industry body is prepared toprovide the kind of support used in this skills recognitionprocess.CITT is a not for profit, national company, whose corebusiness is to promote and facilitate training andemployment opportunities within the Telecommunicationsand IT industries. TITAB Australia Cabler Registry Servicesis an ACMA Accredited Registrar, set up to provide thetelecommunications industry with its own non-profit registryservice. The work with organisations, eg telecommunicationsorganisations and call centres, to ensure there is arigorous and streamlined RCC process, which includes skilldevelopment where necessary.assessment process:As part of the process, CITT:• map training that occurs in the workplace against trainingpackage qualifications, including induction training, onlinecourses for OHS, information about privacy (many of whichare already assessed, to see what training has already beencovered and assessed)• identify areas to gather evidence, and skill gaps to beaddressed through training or activities in the workplace• use supervisors/managers to validate evidence• use workplace assessors to assess the evidence, eg from anRTO• set activities to fill gaps, and use assessment tool to assessafter training and/or activity completed.Strengths of the approach:The main strength of this approach is that the initial mappingforms the basis of the process. This means that the evidenceis based around tasks in the workplace, and helps streamlinethe assessment process.resource implications:For this streamlined process to occur, there are someresource implications, including:• committed organisations• assessment tools, record books, activities to meet gaps• time for the process – it often takes over months• training of workplace assessors within organisation and/orRTO• supervisors who know the content, and understandassessment process broadly.transferability to other contexts:This process is enterprise based, and the organisations do notget government funding. The process would be transferableif there was a funding model, such as taxation relief.NQC VET ASSESSMENT REPORT 2008 - 29

CuStomISABLE ASSESSmEnttooLSDuring the research, some organisations stressed the needfor high quality and customisable assessment tools. Thefollowing case study describes an RTO which has developedassessment tools for a particular industry, and offers a serviceto customise these tools.CASE StuDy 6Performance trainingPerformance Training’s services include provision ofcustomisable learning resources and assessments, mainly, butnot exclusively for, the mining industry.Use of self assessmentSeveral respondents to the research survey talked about theuse of self assessment as part of the assessment process.The following case study shows how this can be used tosuccessfully streamline the assessment process withoutcompromising rigour.CASE StuDy 7Scouts Australiaassessment process:For practical assessment of ski touring instructors, assessmentconsists of demonstration of skills over several weekends, inall conditions. At the end of the weekend activity, participantsself assess their skills against the criteria linked to the relevanttraining package qualification. Those not yet competentusually recognized it themselves. The assessor has to sign offon the practical assessment.assessment process:Assessment tools to be used for mining training packagecompetencies include check lists for practical tasks, directquestions for theoretical aspects and summative assessmenttasks. These assessment tools can be used by enterpriseFormal assessment of theory also occurs.trainers and assessors, or external RTOs, and can becustomized for a particular enterprise. Strengths of the approach:Strengths of the approach:• Generic tools means there is consistency of assessment.• Tools can be customized for the workplace allowing forflexibility, eg using workplace procedures.• All tools are mapped back to competency, so they are validand cover all aspects of the unit of competency.• The tools are well received and accepted in the industry.• Assessment tools can be developed on a needs basis.This self assessment process takes pressure off the assessor,and puts the onus back on the trainee. Participants can see ifthey are not yet ready to cope with the requirements of theelements and performance criteria of the qualification.Results were more accurate than expected when this systemwas first introduced. Some trainees tend to under estimateskills, but rarely overestimate them.A range of assessment methods is used – it does not just relyon the self assessment, as there is also observation by theassessor during the outdoor activities, and formal assessmentof theory.resource implications:Performance Training has a staff of approximately 18,including instructional designers, technical writers, graphicdesigners and desk top publishers.transferability to other contexts:This is a business model that is used by many privateorganisations providing training related services to industry.resource implications:This system relies on an accurate and rigorous selfassessment tool.The self assessment must be done directly after the weekendactivities.transferability to other contexts:This method could be used for any practical skill.0 - NQC VET ASSESSMENT REPORT 2008

Appendix I – Submissions ReceivedACS Distance EducationAustralian Chamber of Commerce and Industry (ACCI)Australian Industry Group (AiGroup)Automotive Training Australia (ATA)Group Training Australia (GTA)GrowcomMeat and Livestock Association (MLA)Outdoors WA (OWA)Racing and Wagering WA (RWWA)Restaurant & Catering Australia (R&CA)Service Skills Australia (SSA)WA Hospitality and Tourism Industry Training Council (Inc)Appendix II – DataAll data is Available as a separate document on the NQC Website:www.nqc.tvetaustralia.com.au/vet_assessment_reportNQC VET ASSESSMENT REPORT 2008 - 1

Appendix III – Survey respondentsABC Learning CentresAFL Sports ReadyAnglican CareApprenticeships Queensland LimitedAustralian Technical College - HunterBCGCapitol Skills CentreCGU InsuranceCharles ParsonsCICT, Box Hill Institute of TAFECitiPower/PowercorCombined Team ServicesCommerce QldConocoPhillips Australia Pty LtdDepartment for Families and CommunitiesEast Africa Cranes LtdElectrical Group Training LtdEmergency Services AgencyFord Motor CompanyGarfish RestaurantGM Holden LtdGreening AustraliaHotel Motel & Accommodation AssociationIAG - CGU Business partners - FI and AffinityIan Barker & associates Pty LtdIIT TrainingIMSInsurance Australia GroupJRD Communications Pty LtdMaxima Group IncMcDonald’s Australia LimitedMG My GatewayMilcom communicationsMission AustraliaMount Isa Group Apprenticeship, Traineeship & Employment(MIGATE)MPA SkillsNational Safety Council of Australia LtdNorske Skog Albury MillQueensland Tourism Industry CouncilRecords Solutions P/LRecreation Training QueenslandRoma Regional councilRoyal Australian NavyRural Skills AustraliaSmall business owner/ RTO lecturer & assessorSouthern Cross CareSydney Training and EmploymentTAFE NSWThe Rare PearTORGASTorgas IncTraining Broker - Grains Industry2 - NQC VET ASSESSMENT REPORT 2008

Appendix IV – Interviewees90 degrees in the shade Resources and Engineering Skills AllianceACTURio Tinto Iron OneAg Consulting CoRSL CareAustralian Institute of Project ManagementS.I.S. LtdAMWU National Vehicle DivisionSydney FerriesAustralian Chicken Growers CouncilSydney Institute of TAFEAustralian Customs ServiceTasmanian Small Business OrganisationAustralian Institute of ManagementThe Federal GroupAutomotive Training AustraliaThe Group Training Association of NSWAutomotive Training Australia (WA Inc.)The Western Hotel Starwood Hotels and ResortsCabrini HospitalThomas and CoffeyCITT (Communications and Information Technology Training), Transport and Logistics Centre (TALC)TITAB (Cable Registry Service)Transport and Storage Industry Training CouncilDefence - (DMO) Defence Material OrganisationWoodside EnergyDefence - Defence Education Training and DevelopmentDepartment of Human Services - Disability ServicesDigital Education ServicesNOTE: some individuals requested that the report did notname their organisation’s participation.Estate WA (Jones Ballard Real Estate WA)Fortescue MetalsGM HoldenGoulburn Murray WaterGrains Industry Training NetworkGreen Corps - Greening AustraliaGroup Training Association QLD and NTGroup Training AustraliaGroup Training Australia SAHeidelberg Graphic EquipmentHertzHyatt Hotel SydneyInsurance Australia GroupKR CastlemaineMaster Plumbers Association NSW and Master PlumbersApprentices Ltd.Maxima Group TrainingMetals, Manufacturing and Services ITCMilparinka Inc.Minerals Council of AustraliaMVA Maritime Union of AustraliaNational Farmers FederationNational Safety Council of AustraliaNSW Business ChamberPatricksPeter J Johnson ConsultingPort Waratah Coal ServicePybar Mining ServicesQTICRay White Real EstateNQC VET ASSESSMENT REPORT 2008 -

Appendix V – ResourcesAustralian Chamber of Commerce and Industry, 2008.ACCI Policy Review, Issue No. 2Australian Industry Group, 2006. World Class Skills for WorldClass Industries - Employers’ Perspectives on Skilling inAustralia,Australian National Training Authority, 200b, High levelreview of training packages. It’s happened… what now?,BrisbaneBusiness Council of Australia, 200. The vocational educationand training system: key issues for large enterprises. A reportprepared for the Business Council of Australia by The AllenConsulting Group, MelbourneClayton, B., Roy, S., Booth, R & House, R., 200. Maximisingconfidence in assessment decision making, NCVER, AdelaideGuthrie, H, Perkins, K, Ngyuen, N, 2006. VET teaching andlearning: The future 2006-2010 – The roles, knowledge andskill requirements of the VET practitioner, Department ofEducation and Training, PerthHarris, R, Simmons, M, Moore, J, 2005. A Huge Learning Curve,TAFE Practitioners’ Way of Working with Private Enterprise,NCVER, AdelaideLeitch, S, 2006. Leitch Review of Skills: Prosperity for all in theglobal economy – world class skills, HM Treasury, UKMcKenna, S & Mitchell, J, 2006. Professional Judgmentin vocational Education and Training: A set of resources,Reframing the Future, DEST, CanberraMitchell, J, Chappell, C, Bateman, A & Roy, S, 2006. Qualityis Key: Critical Issues teaching, learning and assessment invocational education and training, NCVER, AdelaideCouncil on the Ageing, 2005. Editorial – Mature EmploymentKey to Skills Shortage, 11 May. NSW Mitchell, J, McKenna, S, Bald, C, Perry, W, 2006.New Capabilities in VET: Insights from Reframing the FutureDEST, 2005. National Strategic Industry Audit of Training inthe Hospitality Industry, Australian Government, CanberraDEST, 200. National Strategic Industry Audit of Training inthe Transport Industry, Australian Government, CanberraDEST, 2007. Student Experience Roundtable – the key findings,National VET Equity Advisory Taskforce, SydneyDEST, 2007. Australian Quality Training Framework (AQTF)Users’ Guide to the Essential Standards for Registration,Commonwealth of Australia, CanberraDepartment for Innovation, Universities & Skills, 2007.World Class Skills: Implementing the Leitch Review of Skills inEngland, HM Government, UKproject teams how to build capabilities for implementingthe national training system, Commonwealth of Australia,Adelaide,Mitchell, J, Chappell, C, Bateman, A & Roy, S, 2006. Criticalissues: A literature review on teaching, learning andassessment in vocational education and training, NCVER,Adelaide.NCVER, 1997. Employer Satisfaction with vocationaleducation and training, 1997, national report, AdelaideNCVER, 1997. Employer Satisfaction with vocationaleducation and training, 1997 - At a Glance, AdelaideNCVER, 2001. TAFE and university graduates 15-2 years– At a Glance, Adelaide.Department for Innovation, Universities & Skills,Union Learning Fund website, viewed April 2008 atNCVER, 2008. Australian vocational education and training statistics: Employers’ use and views of the VET system 2007– Summary, AdelaideDepartment of Education and Training (DET), 2006.Queensland Skills Plan, BrisbaneOECD, 2007. Economic Policy Reforms: Going for Growth,ParisDEEWR, 2008. Skilling Australia for the Future: DiscussionPaper 2008, Australian Government, Canberra Ridoutt, L, Selby Smith, C, Hummel, K, Cheang, C 2005.What value do Australian employers give to qualification?DEEWR, Institute for Trade Skills Excellence website,viewed April 2008 at DEEWR, Reframing the Future website, viewed April 2008 atDEEWR, Training.com website, viewed April 2008 atFoster, S, Delaney, B, Bateman, A & Dyson, C, 2007. Higherlevelvocational education and training qualifications: Theirimportance in today’s training market, NCVER, AdelaideGroup Training Australia, 2007. Policy Statement – a betterskilled workforce, Group Training Australia, GlebeNCVER, AdelaideTAFE NSW, 2007. Doing Business in the 21st Century,November, Department of Education and Training, NSWGovernmentVECCI, 200. Employer Spending on Education and Training200, MelbourneWerner, M. 1998. ‘Employers’ perceptions of how well theVET system serves their industries training needs’, AVETRAConference Proceedings, UTS, Sydney - NQC VET ASSESSMENT REPORT 2008

n Q C S E C R E t A R I A t o f f I C ELevel 21, 390 St Kilda Road, Melbourne, Victoria, 3004 | PO Box 12211, A’Beckett Street Post Office, Melbourne, Victoria, 8006Phone: +61 3 9832 8100 | Fax: +61 3 9832 8199 | Email: enquiries-nqc@tvetaustralia.com.au | Website: www.nqc.tvetaustralia.com.au