Level 3 Learning and Development - Month 6

Evaluating the Impact of Learning – 

Delivering on Business Needs 

Month 6 - Theme 3 

Level 3 

Learning and Development 

Practitioner 

Apprenticeship Standard

CONTENT 

03 

Introduction to Month 6 

Welcoming you to Month 6 of your L&D Journey 

Welcome to Month 6 of your Learning & Development Practitioner journey 

where, this month, we unpick the evaluation process and explore the theory 

and common practices behind the evaluation stage. 

04 

Reviewing learning from Month 5 

This month you will explore: 

06 

10 

12 

14 

16 

22 

26 

Why Bother with Evaluation? 

Back to the Basics of Evaluation 

Evaluation Theory and Thinking 

Evaluation Processes and Practices 

Planning your Approach to 

Evaluation 

Getting Creative and Flexible with 

Evaluation 

Ensuring your Chosen Tools are Fit 

for Purpose 

Scan the QR Code to read 

through this magazine 

on your mobile or tablet 

device. 

• Why we bother to evaluate 

• How and what we should be evaluating 

• Things to consider when analysing the data and presenting your findings 

Much like Learning Needs Analysis (LNA), the evaluation process can often be 

overlooked by practitioners and undervalued by stakeholders, yet it is a critical 

piece in understanding the value of an intervention. 

If we have not resolved the declared problem or met the need, we have not 

moved the organisation towards achieving its declared objectives. 

If we have not checked learning transfer or performance improvement, then we 

have wasted our resources; expertise, time and money. 

27 

28 

31 

Collecting Information on 

Learning Transfer & Performance 

Improvement 

Presenting your Findings and 

Feedback 

Over to You - Your Tasks to 

Complete This Month 

Icon Key 

To support you in understanding 

all the elements of the programme, 

everything has an icon to denote 

what kind of activity it is: 

Taught Content to 

Learn/Information 

Questions 

Tips 

Research 

Level 3 Learning and Development Practitioner| Month 6 Level 3 Learning and Development Practitioner| Month 6 

02 

All rights reserved copyright©2021 Remit Group 


03

Let’s do some light-touch evaluation of your own learning! 

So, in service of checking learning transfer, let’s 

begin by checking in on some of your learning 

from last month. 

Reflect back and answer the following on 

designing interventions: 

• There are nine key stages in the 

design process; name them and 

describe which resonated most with 

you and why? 

• How would you describe the difference 

between Aims and Objectives? 

• We discussed three levels of 

objectives; can you describe what they 

were and how they are all useful in 

their own way? 

• How does David Rock’s SCARF model 

impact learning design? 

• What were the 2 key takeaways that 

you took from Module 5, why were 

they impactful for you and how will you 

transfer these back into the workplace? 

Capture your responses in your Learner 

Journal. 


04 



05

Why Bother with Evaluation? 

If you think back to the Training Cycle, evaluation is literally at the heart of the 

training process. Yet, much like LNA, it is frequently missed or relegated to a 

marginalised role, possibly because like most of an iceberg, it is less visible. 

The CIPD factsheet on evaluation is a great summary of evaluation processes 

and practices, but more insightful is the honest article by Jo Faragher. And this 

is the reality of evaluation; there is a process and some tools, but the next level 

of learning around evaluation centres around the associated challenges and 

tricks to making it work for you. 

You can learn the “what” pretty easily; and we will. The challenge is developing 

your thinking around the “so what”, so you can optimise its impact for you, the 

organisation and the learner. 

Evaluation can, understandably, be seen as dry and disengaging; it can also not 

be understood. So, if our experience of evaluation processes and their success 

is negative or marginal, it is understandable why there is limited appetite to 

engage with it more rigorously. 

Yet evaluation is the mechanism through which we ensure learning and its 

effective transfer in order to improve performance. Not only does it ensure 

that the need has been met, but also that the resources expended have been 

effectively used. 

A more active and inclusive way of ensuring evaluation is alive and well, and 

front and centre in your thoughts, is to have it as an integral check in point 

at each point in the training cycle; during LNA, design and delivery stages. It’ 

makes sense and it removes some of the stigma around seeing evaluation as a 

dry, transactional experience. 

It remains ironic that with time and money being premium resources and 

organisations needing to work hard to engage, enable and retain their people, 

that evaluation either remains an afterthought or is a transactional experience 

focusing on performance in role. 

This raises two thoughts: 

1. 

Should we be 

bothering with 

evaluation at 

all? 

2. 

Can we do it 

differently and 

reset our collective 

relationship to 

evaluation? 

Matthew Channell presents an interesting argument for the need to reset our 

view on evaluation being the Holy Grail. 

It is without doubt that evaluation presents benefits, such as: 

• Measuring impact on performance , knowledge and skills 

• Where deltas still exist and therefore what needs to be addressed 

• Targeting better use of resources 

• L&D doing its part to meet business objectives 

• Employee engagement and retention 

But if we are not measuring the right things accurately, there is an inevitable 

argument to be had as to whether we should aim to evaluate interventions, or 

at least in the way we have traditionally. 




06 07

Further Research: 

Undertake some research of your own. Explore 

thinking out on the internet about the role 

evaluation has as a useful part of the training cycle. 

And capture your thoughts on the following: 

1. What are the key themes you are observing 

about evaluation? 

2. Talk to a colleague in your organisation and 

understand practically, what evaluation exercises 

are and are not undertaken and why? 

3. What thoughts are emerging for you on 

evaluation and its changing role within the L&D 

Function? 

Capture your responses in your Learner Journal. 


08 



09

If evaluation is about measuring and reviewing aspects of an intervention, there 

are other processes which are useful to also understand, such as: 

• Validation – Which establishes a truth or corroborates something. For 

instance, was an objective met. Evaluation generally includes validation, but 

evaluation is larger in scope. 

• Assessment – Centres on progress and measurement against set criteria. 

For instance, an exercise to measure learning level and retention before and 

after development 

• Testing - A more controlled and exact type of assessment. 

• Monitoring and reviewing – A series of activities to understand a learner’s 

progress on a journey. A more dynamic and less exact approach, but likely to 

offer rich insight. 

Let’s Go Back to the Basics of Evaluation 

Evaluation within the L&D environment is about measuring and reviewing 

different aspects of an intervention to establish if it was effective and valuable 

in delivering a need. It can cover aspects such as impact, it’s value and the 

extent to which it was engaging. 

There is an argument to see it as the final stage of the training cycle which 

looks back at what has been undertaken. A more dynamic and relevant 

approach is to consider it both real-time and as a stage which is future focused, 

much like LNA. 

Evaluation does not need to be seen as a series of spreadsheets and data, it 

can also be seen as insight being collected real-time in sessions which is then 

used to inform adjustments in the design activities to ensure learning transfer 

takes place and is continuously improved. 

Seeing it as part of a continuous process of adjustment, moves your evaluation 

practice into a more agile and manageable space, which is more engaging, 

relevant and impactful for you, the learner and your stakeholders. 

• Return on Investment (ROI) – A measure of financial return. A higher level of 

evaluation which has mixed success in being able to easily and accurately 

assess success. 

• Return on Expectation (ROE) – This measures stakeholder satisfaction 

and whether it met their expectations, which may be varied. An important 

approach and one which requires early effort and ongoing stakeholder 

management. 

The key message here is 

that there are numerous 

ways to capture data, 

which can be transferred 

into insight for use. 

If it is not the right 

insight, or if it is not 

subsequently used 

to effect, then the 

process of evaluation is 

redundant. 


10 



11

Over recent years, alternative models are being developed, such as Robert 

O Brinkerhoff’s The Success Case Model, which moves more towards using 

qualitative data. 

Interestingly, it uses surveys learners to identify transfer of learning, and then 

moves to the next stage where it uses story telling for learners to unpick their 

actual learning transfer, rooted in their workplace experiences. 

What is interesting about use of storytelling and direct links to experiences, 

is that it is engaging, directly relevant to their predicament and demonstrates 

impact; some of the key themes in adult learning theory. 

Whilst not quantitative, it does allow for that aspect of enjoyment and a more 

human aspect to learning. 

If you’re interested in this, here’s a short article on the Success Case Model. 

Let’s Look at Evaluation Theory and Thinking 

The main character in the history of evaluation is Donald Kirkpatrick and his 

Model of Evaluation, and it is his model you will typically hear referred to within 

L&D circles. 

There are four levels of evaluation from Reaction, to Learning to Behaviour and 

then Results. The model is a helpful structure, which in recent years has been 

developed to suggest a 5th Level by Philips and Philips, which represents the 

Financial ROI. 

Whilst not perfect, it remains a helpful guide to realistically track your approach 

to evaluation. 

Separate to this is the Anderson’s Value of Learning which focuses on ensuring 

the learning intervention is aligned with and focused on the strategic needs of 

the business. A three staged process, it aims to suggest best fit for purpose 

interventions to meet needs. 

Further Research: 

Make yourself familiar with 

the three models mentioned 

about and share your initial 

thoughts about the benefits 

and disadvantages of each in 

comparison to each other. 

Capture your thoughts in your 

Learner Journal. 


12 



13

Let’s Look at some Evaluation Processes and Practices 

In a bid to make evaluation useful, engaging and 

values, the trick is to keep it relevant and simple, 

and importantly, to keep it as a dynamic, real-time 

experience. 

Typically, the CIPD suggests a 10 point plan to 

planning and implementing evaluation practices: 

Planning Evaluation 

Implementing Evaluation 

7. Collate your insight or information 

8. Analyse it as per your structured criteria 

9. Draw your conclusions and proposed actions 

for improvement 

10. Present your findings and recommendations to 

your stakeholders 

1. Identify the activity to be evaluated, i.e. the 

“scope” 

2. Be clear on the purpose of the evaluation, i.e. 

what are you trying to achieve (the criteria) and 

why 

3. Work out what information you will need to 

establish each criteria 

4. Choose the right method and the right time to 

evaluate 

5. Design a tool, i.e., survey, interview, observation 

It’s a sensible approach within 

which you can overlay your 

insight into the context and the 

organisation to establish the most 

effective way forwards. 

Let’s look at them all. 

6. Embed this evaluation plan into your learning 

intervention plan and engage your stakeholder to 

agree your approach 


14 



15

Planning your Approach to Evaluation 

As we know, evaluation should not be an afterthought, but an ongoing part of 

your thought processes. When planning it, the key elements to consider are: 

1. Scoping out what to evaluate 

You can choose what it is you want to evaluate. It might be a single skill or 

practice whether you are running a single session or a programme. The size of 

an intervention does not necessarily dictate a need to evaluate it all. 

For instance, I was running a programme of Skill Bites across a community, 

but because my driver was primarily to re-engage these employees as part 

of a retention strategy, the primary evaluation focus was on engagement and 

wellbeing, not the skill build or learning transfer. 

2. Clarifying your purpose 

More often than not, the main purpose is to see if your Aims and Objectives 

have been met, which infers the desired impact on business performance. 

As you know, there needs to be a direct correlation between LNA, aims and 

objectives through to evaluation criteria. 

Using the example above, the LNA presented a disengaged, disempowered, 

disenfranchised workforce. The Aim was to “make them feel loved, valued 

and invested in”. The single common Objectives across the suite of nine Skill 

Bites was “to empower the individual with skills to feel empowered to manage 

their challenges in relation to xxx”. It followed, that although individuals were 

enhancing knowledge and skills in nine different areas, the evaluation exercise 

centred around their ability to feel empowered and therefore engaged. 

Scope is important as it allows you to focus in on the element to be evaluated 

without distraction or influence. It also provides clear context when you are 

presenting your findings. It is not unusual for stakeholder and business leaders 

to become excited about insight and seek more; setting boundaries prevents 

curiosity creating scope creep. 

Having clarity around scope and purpose, the evaluation criteria were: 

• The impact of development investment on their level of engagement with 

work 

• The extent to which they feel empowered to take action in challenging 

circumstance 

• The impact on retention over time of our people 

More typically, the evaluation would focus on one of the skills or knowledge 

builds. For instance, if one Skill Bite was “How to Build Trust”, the purpose of the 

evaluation might be : 

• The extent to which the Aims and Objectives of that session were met 

• The learner’s response to development and methods used 

• The cost and efficiency of the learning activity 




16 17

From our scope and purpose, we 

establish our evaluation criteria. Using 

the Trust example, criteria might 

include: 

• The extent to which the behavioural 

changes relating to building trust 

have been exhibited 

• The workplace improvements 

resulting 

These evaluation criteria direct us to 

the information we will need to collect 

and the questions we will need to 

ask. The most appropriate methods 

to gather data are dependent on the 

context within which you are operating, 

accepted norms, digitalisation and 

culture. 

A cautionary note continues throughout 

this module, which is that evaluation is 

not an exact science, and it is sensible 

to manage your own expectations and 

those of others accordingly. It is also 

useful for you to be confident, as a 

Practitioner, to be able to stand your 

ground when you are clear and have 

chosen what you plan to evaluate and 

how. 

3. Establishing the information you will need 

Having your Scope and purpose clear, you have your criteria established. The 

information needed can then be confirmed. 

Types of Information 

For instance, using the re-engage, empower and retain example, we would 

need to gather data for these three criteria. To ensure we can show causal 

impact, we would need to show the following: 

• Levels of engagement before and after the programme 

• Levels of perceived worth and empowerment before and after the 

programme 

• Attrition or “likely stay rates” before and after the programme 

With this example, qualitative interviews, surveys (both engagement and 

wellbeing surveys), interviews and pulse “stay rate” surveys were the most 

appropriate approach to data mining. 

The example of empowering and 

engaging a disenfranchised workforce 

as the primary driver was met with 

resistance from the Executive of an 

organisation who were accustomed to 

more obvious aim and outcome from 

L&D interventions, until I reinforced 

the context and highlighted the critical 

need to have their people feel invested 

in. 




18 19

Sources of Information 

Information can come from multiple sources depending on the organisation. 

Equally, the nature of the issue and the culture can play a part in you deciding 

where to gather the data. 

Rich sources of data can include: 

• The learner themselves, their experience, the impact of their learning 

• Line managers, what they observe 

• The L&D Practitioner and what they see, test and experience with the learner 

• Business intelligence and financials 

• Customer or user experience survey/feedback, qualitative and quantitative 

• Peers and stakeholders, both internal and external around the experience 

they have 

• System and process adherence 

It is useful to also look at it from the perspective of what has worked before 

in an organisation and what hasn’t. Talk to colleagues, mentors and external 

providers to get others’ views on what has worked for them and why. Don’t 

be afraid to consider alternative approaches and resist measuring everything 

simply because you can. 

4. Picking the right evaluation method 

Again, let’s keep it simple, impactful and engaging; we want people to become 

advocates of evaluation! 

Let’s look at a few methods available to you: 

• The programme end questionnaire – that one you used to complete as you 

were running out of the room; the “happy clappy sheet” 

• Pre & post-testing or assessment – be they tests or survey feedback 

• Learner self-assessment – providing rich information as directed on skill 

enhancements, gaps, confidence and ongoing management of barriers to 

further learning 

• Line manager assessment – where structured and consistently applied, this 

can provide insight and subtle observation feedback on knowledge, skills 

and behaviours 

• Surveys – potential for use in a plethora of ways, they offer qualitative and 

quantitative data which can be tracked over a period of time 

• Focus or Learning Groups – useful insight real-time in a structured or 

unstructured format 

• Metrics and Research – Depending on the organisational setup and data 

capture practices, often data is readily available, or can be manipulated to 

allow you to extract insight 

Whilst these are all established sources, consider other ways of capturing data 

or insight. Speak to experts and consider what else is available to you. 

Don’t be constrained by current practice and be comfortable with data that is 

qualitative, rather than clinical. Insight is insight. 




20 21

Getting Creative and Flexible with Evaluation 

In order to engage yourself and others in the evaluation process, don’t be afraid to get creative; and don’t’ get hung up on being quantitatively driven. 

• Having learners self-assess 

their levels of engagement 

or knowledge around topics 

before and after, using 

emojis, selfies or cut-out 

images, gets them active, 

creative and reflective. 

• A learning equivalent of 

a pub quiz, with teams 

competing and answering 

questions in groups to 

check learning. 

• Undertake the 3-2-1 test, where 

the learner shares three aspects 

of their learning which impacted 

them & why, two people they 

plan to discuss their learning 

with and one thing they plan 

to do differently tomorrow as a 

consequence of their learning. 

• Ask for the cohort to 

provide a single feedback 

questionnaire between them 

on learning, application 

in the workplace and 

continuing gaps for further 

self-development. 

• Evaluation cafes are engaging, 

seemingly free-style sessions 

where learners and their 

managers unpick learning 

that has resonated and been 

successfully applied in the 

workplace, the impact on their 

performance/expertise and 

continuing gaps for further 

development. 




22 23

Whilst these may not signal obvious measurables, you will be getting both 

qualitative feedback and encouraging further refection which helps consolidate 

their learning, often in a social setting. Whilst you always need to make sure the 

approaches are appropriate for the group and the competence you are looking 

to evaluate, just remember it is also about assessing their levels of engagement 

and interest in the learning experience. 

When you open your mind to the broader range of evaluative approaches, you 

can also release yourself from the constraints of when to evaluate learning. 

Recognising that options include, before learning, at the beginning of learning, 

during and immediately after the learning intervention, as well as months later, 

you can be as creative in choosing when to evaluate as you can be with how to 

evaluate. 

Considered at the design phase, you may need to assess learning before 

moving to the next stage of learning, as previously mentioned, or simply to mix 

it up and retain engagement and stimulation. 

Questions for you: 

If you were creating three 90 minute skill bites on Listening Skills, 

Questioning Skills and Coaching Skills: 

1. When would you plan to assess their levels of: 

a) Reaction to the learning (level 1 Kirkpatrick) 

b) Learning itself (level 2 Kirkpatrick) 

c) Self-assessed future development need in these three skill sets 

2. How would you choose to creatively assess items a-c? Why? 

3. When would you propose to and why? 

Capture your suggestions in your E-Portfolio. In terms of expanding 

your thinking, consider previous examples you have experienced, 

talked to L&D professionals or forums on-line. 


24 



25

Collecting Information on Learning 

Transfer & Performance Improvement 

Accepting that you may be collating data that may be qualitative or quantitative 

in nature, it is important that you ensure you are familiar with and respectful 

of the relevant data protection legislation, in terms of permissions, usage and 

storage. 

When it comes to actually analysing the data to turn it into useable information, 

and from there gather insight, you need to consider how to store and organise 

the data in order to “cut it”. 

Ensuring your Chosen Tools are Fit for Purpose 

Where organisations do undertake evaluation, you can find yourself in a 

position where you are simply using pre-existing evaluation tools because they 

are in place and accepted. 

Spreadsheets are a handy way of cutting the data; but again, it is important to 

know exactly what you want to analyse. Just because you can analyse data 

does not mean that you should, so actively avoid scope creep and fall into 

analysis paralysis; stick to the core metrics and measures you actually need to 

review. 

It is sensible to double check whether they continue to be fit for purpose. 

Sometimes, the objectives of a learning intervention change over time, but the 

evaluation mechanism or approach does not. 

Other things to consider are: 

• Whether virtual, self-directed or blended learning demands a change in 

approach to evaluation 

• Whether social learning can encourage greater creativity in evaluation 

• What an organisation actually values in development and therefore what it 

wants to evaluate now, in comparison to historical approaches 

• The agility demanded in development interventions which might demand 

greater creativity, speed, diversity of evaluation techniques 

• The extent to which the manager should and wants to be involved in 

evaluation 

All of the above and more, should encourage you to continuously challenge 

yourself and others, as to whether the evaluation practices, and core intent, 

need continuous review and improvement. 




26 27

The choice you have to measure the data around learning outcomes are many and varied, as 

are the ways in which you can cut that data.. 

In terms of learning outcomes, you can look at changes in learning/knowledge, or learning 

transfer into the workplace, or impact on performance improvement. You can even consider 

simply assessing shifts in levels of confidence in a subject or task. 

How you can cut the data, could consider assessing the data in terms of: 

• The different cohorts, environment or delivery factors (such as delivery methods, cohorts, 

facilitators, environmental factors, such as time of year or transformation) 

• Performance improvement by cohort, individuals, functions or manager 

• Greatest and least knowledge or skill improvements 

• Most impactful learning transfer to the workplace assessed at different timelines 

What other ways to look at data might you consider using? 

What are the factors you would take into account when deciding? 

Choosing how to cut the data will depend on the original need for change, the extent to which 

the organisation is interested and what you agreed with the stakeholders initially. 

A top tip is to avoid scope creep and consider where the key impacts are on performance 

improvement or achievement of the key business objectives. 

By having good, relevant data, you will gather advocates amongst your stakeholder groups 

which will smooth future support for well defined and managed learning interventions. 


26 



27

Presenting your Findings and Feedback 

Depending on how your 

stakeholders what the insight 

delivering, you can expect it to be in 

report format, through presentation 

at meetings or in one to one 

sessions. 

Again, consider the needs of your 

stakeholder; ensure you understand 

how they like their insight delivering. 

It is important that it is digestible 

and accessible for their needs; so 

understand the common practices 

in the business and the audience. 

Some like written word, others prefer 

data tables or charts or PowerPoint. 

Consider your audience and not only their personal preferences for insight, but 

also consider their work based perspective, For instance: 

• Line Managers might be interested in learning transfer to drive performance 

improvement, and their opportunity to further coach in learning to gain 

further improvements 

• Learners might want to see their own relevant performance improvement or 

how it supports the development career pathways 

• Executive Sponsors might want to see impacts on retention, satisfaction and 

insight impacting future resource allocation decisions 

Depending on your own objectives for the presentation, you can provide 

insight in data format and remember that creative ways of providing this insight 

can create engagement, so long as the content is meaningful, such as video 

montages from participants, word collages or voice over insight. 

The importance is that the content is clear, relevant and accurate and 

appropriate for your audience; this way you will use delivering the evaluation 

process to gain future sponsors and advocates. 

Typically, any presentation of insight 

requires clarity around the scope of 

the evaluation and why it has been 

evaluated this particular way. Then 

a summary of how evaluation has 

been undertaken positions you to 

share your findings. From there, 

you can articulate clearly defined 

recommendations. 

Level 3 Learning and Development Practitioner| Month 6 

As with all stakeholder management 

practice, you need to understand the 

differing needs of a diverse audience. 

You could consider meeting key 

stakeholders off-line ahead of the 

formal feedback session in order 

to have them briefed and engaged 

ahead of time. You may choose to 

share pre-reading to ensure all needs 

are met in terms of the data provision, 

allowing the presentation session to 

be a deeper conversation and debate. 

So, you have completed Month 6. 

Next stop … Delivery! So, be sure to be up to date and talk with your 

Development Coach or Manager on any areas where you are unsure. 

Well done on completing this session and we hope you’re motivated to ensure 

Evaluation remains on your agenda at all times. 

Level 3 Learning and Development Practitioner| Month 6 

28 



29

Over to You - 

Your Tasks to Complete This Month 

Discuss your learnings so far with your Development Coach, seeking clarification 

as necessary. 

You need to see your Workplace Manager or Mentor for two separate sessions 

this month 

1. Mentoring Discussion – How can we ensure our learning & development 

interventions are positively impacting the Employer Value Proposition? 

2. Coaching Conversation - How can we improve the evaluation of our learning 

interventions? 

Month 6 – Your Work Based Activity. 

Critically review and evaluate the Evaluation Process undertaken within your 

Organisation for the same specific learning intervention or short programme 

(previously identified). 

Hold meetings to explain your activities and gain information. 

Create a presentation and present your findings and recommendations to your 

Manager and relevant parties. Content to include: 

• Explanation of current evaluation practices use; reference evidence, reports or 

analysis found 

• Assessment of the evaluation processes used 

• Explanation of possible evaluation models for use 

• Recommendation of improvements or proposal for implementation of a specific 

evaluation mechanism, and why 

Complete your Learner Journal for Month 6, capturing key learnings and 

reflections arising from this activity and interactions, collaborations with others. 

Remember it is critical you capture all your learnings and reflections as you 

progress. Your End Point Assessment (EPA) is based upon the quality of your 

Learner Journal. 


30 



31

www.remit.co.uk 

0115 975 9550

Level 3 Learning and Development - Month 6

Create successful ePaper yourself

Delete template?

Save as template?