Student response to Mangahigh - PDST

Mangahigh Research Trial
Scotland Results
January to March 2010

List of schools involved
Castlemilk High
Cleveden Secondary
Forrester High
Govan High John Paul Academy
Jordan Hill
St Paulʼs High
Lochend Community High
Lourdes Secondary
Knightswood Secondary
Summary of findings
• 75% of students said they liked Mangahigh v and 66% said their Maths
© Copyright 2009-2010 Mangahigh.com
All rights reserved Various trademarks held by their respective owners.
Research trial – Methodology
• 10 secondary schools in Scotland took part in a 3 month
trial to assess the impact of Mangahigh on studentʼs
performance and attitudes in mathematics.
• The trial involved 1000 students and 20 teachers.
• The students made extensive use of both the games
and Prodigi® lessons to explore new topics and
reinforce previous learning.
• Students sat assessment tests before and after the trial
to measure the impact of Mangahigh.
• Students are likely to make more progress and complete more
mathematical operations when using Mangahigh compared to a ʻtypicalʼ
lesson
• Greater use of Mangahigh leads to higher results
• Students were more engaged when using Mangahigh compared to a
ʻtypicalʼ lesson
improved
• Over 90% of students would enjoy Maths more if they could use
Mangahigh

Student response to Mangahigh
40% of students also said they did more maths
outside of lesson time because of Mangahigh
Definitely not
No, really
didnʼt like
No, not
really
(based on approx 270 students at 7 schools)
No
Do you think you’d enjoy maths more if
you could use Mangahigh?
Probably not
(based on approx 290 students at 7 schools)
Helped
a little
(based on approx 270 students at 7 schools)
Yes,
probably
Did you like using Mangahigh?
Yes,
a little
Did your mathematics ability
improve using Mangahigh?
Helped most of
the time
Yes,
definitely
Yes,
a lot
Helped a
huge amount
© Copyright 2009-2010 Mangahigh.com
All rights reserved Various trademarks held by their respective owners.
60
40
20
0
50
40
30
20
10
0
50
40
30
20
10
0
70% of students said they wanted to learn
more mathematics to do better in the games
Student comments
“Mangahigh was a lot more
enjoyable than normal homework”
Kate D, age 13 - Jordanhill
“It’s great fun and really helps me to
learn and understand the world of
mathematics!”
Marc F, age 14 - Jordanhill
“Mangahigh was really fun and I
wish I could play it more during
maths”
Ryan O, age 12 -
Lourdes Secondary School
“Mangahigh, is good, easy to use
and really entertaining. It was good
overall”
Yasin T, age 12 -
Lourdes Secondary School
“I love Mangahigh because it helps
me to improve my maths. Without it I
would not be so good. Thank you”
Kirstie W, age 12 -
Cleveden Secondary School
“Would like to see more games and
it was fantastic”
Jodi L, age 11 -
Lochend Community high school

Really
disike
Teacher response to Mangahigh
How would you best describe the general response
from students towards using Mangahigh?
Dont
like
Neither like
nor dislke
(Feedback from 14 teachers from 7 schools)
Like a bit Really
like
While using Mangahigh, how would you compare
the general level of student concentration
compared to a 'typical' lesson?
A lot less A little less The same A little more A lot more
(Feedback from 14 teachers from 7 schools)
How would you compare the number of mathematical
calculations performed using Mangahigh
versus a 'traditional' lesson?
Significantly
fewer
Slightly
fewer
(Feedback from 13 teachers from 7 schools)
Same Slightly
more
Significantly
more
© Copyright 2009-2010 Mangahigh.com
All rights reserved Various trademarks held by their respective owners.
10
7.5
5
2.5
0
6
4
2
6
0
4
2
0
Teacher comments
“Exciting and stimulating”
Loudres teacher
“Very positive experience”
Lochend teacher
“It has been very worthwhile and
enjoyable”
Catriona Brand, Cleveden
“Pupils are motivated to do maths
when they know that part of their
experience will be with Mangahigh”
John Dowman, Lourdes
“MH encourages the pupils to not
only enhance their skills but to also
take up new challenges and attempt
topics that they have never seen”
Catriona Brand, Cleveden
“The pupils are definitely motivated
by the competition element”
Catriona Brand, Cleveden
“The pupils have enjoyed it and in
particular have engaged with the
games”
Lochend teacher
“Most students were actively
engaged when using Mangahigh
and able to overcome some
problems that I don't think they
would have persevered with so much
if it were a paper based exercise they
were doing”
Knightswood teacher

Performance improvement
• Students sat tests based on the mathematical
• Concepts covered by Mangahigh games.
• Tests were taken before and after the trial.
• Results showed an overall improvement in understanding.
Average test
scores before
Mangahigh
(based on approx 260 students at 6 schools)
Average
Prodigi®
score for
a class
10000
8000
6000
4000
2000
Increase of 13%
0
Average test
scores after
Mangahigh
Greater use of Mangahigh means higher results
and more pass marks
70%
60%
50%
40%
30%
20%
10%
0 15 30 45 60
(Results from 44 classes in 9 schools)
© Copyright 2009-2010 Mangahigh.com
All rights reserved Various trademarks held by their respective owners.
0%
Pass mark
Average number of questions attempted for a specific lesson
More teacher comments
“Enjoyable, pupils asked for
knowledge to help proceed in the
game“
Mandy McCreaner – Cleveden
“A positive experience the software is
really good and enjoyed setting the
class assignments” Lochend teacher
“I thought it was refreshing and
extremely enjoyable.”
Marie-Clare Doherty, Jordanhill
“I think MH has improved both
problem solving skills and confidence
among the students because they
were not afraid to try the problems
and they had to think for
themselves.”
Catriona Brand, Cleveden
“Increased motivation and healthy
competitiveness”
Lourdes teacher
“Huge variety of lessons in Prodigi for
all levels. It is excellent to see games
for higher levels too.”
Sinead OʻGrady, Lourdes
“Excellent consolidation and
extension work in an enjoyable
format”
K McQuillan, Jordan Hill

Method of calculation for mathematical
improvement from trials
10 Schools provided data for use with Mangahigh. This included a mixture
of games and Prodigi® use as well as attitudinal surveys and maths tests,
both pre and post trial.
7 schools responded with pre trial and post trial Maths tests.
Of these 7 schools, 6 schools had an average number of games played
per pupil of between 10 and 25 games. One school had an average of just
3 games played per student and the impact of the games on pupils at that
school was excluded.
In consequence the data from 6 schools was used to analyse any changes
in maths scores. 260 student results met the criteria for analysis (see
below).
The following list shows the number of students at each school who sat pre
and post evaluation Maths tests that were used in the analysis:
Lourdes: 70, Knighstwood: 43, Lochend: 84, St-Paulʼs: 21, Cleveden: 30,
Jordanhill: 12.
Each class sat a maths test of 10 questions based on the mathematical
concepts in each Mangahigh games:
• Test on mental mathematics and order of operations (Bidmas Blaster)
• Test on mental mathematics and estimation (Ice Ice Maybe)
• Test on shape (Pyramid Panic)
• Test on transformations (TranStar)
• Test on equations of lines and curves (Save Our Dumb Planet)
• Test on ordering fractions, decimals and percentages (Flower Power)
Following a period of approximately 10 weeks, (periods varied between 3
weeks and 12 weeks for different schools) each class sat a maths test of
10 questions that related just to the games they used within the trial period.
These post tests contained questions that were of identical difficulty to the
pre-trial questions. The tests based on the games ʻTranStarʼ and ʻSave Our
Dumb Planetʼ contain a substantial amount of material that was not
appropriate for the age or ability of the students who took the pre and post
Maths tests.
Over 90% of the students were from the first two years of secondary
education. The majority of the rest of the students were in foundation
groups.
This meant that, in most cases students completed post trial maths tests
based on material drawn from one of the following games:
• Test on mental mathematics and order of operations (Bidmas Blaster)
• Test on mental mathematics and estimation (Ice Ice Maybe)
• Test on shape (Pyramid Panic)
• Test on ordering fractions, decimals and percentages (Flower Power)
The results of the trial are therefore based on tests drawn from material
covered by the four games above.
Where a class recorded fewer than 8 results for a particular game, the
results were not included due to an insufficient sample size.
Where a student or class recorded multiple initial or final test results the
results were not included in the analysis.
Where more than half a class did not play a particular game, the test
results for that game by that class were not used.
Each question in each test was answered from a multiple choice of four
options.
A correct answer scored 1 point and an incorrect answer 0 points.
Each class was considered separately and the average score (out of 10)
for the pre and post trial tests was calculated for each game that the class
played. Then, taking each class separately, the percentage change in the
average score for each game was then calculated.
The mean of these percentage changes was then calculated to give an
overall percentage change in test scores for each school.
The results of the 6 schools were as follows (names of schools removed):
12.2%, 15.8%, 8.5%, 23.7%, 13.7%, 12.4%
The contribution to the overall percentage change was weighted against
the sample size from each school (listed previously).
© Copyright 2009-2010 Mangahigh.com
All rights reserved Various trademarks held by their respective owners.
This showed an overall improvement in test scores of 13% (to the nearest
percentage).

Contact us:
visit: www.mangahigh.com
email: info@mangahigh.com
call: (020) 7307 8670
© Copyright 2009-2010 Mangahigh.com
All rights reserved Various trademarks held by their respective owners.