Student response to Mangahigh - PDST

Method of calculation for mathematical
improvement from trials
10 Schools provided data for use with Mangahigh. This included a mixture
of games and Prodigi® use as well as attitudinal surveys and maths tests,
both pre and post trial.
7 schools responded with pre trial and post trial Maths tests.
Of these 7 schools, 6 schools had an average number of games played
per pupil of between 10 and 25 games. One school had an average of just
3 games played per student and the impact of the games on pupils at that
school was excluded.
In consequence the data from 6 schools was used to analyse any changes
in maths scores. 260 student results met the criteria for analysis (see
below).
The following list shows the number of students at each school who sat pre
and post evaluation Maths tests that were used in the analysis:
Lourdes: 70, Knighstwood: 43, Lochend: 84, St-Paulʼs: 21, Cleveden: 30,
Jordanhill: 12.
Each class sat a maths test of 10 questions based on the mathematical
concepts in each Mangahigh games:
• Test on mental mathematics and order of operations (Bidmas Blaster)
• Test on mental mathematics and estimation (Ice Ice Maybe)
• Test on shape (Pyramid Panic)
• Test on transformations (TranStar)
• Test on equations of lines and curves (Save Our Dumb Planet)
• Test on ordering fractions, decimals and percentages (Flower Power)
Following a period of approximately 10 weeks, (periods varied between 3
weeks and 12 weeks for different schools) each class sat a maths test of
10 questions that related just to the games they used within the trial period.
These post tests contained questions that were of identical difficulty to the
pre-trial questions. The tests based on the games ʻTranStarʼ and ʻSave Our
Dumb Planetʼ contain a substantial amount of material that was not
appropriate for the age or ability of the students who took the pre and post
Maths tests.
Over 90% of the students were from the first two years of secondary
education. The majority of the rest of the students were in foundation
groups.
This meant that, in most cases students completed post trial maths tests
based on material drawn from one of the following games:
• Test on mental mathematics and order of operations (Bidmas Blaster)
• Test on mental mathematics and estimation (Ice Ice Maybe)
• Test on shape (Pyramid Panic)
• Test on ordering fractions, decimals and percentages (Flower Power)
The results of the trial are therefore based on tests drawn from material
covered by the four games above.
Where a class recorded fewer than 8 results for a particular game, the
results were not included due to an insufficient sample size.
Where a student or class recorded multiple initial or final test results the
results were not included in the analysis.
Where more than half a class did not play a particular game, the test
results for that game by that class were not used.
Each question in each test was answered from a multiple choice of four
options.
A correct answer scored 1 point and an incorrect answer 0 points.
Each class was considered separately and the average score (out of 10)
for the pre and post trial tests was calculated for each game that the class
played. Then, taking each class separately, the percentage change in the
average score for each game was then calculated.
The mean of these percentage changes was then calculated to give an
overall percentage change in test scores for each school.
The results of the 6 schools were as follows (names of schools removed):
12.2%, 15.8%, 8.5%, 23.7%, 13.7%, 12.4%
The contribution to the overall percentage change was weighted against
the sample size from each school (listed previously).
© Copyright 2009-2010 Mangahigh.com
All rights reserved Various trademarks held by their respective owners.
This showed an overall improvement in test scores of 13% (to the nearest
percentage).