MQP â 014 EQA HANDBOOK EDITION 6 A participant ... - Thistle QA
MQP â 014 EQA HANDBOOK EDITION 6 A participant ... - Thistle QA MQP â 014 EQA HANDBOOK EDITION 6 A participant ... - Thistle QA
P.O. Box 131375, Bryanston, 2074 Ground Floor, Block 5 Bryanston Gate, 170 Curzon Road Bryanston, Johannesburg, South Africa 804 Flatrock, Buiten Street, Cape Town, 8001 www.thistle.co.za Tel: +27 (011) 463 3260 Fax: +27 (011) 463 3036 Fax to Email: + 27 (0) 86-557-2232 e-‐mail : service@thistle.co.za MQP-‐014 Edition 10 How are results statistically treated We have three sets of statistical approaches when grouping results in a database, namely: Stats Level 1: All results are grouped in the same method group, when the results are the same regardless of which method is used. This was our historical default group, and has been changed based on new information from suppliers, poor distribution of results (high SDs), bimodal distribution of results, customer comments/ criticisms and AdCom recommendations. Stats Level 2: Most methods give results within one modal distribution except for one method, e.g. dry chemistry. Stats Level 3: Each method is statistically considered separately as none of the methods available for that test can be compared to any other, e.g. HDL on the Chemistry EQA a) Participants are permitted to use a method of their choice in analysing PT/ EQA samples. We use the information supplied to us by participants on our Method Questionnaire to allocate method groups to participants on our database, based on the methodology of their chosen method. In addition, we receive information on the instrument used and this allows us to allocate the participant to an instrument group, comprising labs using the same methodology on the same instrument. Labs are thus able to look at both aspects on our reports. b) This aspect is assessed on an ongoing basis based on continued monitoring of SDs and information supplied by instrument and reagent suppliers. We use two different, but complementary systems of performance evaluation. Firstly, we produce the usual statistical comparisons of mean and Standard Deviation (SD), sometimes called Standard Deviation Index (SDI). This can only take place after a data clean up procedure. We also use robust mean and SD calculations along with Chauvenet's criterion to exclude outliers, followed by a 95% trim. Chauvenet's criterion is a unidirectional and highly sensitive system that identifies outliers from the mean based on the expected SDI for the number of data points. Consecutive passes are made until the data satisfies the Criterion. At this point, the mean and SD Page 7 of 12
P.O. Box 131375, Bryanston, 2074 Ground Floor, Block 5 Bryanston Gate, 170 Curzon Road Bryanston, Johannesburg, South Africa 804 Flatrock, Buiten Street, Cape Town, 8001 www.thistle.co.za Tel: +27 (011) 463 3260 Fax: +27 (011) 463 3036 Fax to Email: + 27 (0) 86-557-2232 e-‐mail : service@thistle.co.za MQP-‐014 Edition 10 are calculated and all results, including those excluded by Chauvenet’s Criterion, are calculated as numbers of SDI away from the mean. Traditionally, this system is used but has the limitation that 5% of labs will always fall outside the +/- 2 SD range, whether more or less labs should have been labeled as out of control. We have noted that the SDIs have tended to fall from the beginning of our EQA service, thus they are lower now than before. This means that overall performance, as measured by SDI, has improved. However, the continued use of SDI means that 5% of labs still are told they are out of control. If the labs who are outside the 2 SD limit improve their performance, the calculation will become tighter – and 5% will still be told they have a problem. The SD or SDI can thus be called a Variable Limit. It changes as performance changes, but always excludes 5%. It is still useful to use this purely statistical assessment. If your result falls outside 2 SDs, you are in the poorest 5% of performers for that test or analyte. This is definitely real and relevant. We have performed preliminary evaluations of iteration (ISO 13528) to reduce our outlier exclusion to one system. This was successful and will be rolled out onto the various EQAs as we re-write our software, planned for 2013. In addition, we classify results as either poor or acceptable based on a percentage range about the mean called 'Clinical CV'. We introduced this concept of “how well we need to do tests for the result to be clinically useful”, in other words, Clinical Limits. Each analyte has its own Clinical CV, these are percentage ranges about the consensus mean that are considered acceptable. The yardsticks we use for these fixed limits are those published in the USA document CLIA’88 (the Clinical Laboratory Improvement Amendment of 1988), and Biological Variation. In addition a local group of pathologists has evaluated these ranges and given us an African Acceptable Clinical Limits range. If your result is inside these percentage ranges of the consensus mean, it will be called “Acceptable” on your report. Esoteric analytes or those with low participant numbers are treated differently: a) A warning flag will print on the report when the participant number is low. Page 8 of 12
- Page 1 and 2: P.O. Box 131375, Bryanston, 2074 Gr
- Page 3 and 4: P.O. Box 131375, Bryanston, 2074 Gr
- Page 5: P.O. Box 131375, Bryanston, 2074 Gr
- Page 9 and 10: P.O. Box 131375, Bryanston, 2074 Gr
- Page 11: P.O. Box 131375, Bryanston, 2074 Gr
P.O. Box 131375, Bryanston, 2074<br />
Ground Floor, Block 5<br />
Bryanston Gate, 170 Curzon Road<br />
Bryanston, Johannesburg, South Africa<br />
804 Flatrock, Buiten Street, Cape Town, 8001<br />
www.thistle.co.za<br />
Tel: +27 (011) 463 3260<br />
Fax: +27 (011) 463 3036<br />
Fax to Email: + 27 (0) 86-557-2232<br />
e-‐mail : service@thistle.co.za<br />
<strong>MQP</strong>-‐<strong>014</strong><br />
Edition 10<br />
How are results statistically treated<br />
We have three sets of statistical approaches when grouping results in a<br />
database, namely:<br />
Stats Level 1: All results are grouped in the same method group, when the<br />
results are the same regardless of which method is used. This was our historical<br />
default group, and has been changed based on new information from suppliers,<br />
poor distribution of results (high SDs), bimodal distribution of results,<br />
customer comments/ criticisms and AdCom recommendations.<br />
Stats Level 2: Most methods give results within one modal distribution except<br />
for one method, e.g. dry chemistry.<br />
Stats Level 3: Each method is statistically considered separately as none of the<br />
methods available for that test can be compared to any other, e.g. HDL on the<br />
Chemistry <strong>E<strong>QA</strong></strong><br />
a) Participants are permitted to use a method of their choice in analysing<br />
PT/ <strong>E<strong>QA</strong></strong> samples. We use the information supplied to us by <strong>participant</strong>s<br />
on our Method Questionnaire to allocate method groups to <strong>participant</strong>s on<br />
our database, based on the methodology of their chosen method. In<br />
addition, we receive information on the instrument used and this allows us<br />
to allocate the <strong>participant</strong> to an instrument group, comprising labs using<br />
the same methodology on the same instrument. Labs are thus able to look<br />
at both aspects on our reports.<br />
b) This aspect is assessed on an ongoing basis based on continued monitoring<br />
of SDs and information supplied by instrument and reagent suppliers.<br />
We use two different, but complementary systems of performance evaluation.<br />
Firstly, we produce the usual statistical comparisons of mean and Standard<br />
Deviation (SD), sometimes called Standard Deviation Index (SDI). This can only<br />
take place after a data clean up procedure.<br />
We also use robust mean and SD calculations along with Chauvenet's criterion to<br />
exclude outliers, followed by a 95% trim. Chauvenet's criterion is a unidirectional<br />
and highly sensitive system that identifies outliers from the mean<br />
based on the expected SDI for the number of data points. Consecutive passes<br />
are made until the data satisfies the Criterion. At this point, the mean and SD<br />
Page 7 of 12
Change language