STATISTICSCORNER
PROJECTUPDATE References 1. R. Steketee,A Macheso, D Heymmn, et al. A decade of progress in malaria policy and program development in Malawi: 1984-1993. United States Agency for Intemational Development md US depafiment of Health md Human Sewices. 1995. 2. TR Cullinan et al. Packaged treatment for firstline cue in cerebral malria meningitis. Bull WHO | 998:16(3):257 -64. 3. BlolandPB,ELackdtz,PKazembe,etal. Beyondchloroquine:implicationsofdrug resistancefor evaluating maltria therapy efficacy and treatment policy in Africa. Journal of Infectious Diseases 1993;167(4):932-7. 4. Kidane G, R Monow. Teaching mothers to provide home treatment of malaria in Tigray, Ethiopia: a rmdomized trial. Lancet 2000;356(9229):550-5. 5. S J Rogerson et al. Intemittent sulfadoxine-pyrimethamine in pregnancy: effectiveness against malaria morbidity in Blantyre, Malawi in 1997 -99. Trans of the Royal Soc of Trop Med and Hygiene. 2000;94:1-5. 6. Greenwood BM, et al. The effects of malaria chemoprophylaxis given by traditional bifih attendants on the course and outcome of pregnancy. Trans R Soc Trop Med Hyg 1989;83:589-94. 7. Steketee RW et al. The effect of malaria and malaria prevention in pregnancy on off spring birth weight, prematudty, md intrauterine growth retardation in rural Malawi. Am J Trop Med Hyg 1996;55(1):33-41. 8. Binka FN, et al. Impact of permethrin impregnated bednets on child mortality in Kassena-Nankma District, Ghana: a randomized controlled trial. Trop Med Interl Health 1996:lQ):141- 54. 9. Nevill CG, et al. insecticide-treated bednets reduce mortality md severe morbidity from malaria mong children on the Kenym coast. Trop Med Interl Health 1996:1(2):139-46. I 0. Alonso PL, et al. A malria control trial using insecticide-treatedbed nets md ttrgeted chemoprophylaxis in a rural rea of The Gambia, West Africa. 6. The impact of the interventions on mortality and morbidity from malaria. Trans R Soc Trop Med Hyg | 993:87tSuppl 2):37-44. 11.Ter Kuile FO, Terlouw DJ, Phillips-Howard PA, et al.. Permethrin-treatedbednets reduce malaria in pregnancy in an area of intense perennial malaria transmission in western Kenya. Abstract 1 1, Amer Soc Trop Med Hygiene 48th Annual Meeting, Nov 28-Dec 2, 1999; Washington DC.
STATISTICSCORNER S White Welcome to this new feature in the JoumaMn this "Corner" we will look at a specific situation and the statistical methods that can be applied to the analysis of data. Consider this scenario: You want to know how well a simple test may diagnose something, and save the need for a more dfficult' invasive or expensivetest that would provide a definitive answer. For example, in a patient with palpable lymph nodes, is the c&usetuherculosis or not? To answer this with confidence you need an invasive procedure, preferably excision biopsy and histology (EBH). But such a procedure is not only invasive but also difficult, expensive and slow. You may therefore want to know how well a non-invasive, quick method would provide the answer. One method (M7) could be whether simple examination finds 'matted' or not, ie whether they seem to be that the nodes are stuck together in groups. Another method could be a Mantoux (tuberculin) test (M). You want to evaluate each of thesetests on its own for usefulnessin diagnosing TB as the cause of the lymphadenopathy. You also want to know if one test is superior to the other. You plan to conduct a study to evaluate and compare these two methods. The best design uses each method, M 1 and M2, independently, as well as the invasive one, on all patients studied who have palpable lymph nodes. In all the calculations that follw, we will assume that Excission Biopsy and History '7old standard' (EBm is l00%o accurate - it will serve as ovr How should you plan to analyse your data? You will need to select statistical tests to: A Evaluate each method; B
Compare the two methods,
We will consider these questionsin turn. To illustrate the statistical tests to be described suppose you have collected data on 100 patients. Some of the data are shown in Table 1 (this only shows 6 patients - the full table would list all 100 cases),which can be summa.risedin a three-way cross-tabulation(Table 2).
le I Datalisting accordingto diagnosticmethod(EBH = excisionUtpsy andhistology,M1 = pa$ation,M2 = mantouxtest)
Patient ITBTBTB 2 3 4TBTBTB 5 (etc): 100
EBH
M1
M2
TB NOTTB
NotTB NOTTB
TB TB
Not TB : Not TB
Not TB : TB
not TB : not TB Malawi Medical Joumal
STMISTICSCORNER .Tssessment of the accuracy of each method When the true diagnosisis known there are four commonly used statisticsto measurethe accuracy.Do you know what they are? Sensitivity is one of the four methods. It indicates how many of those with the disease(TB) are correctly classified. It can be calculatedusing the first column of figures in Table 2. Table 2 Cross-tabulationof classifications using methods Ml andM2 by actualTB sratus(deteminedby EBff) Result of methods Mt
M1
Actual status TB
Comparison of the two methods The first step is to decide what to compare and the secondis to identify a table (or tables) that summarisethe data in a suitable way. What should we compare? Can you identify a two-by-two cross-tabulationthat would be appropriate? Four optionsthat you could considerare shown in Tables4 to 7, each of which can be derived from Table 2. Let's look at what thesetablestell us.
Not TB
TBTB39O TB
Not TB
18
1l
NotTB
TB
0
2
Not TB
Not TB
6
24
63
tt
Total
From visual inspectionof this Table we see some differences, but how can we compare the accuracy of the two tests?
Thencethe sensitivity of method M1 is 907o ({39+18}/63) and ior method M2 it is 627o ({39+0}/63). So, in the sample consideredM' is more sensitivethanM2 - ie it is betterat diagnos-
Table 4 simply tells us how many of the samplewere classified as diseased,without reference to the true status. A1l this table tells us is that more subjectswere classified as diseasedusing method M1. Table 4 Cross-tabulationof Classifications by methodsused Method
Classification TB
NotTB
Ml
68
JL
M2
41
59
in,eTB when the patient has TB. The second method is specificity. This indicates how many caseswithoul TB are coffect identified. This can be calculated using the secondcolumn of figures in Table 2. The specificity of method Ml is 707o (26/31) and for method M2 it is 95Vo '35/37). So now we see that M, is more specificin this sample than M, - ie it is better at not diagnosing TB when the patient doesnot have TB. In summarymethodM, is more sensitivebut .11_. seemsmore specific. Thesetwo statisticsindicatethe proporlionscoruectlyclassified. Bv contrast the positive predictive value fand negative predictive valuel indicate what proportion of thoseclassifiedto have -not havel the diseaseactuallydo [not] have it. Thesevaluesare Jependenton the mix of proportions in the sampleused. To find the positive predictive value for M1, we must first count all thosepatientsdiagnosedas TB by M1 (39+0+18+11=68),and ihen ask 'what percentageof thesepatientsactually have TB?' The numberwith TB is actually 39+18=51,so the PPV is 57168, t 847a. Use the same approach to calculate the PPV for M2, -rndthe NPVs for both M 1 and M2 (seeTable 3). Table 3 Statistic
Table 5 Cross-tabulationof correctnessof classification by method Method
Classification
Ml M2
Correct
Incorrect
83 14
t7 26
Tables 6 andl cross-tabulatethe sensitivity and specificity data for the two methods respectively. (The sensitivities of each method are derived from the (emboldened)marsins of Table 6.)
Summary statisticsfor the two methods Method Mt
Method M2
Sensitivity Specificity Positive predictive 'u'alue
907o 70Vo
627o 95Vo
84Vo (5V168)
95Va (39/41)
Negative predictive Yalue
8l7o (26/32)
1;hli Medical Jomal
Table 5 does use the true diagnosis,enabling us to know what proportion of the subjects were conectly classified by each method. So now method M, seemsto be better. However a problem with this table is that it gives no indication of whether the method is both sensitiveand specific. We've already seen that Ml seemsto win on sensitivitvand M2 on specificitv.
597o (35/59)
Table 6 Cross-tabulationof classifications by methods for pauents orseaseo
Method
Method M7
Total
M2
TB
TB
39 (a)
0 (c)
39
NotTB
18(b)
6 (d)
24
Total
5/
Not TB
63
STATISTICSCORNER Table 7 Cross-tabulationof classificationsby methods for non-diseasedpatients Total
MethodM1
Method
TB
M2
NotTB
For our example the 95Vo confidence interval is b ie (21.1Vo,36.1Vo). Similarly a95Vo confidenceinterval for the dif31.2Vo).So methodMl isbetferencein sensitivitiesis(.17.5Vo, ter in at least 27Voof patients who have TB, but worse in at least I77o patientswho do not have TB.
7.
In this example the two methods are significantly different both in terms of sensitivity and specificity. In this casethe method to 35 11 Not Tb be preferred depends on whether sensitivity or specificity is 37 26 11 Total more important. The priority might be to diagnose those who have the diseasecorrectly, and thus for sensitivity to be more important. But this has to be balancedwith the numbers of indiThese are the two tableswe'll use, to comparethe sensitivities viduals who are misdiagnosed, ie who really have a different and specificities. But before we identify a test to tse, what are disease. The lower the prevalenceof the diseasethe higher is the our hypotheses? proportion that is wrongly diagnosedas diseased.
0
TB
1
aA
The null hypothesis (to be acceptedif the result is not significant) is that the methods are equally accurate, or that each method is equally likely to give a correct classification. The alternative is that one method is better than the other. A simple statisticaltest you might consideris Pearson'sXtest or Fisher's exact. For Table 6 there are too few observationsin the secondcolumn to use Pearson's)P. For Fisher's exact test the two-tailed p-value is 0.004. This indicates that there is a significant associationbetween the classifications from the two methods. But that is to be expected.What it doesn'ttell us is whether one test is better than the other. A test that compares the error rates using the two methods is McNemar's testl. Let's re-considerTable 6. When the two methods agree (cells labelled (a) and (d)) this contributes nothing to McNemar's test statistic. His test compares the pairs of classifications where there is disagreementbetween the assessors (cells (b) and (c)). The test statistic can be derived using the formula:
Summary Statisticscommonly usedto assessaccuracyof a diagnostictool are sensitivity,specificity,positivepredictive value and negative predictive value. If two diagnostic tools are to be compared and both have been applied to the same subjects then McNemar's test provides a method of testing whether there are any differencesbetween the accuracyof two tests. A confidence interval for the difference in sensitivities or specificities can also be formed. These enable the difference to be estimated and hence for a valid statistically basedcomparisonto be made. Invitation If you have a suggestionof a statistical issue that you would like to be considered in this Corner please send it to me, either by email: [email protected] at the Department of Community Health, College of Medicine, Private Bag 360' Chichiri, Blantyre 3. S White
x-
{lb-cl-I}' b+c
,or equivalently.T =
[b - c]-1 /b*"
Reference 1. Altman Chanmm
DG. Practical
Statistics for Medical
and HaIVCRC,
Research. Boca Raton; London;
New
York:
1991; 237 - 259.
distributionland z with the Standard F is comparedwith a Normal distributions).For Table2X=16.1, for which p
PROJECTUPDATE References 1. R. Steketee,A Macheso, D Heymmn, et al. A decade of progress in malaria policy and program development in Malawi: 1984-1993. United States Agency for Intemational Development md US depafiment of Health md Human Sewices. 1995. 2. TR Cullinan et al. Packaged treatment for firstline cue in cerebral malria meningitis. Bull WHO | 998:16(3):257 -64. 3. BlolandPB,ELackdtz,PKazembe,etal. Beyondchloroquine:implicationsofdrug resistancefor evaluating maltria therapy efficacy and treatment policy in Africa. Journal of Infectious Diseases 1993;167(4):932-7. 4. Kidane G, R Monow. Teaching mothers to provide home treatment of malaria in Tigray, Ethiopia: a rmdomized trial. Lancet 2000;356(9229):550-5. 5. S J Rogerson et al. Intemittent sulfadoxine-pyrimethamine in pregnancy: effectiveness against malaria morbidity in Blantyre, Malawi in 1997 -99. Trans of the Royal Soc of Trop Med and Hygiene. 2000;94:1-5. 6. Greenwood BM, et al. The effects of malaria chemoprophylaxis given by traditional bifih attendants on the course and outcome of pregnancy. Trans R Soc Trop Med Hyg 1989;83:589-94. 7. Steketee RW et al. The effect of malaria and malaria prevention in pregnancy on off spring birth weight, prematudty, md intrauterine growth retardation in rural Malawi. Am J Trop Med Hyg 1996;55(1):33-41. 8. Binka FN, et al. Impact of permethrin impregnated bednets on child mortality in Kassena-Nankma District, Ghana: a randomized controlled trial. Trop Med Interl Health 1996:lQ):141- 54. 9. Nevill CG, et al. insecticide-treated bednets reduce mortality md severe morbidity from malaria mong children on the Kenym coast. Trop Med Interl Health 1996:1(2):139-46. I 0. Alonso PL, et al. A malria control trial using insecticide-treatedbed nets md ttrgeted chemoprophylaxis in a rural rea of The Gambia, West Africa. 6. The impact of the interventions on mortality and morbidity from malaria. Trans R Soc Trop Med Hyg | 993:87tSuppl 2):37-44. 11.Ter Kuile FO, Terlouw DJ, Phillips-Howard PA, et al.. Permethrin-treatedbednets reduce malaria in pregnancy in an area of intense perennial malaria transmission in western Kenya. Abstract 1 1, Amer Soc Trop Med Hygiene 48th Annual Meeting, Nov 28-Dec 2, 1999; Washington DC.
STATISTICSCORNER S White Welcome to this new feature in the JoumaMn this "Corner" we will look at a specific situation and the statistical methods that can be applied to the analysis of data. Consider this scenario: You want to know how well a simple test may diagnose something, and save the need for a more dfficult' invasive or expensivetest that would provide a definitive answer. For example, in a patient with palpable lymph nodes, is the c&usetuherculosis or not? To answer this with confidence you need an invasive procedure, preferably excision biopsy and histology (EBH). But such a procedure is not only invasive but also difficult, expensive and slow. You may therefore want to know how well a non-invasive, quick method would provide the answer. One method (M7) could be whether simple examination finds 'matted' or not, ie whether they seem to be that the nodes are stuck together in groups. Another method could be a Mantoux (tuberculin) test (M). You want to evaluate each of thesetests on its own for usefulnessin diagnosing TB as the cause of the lymphadenopathy. You also want to know if one test is superior to the other. You plan to conduct a study to evaluate and compare these two methods. The best design uses each method, M 1 and M2, independently, as well as the invasive one, on all patients studied who have palpable lymph nodes. In all the calculations that follw, we will assume that Excission Biopsy and History '7old standard' (EBm is l00%o accurate - it will serve as ovr How should you plan to analyse your data? You will need to select statistical tests to: A Evaluate each method; B
Compare the two methods,
We will consider these questionsin turn. To illustrate the statistical tests to be described suppose you have collected data on 100 patients. Some of the data are shown in Table 1 (this only shows 6 patients - the full table would list all 100 cases),which can be summa.risedin a three-way cross-tabulation(Table 2).
le I Datalisting accordingto diagnosticmethod(EBH = excisionUtpsy andhistology,M1 = pa$ation,M2 = mantouxtest)
Patient ITBTBTB 2 3 4TBTBTB 5 (etc): 100
EBH
M1
M2
TB NOTTB
NotTB NOTTB
TB TB
Not TB : Not TB
Not TB : TB
not TB : not TB Malawi Medical Joumal
STMISTICSCORNER .Tssessment of the accuracy of each method When the true diagnosisis known there are four commonly used statisticsto measurethe accuracy.Do you know what they are? Sensitivity is one of the four methods. It indicates how many of those with the disease(TB) are correctly classified. It can be calculatedusing the first column of figures in Table 2. Table 2 Cross-tabulationof classifications using methods Ml andM2 by actualTB sratus(deteminedby EBff) Result of methods Mt
M1
Actual status TB
Comparison of the two methods The first step is to decide what to compare and the secondis to identify a table (or tables) that summarisethe data in a suitable way. What should we compare? Can you identify a two-by-two cross-tabulationthat would be appropriate? Four optionsthat you could considerare shown in Tables4 to 7, each of which can be derived from Table 2. Let's look at what thesetablestell us.
Not TB
TBTB39O TB
Not TB
18
1l
NotTB
TB
0
2
Not TB
Not TB
6
24
63
tt
Total
From visual inspectionof this Table we see some differences, but how can we compare the accuracy of the two tests?
Thencethe sensitivity of method M1 is 907o ({39+18}/63) and ior method M2 it is 627o ({39+0}/63). So, in the sample consideredM' is more sensitivethanM2 - ie it is betterat diagnos-
Table 4 simply tells us how many of the samplewere classified as diseased,without reference to the true status. A1l this table tells us is that more subjectswere classified as diseasedusing method M1. Table 4 Cross-tabulationof Classifications by methodsused Method
Classification TB
NotTB
Ml
68
JL
M2
41
59
in,eTB when the patient has TB. The second method is specificity. This indicates how many caseswithoul TB are coffect identified. This can be calculated using the secondcolumn of figures in Table 2. The specificity of method Ml is 707o (26/31) and for method M2 it is 95Vo '35/37). So now we see that M, is more specificin this sample than M, - ie it is better at not diagnosing TB when the patient doesnot have TB. In summarymethodM, is more sensitivebut .11_. seemsmore specific. Thesetwo statisticsindicatethe proporlionscoruectlyclassified. Bv contrast the positive predictive value fand negative predictive valuel indicate what proportion of thoseclassifiedto have -not havel the diseaseactuallydo [not] have it. Thesevaluesare Jependenton the mix of proportions in the sampleused. To find the positive predictive value for M1, we must first count all thosepatientsdiagnosedas TB by M1 (39+0+18+11=68),and ihen ask 'what percentageof thesepatientsactually have TB?' The numberwith TB is actually 39+18=51,so the PPV is 57168, t 847a. Use the same approach to calculate the PPV for M2, -rndthe NPVs for both M 1 and M2 (seeTable 3). Table 3 Statistic
Table 5 Cross-tabulationof correctnessof classification by method Method
Classification
Ml M2
Correct
Incorrect
83 14
t7 26
Tables 6 andl cross-tabulatethe sensitivity and specificity data for the two methods respectively. (The sensitivities of each method are derived from the (emboldened)marsins of Table 6.)
Summary statisticsfor the two methods Method Mt
Method M2
Sensitivity Specificity Positive predictive 'u'alue
907o 70Vo
627o 95Vo
84Vo (5V168)
95Va (39/41)
Negative predictive Yalue
8l7o (26/32)
1;hli Medical Jomal
Table 5 does use the true diagnosis,enabling us to know what proportion of the subjects were conectly classified by each method. So now method M, seemsto be better. However a problem with this table is that it gives no indication of whether the method is both sensitiveand specific. We've already seen that Ml seemsto win on sensitivitvand M2 on specificitv.
597o (35/59)
Table 6 Cross-tabulationof classifications by methods for pauents orseaseo
Method
Method M7
Total
M2
TB
TB
39 (a)
0 (c)
39
NotTB
18(b)
6 (d)
24
Total
5/
Not TB
63
STATISTICSCORNER Table 7 Cross-tabulationof classificationsby methods for non-diseasedpatients Total
MethodM1
Method
TB
M2
NotTB
For our example the 95Vo confidence interval is b ie (21.1Vo,36.1Vo). Similarly a95Vo confidenceinterval for the dif31.2Vo).So methodMl isbetferencein sensitivitiesis(.17.5Vo, ter in at least 27Voof patients who have TB, but worse in at least I77o patientswho do not have TB.
7.
In this example the two methods are significantly different both in terms of sensitivity and specificity. In this casethe method to 35 11 Not Tb be preferred depends on whether sensitivity or specificity is 37 26 11 Total more important. The priority might be to diagnose those who have the diseasecorrectly, and thus for sensitivity to be more important. But this has to be balancedwith the numbers of indiThese are the two tableswe'll use, to comparethe sensitivities viduals who are misdiagnosed, ie who really have a different and specificities. But before we identify a test to tse, what are disease. The lower the prevalenceof the diseasethe higher is the our hypotheses? proportion that is wrongly diagnosedas diseased.
0
TB
1
aA
The null hypothesis (to be acceptedif the result is not significant) is that the methods are equally accurate, or that each method is equally likely to give a correct classification. The alternative is that one method is better than the other. A simple statisticaltest you might consideris Pearson'sXtest or Fisher's exact. For Table 6 there are too few observationsin the secondcolumn to use Pearson's)P. For Fisher's exact test the two-tailed p-value is 0.004. This indicates that there is a significant associationbetween the classifications from the two methods. But that is to be expected.What it doesn'ttell us is whether one test is better than the other. A test that compares the error rates using the two methods is McNemar's testl. Let's re-considerTable 6. When the two methods agree (cells labelled (a) and (d)) this contributes nothing to McNemar's test statistic. His test compares the pairs of classifications where there is disagreementbetween the assessors (cells (b) and (c)). The test statistic can be derived using the formula:
Summary Statisticscommonly usedto assessaccuracyof a diagnostictool are sensitivity,specificity,positivepredictive value and negative predictive value. If two diagnostic tools are to be compared and both have been applied to the same subjects then McNemar's test provides a method of testing whether there are any differencesbetween the accuracyof two tests. A confidence interval for the difference in sensitivities or specificities can also be formed. These enable the difference to be estimated and hence for a valid statistically basedcomparisonto be made. Invitation If you have a suggestionof a statistical issue that you would like to be considered in this Corner please send it to me, either by email: [email protected] at the Department of Community Health, College of Medicine, Private Bag 360' Chichiri, Blantyre 3. S White
x-
{lb-cl-I}' b+c
,or equivalently.T =
[b - c]-1 /b*"
Reference 1. Altman Chanmm
DG. Practical
Statistics for Medical
and HaIVCRC,
Research. Boca Raton; London;
New
York:
1991; 237 - 259.
distributionland z with the Standard F is comparedwith a Normal distributions).For Table2X=16.1, for which p
