Journal of the Pediatric Infectious Diseases Society Advance Access published July 31, 2014
Brief Report
Toward an Improved Scale for Assessing Symptom Severity in Children With Acute Otitis Media Nader Shaikh,1 Alejandro Hoberman,1 Howard E. Rockette,2 Marcia Kurs-Lasky,1 and Jack L. Paradise1 1
University of Pittsburgh School of Medicine, Children’s Hospital of Pittsburgh of UPMC, Division of General Academic Pediatrics, Pennsylvania; and 2University of Pittsburgh Graduate School of Public Health, Department of Biostatistics, Pennsylvania Corresponding Author: Nader Shaikh, MD, MPH, Children’s Hospital of Pittsburgh of UPMC, One Children’s Hospital Drive, 4401 Penn Ave, Pittsburgh, PA 15224. E-mail: [email protected]. Received January 16, 2014; accepted May 30, 2014.
The objective of the present study was to determine whether changes in the previously developed 7-item Acute Otitis Media Severity of Symptoms scale could improve its responsiveness and its longitudinal construct validity. The items “diminished activity” and “diminished appetite” had low or borderline levels of responsiveness and longitudinal construct validity. Dropping these items seems to be potentially advantageous. Key words.
acute otitis media; parent-reported symptoms; severity of symptoms; symptom scale.
In previous studies, we described the development and preliminary evaluation of a parent-reported scale, the Acute Otitis Media Severity of Symptoms (AOM-SOS) scale, for use in tracking the severity of symptoms of children with AOM [1, 2]. The current version of the scale (version 3) consists of 7 discrete items: tugging of ears, crying, irritability, difficulty sleeping, diminished activity, diminished appetite, and fever. Parents are asked to rate these symptoms, as “none,” “a little,” or “a lot,” with corresponding scores of 0, 1, and 2. Thus, scores may range from 0 to 14. In the present study, we analyze data from our recently conducted clinical trial of amoxicillinclavulanate in 291 children 6 to 23 months of age with AOM [3], with a view to improving the AOM-SOS scale’s responsiveness, ie, its ability to detect change, and its construct validity, ie, its ability to assess the actual severity of symptoms. In the clinical trial, we randomly assigned 291 children 6 to 23 months of age with AOM to receive either amoxicillin-clavulanate or placebo for 10 days [3]. We asked parents to rate their child’s symptoms on the version 3 AOM-SOS scale twice daily for 3 days, and daily thereafter for 7 additional days. We defined clinical failure (1) at or before day 4–5 as either failure to achieve substantial improvement in symptoms or worsening of otoscopic signs and (2) at or before day 10–12 as the persistence of otoscopic signs of acute infection, without regard to the persistence or resolution of middle-ear effusion. We found that using the AOM-SOS, results were modestly more favorable
among children who received amoxicillin-clavulanate than among those who received placebo. More substantial, however, were differences favoring the amoxicillinclavulanate group in the rates of clinical failure. This difference pointed to a need to consider whether the AOM-SOS scale could be refined further. In approaching this task, we also took into account data from an earlier study in which we examined the degree to which individual scale items correlated with parental assessment of ear pain. In that study, we found that the symptoms parents considered most likely to indicate ear pain were ear tugging and irritability, and that those least likely to do so were diminished activity and diminished appetite [4]. METHODS Measuring Responsiveness of Individual AOM-SOS Symptoms
Given that the median time to resolution for most symptoms was 1–2 days, we focused on the change from the baseline version 3 AOM-SOS score to the score on the afternoon or evening of day 2, a span, on average, of 32 hours. To measure responsiveness of individual items we used the standardized response mean (SRM), calculated by dividing the change in score by the standard deviation of the change, in the subgroup of children who did not experience clinical failure, ie, in children who were considered improved by study investigators [5, 6]. An SRM >0.5 is generally considered adequate to detect change on a day-to-day basis [7].
Journal of the Pediatric Infectious Diseases Society pp. 1–3, 2014. DOI:10.1093/jpids/piu062 © The Author 2014. Published by Oxford University Press on behalf of the Pediatric Infectious Diseases Society. All rights reserved. For Permissions, please e-mail: [email protected].
2
Shaikh et al
Measuring Longitudinal Construct Validity of Individual AOM-SOS Symptoms
For a scale intended to assess symptom trajectories over time to be valid, it needs to discern groups with clearly different symptom trajectories. Based on studies to date [3, 8–11], one would expect children treated with amoxicillin-clavulanate to have fewer symptoms over time attributable to AOM than children treated with placebo. The same is perforce the case with children deemed to have had a successful clinical outcome compared with those categorized as having failed treatment. Using generalized estimating equations, for each item of the version 3 AOM-SOS scale, we examined whether the distribution of scores during the initial 7 days of treatment differed as between the 2 treatment groups (amoxicillin-clavulanate vs placebo) and as between the 2 outcome groups (clinical success vs clinical failure). Comparison of Results Using Version 3 and Version 4 of the AOM-SOS Scale
We combined findings of this report to consider whether any modification of version 3 appeared to show promise, and if so, how using that modified version in the aforementioned amoxicillin-clavulanate vs placebo trial [3] in place of version 3 might have affected the trial’s results. Namely, we compared the results using the 2 versions on the following 3 outcomes: (1) the proportion of children resolving, with resolution defined as reaching an AOM-SOS score of 0 or 1; (2) the proportion of children resolving with resolution, defined by 2 successive AOM-SOS scores of 0 or 1; and (3) mean AOM-SOS score over the first 7 days. For the analyses of time to resolution of symptoms, we used tests of equal hazard functions using a proportional hazards model, and for the mean score we used and generalized estimating equations. RESULTS Responsiveness of Individual AOM-SOS Items
The respective SRM values of each of the 7 version 3 AOM-SOS scale items, in decreasing order of magnitude,
were 0.79, 0.76, 0.74, 0.68, 0.62, 0.48, and 0.38 for fussiness, sleep disturbance, fever, crying, tugging, diminished activity, and diminished appetite. Longitudinal Construct Validity of Individual AOM-SOS Items
The distribution of scores during the initial 7 days of treatment differed significantly as between the 2 treatment groups (amoxicillin-clavulanate vs placebo) for difficulty sleeping (P = .03) and fever (P = .005); differences regarding ear tugging (P = .06) and irritability (P = .09) were borderline; and differences regarding crying (P = .24), diminished activity (P = .26) , and diminished appetite (P = .55) were not significant. The distribution of scores during the initial 7 days of treatment differed significantly as between the 2 outcome groups (clinical success vs clinical failure), irrespective of treatment group, for ear tugging (P = .005), crying (P = .04), irritability (P = .004), and difficulty sleeping (P = .04); the difference regarding diminished activity (P = .09) was borderline; differences regarding diminished appetite (P = .11) and fever (P = .15) were not significant. Combined Analyses Regarding the Version 3 AOM-SOS Scale, and Comparison With Results Using a Proposed Revised Scale (Version 4)
Table 1 summarizes the results of these analyses regarding individual symptoms, together with results of the aforementioned earlier analysis of parental opinions about those symptoms in relation to ear pain [4]. When the findings from the various analyses were taken as a whole, diminished activity and diminished appetite emerged as the symptoms that appeared weakest psychometrically, as indicated by the heavy-lined rectangle in the table. Given these findings, we deleted those 2 symptoms from version 3 of the AOM-SOS scale to form a revised, 5-item version 4. We then compared results applying version 4 to the data on symptomatic response in the aforementioned clinical trial with the results that had been obtained in the trial using version 3 as described previously [3]. For each of
Table 1. Summary of Psychometric Properties of Individual Version 3 AOM-SOS Scale Items* Responsive?
Valid in Differentiating Treatment Groups?
Valid in Differentiating Outcome Groups?
Indicative of Otalgia (according to parents) [4]?
Ear tugging Crying Irritability Difficulty sleeping
Yes Yes Yes Yes
Borderline No Borderline Yes
Yes Yes Yes Yes
Yes Yes Yes Yes
Diminished activity Diminished appetite
Borderline No
No No
Borderline No
Borderline Borderline
Fever
Yes
Yes
No
Yes
Abbreviation: AOM-SOS, Acute Otitis Media Severity of Symptoms. *Heavy-lined rectangle identifies the 2 items with the weakest psychometric properties.
Modification of the AOM-SOS Scale
these analyses, results applying version 4 closely paralleled results that had been obtained using version 3, the main difference being a lesser degree of variability in the results when using version 4, with a corresponding increase in the levels of confidence (lower P values) concerning the trial’s results. The P values corresponding to analyses 1, 2, and 3 described in the methods using the AOM-SOS version 4 were .06, .01, and .005, respectively. The corresponding values for version 3 were .14, .04, and .02. DISCUSSION In considering how the AOM-SOS scale might be further refined, we used a 3-pronged approach. First, in examining the responsiveness of each item in the scale, we attempted to identify items that exhibited little change in patients who were clearly improving in other important clinical respects. Such unresponsive items, even if they might seem important to parents or might be valid measures of AOM symptoms, would impede the ability of the scale as a whole to measure change. Second, because no reference standard exists for measuring improvement or worsening of symptoms, comparing symptoms in groups expected to differ from each other clinically constitutes one of the best methods of assessing the symptoms’ construct validity. Thus, we sought to determine, for the 7 symptoms of the version 3 AOM-SOS scale, the degree to which each showed differences over time as between the 2 treatment groups (amoxicillin-clavulanate vs placebo) and as between the 2 outcome groups (clinical success vs clinical failure). Third, we examined parental views regarding the importance of each scale item in regard to the parents’ estimates of their children’s ear pain. From the various analyses described above and summarized in Table 1, it appeared that eliminating the “diminished activity” and “diminished appetite” items from the 7-item version 3 of the AOM-SOS scale might improve the scale. What we found was that clinical trial results applying the reduced, 5-item version 4 were virtually indistinguishable from the results that had been obtained using version 3. That finding notwithstanding, the parsimony gained by moving from a 7-item to a 5-item scale, and the reduction in variability that we found in the clinical trial’s results when applying the 5-item scale, with a
corresponding increase in the levels of confidence concerning those results, suggest that change to the 5-item scale in subsequent clinical trials would be advantageous. However, before considering adoption of this change in the scale, confirmation of our findings will be required in other study populations, preferably in other locales, through analyses of the type described in the present report. Acknowledgments Potential conflicts of interest. The University of Pittsburgh owns the copyrights of the AOM-SOS scale. N. S., A. H., and J. L. P. receive remuneration if the scale is licensed to a for-profit entity. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.
References 1. Shaikh N, Hoberman A, Paradise JL, et al. Responsiveness and construct validity of a symptom scale for acute otitis media. Pediatr Infect Dis J 2009; 28:9–12. 2. Shaikh N, Hoberman A, Paradise JL, et al. Development and preliminary evaluation of a parent-reported outcome instrument for clinical trials in acute otitis media. Pediatr Infect Dis J 2009; 28: 5–8. 3. Hoberman A, Paradise JL, Rockette HE, et al. Treatment of acute otitis media in children under 2 years of age. N Engl J Med 2011; 364:105–15. 4. Shaikh N, Kearney DH, Colborn DK, et al. How do parents of preverbal children with acute otitis media determine how much ear pain their child is having? J Pain 2010; 11:1291–4. 5. Beaton DE, Bombardier C, Katz JN, Wright JG. A taxonomy for responsiveness. J Clin Epidemiol 2001; 54:1204–17. 6. Terwee CB, Dekker FW, Wiersinga WM, et al. On assessing responsiveness of health-related quality of life instruments: guidelines for instrument evaluation. Qual Life Res 2003; 12:349–62. 7. Husted JA, Cook RJ, Farewell VT, Gladman DD. Methods for assessing responsiveness: a critical review and recommendations. J Clin Epidemiol 2000; 53:459–68. 8. Le Saux N, Gaboury I, Baird M, et al. A randomized, doubleblind, placebo-controlled noninferiority trial of amoxicillin for clinically diagnosed acute otitis media in children 6 months to 5 years of age. Can Med Assoc J 2005; 172:335–41. 9. Little P, Gould C, Williamson I, et al. Pragmatic randomised controlled trial of two prescribing strategies for childhood acute otitis media. BMJ 2001; 322:336–42. 10. Sanders SL, Glasziou PP, Del Mar CB, Rovers MM. Antibiotics for acute otitis media in children. Cochrane database of systematic reviews (Online) 2004:CD000219. 11. Tähtinen PA, Laine MK, Huovinen P, et al. A placebo-controlled trial of antimicrobial treatment for acute otitis media. N Engl J Med 2011; 364:116–26.
3
Brief Report
Toward an Improved Scale for Assessing Symptom Severity in Children With Acute Otitis Media Nader Shaikh,1 Alejandro Hoberman,1 Howard E. Rockette,2 Marcia Kurs-Lasky,1 and Jack L. Paradise1 1
University of Pittsburgh School of Medicine, Children’s Hospital of Pittsburgh of UPMC, Division of General Academic Pediatrics, Pennsylvania; and 2University of Pittsburgh Graduate School of Public Health, Department of Biostatistics, Pennsylvania Corresponding Author: Nader Shaikh, MD, MPH, Children’s Hospital of Pittsburgh of UPMC, One Children’s Hospital Drive, 4401 Penn Ave, Pittsburgh, PA 15224. E-mail: [email protected]. Received January 16, 2014; accepted May 30, 2014.
The objective of the present study was to determine whether changes in the previously developed 7-item Acute Otitis Media Severity of Symptoms scale could improve its responsiveness and its longitudinal construct validity. The items “diminished activity” and “diminished appetite” had low or borderline levels of responsiveness and longitudinal construct validity. Dropping these items seems to be potentially advantageous. Key words.
acute otitis media; parent-reported symptoms; severity of symptoms; symptom scale.
In previous studies, we described the development and preliminary evaluation of a parent-reported scale, the Acute Otitis Media Severity of Symptoms (AOM-SOS) scale, for use in tracking the severity of symptoms of children with AOM [1, 2]. The current version of the scale (version 3) consists of 7 discrete items: tugging of ears, crying, irritability, difficulty sleeping, diminished activity, diminished appetite, and fever. Parents are asked to rate these symptoms, as “none,” “a little,” or “a lot,” with corresponding scores of 0, 1, and 2. Thus, scores may range from 0 to 14. In the present study, we analyze data from our recently conducted clinical trial of amoxicillinclavulanate in 291 children 6 to 23 months of age with AOM [3], with a view to improving the AOM-SOS scale’s responsiveness, ie, its ability to detect change, and its construct validity, ie, its ability to assess the actual severity of symptoms. In the clinical trial, we randomly assigned 291 children 6 to 23 months of age with AOM to receive either amoxicillin-clavulanate or placebo for 10 days [3]. We asked parents to rate their child’s symptoms on the version 3 AOM-SOS scale twice daily for 3 days, and daily thereafter for 7 additional days. We defined clinical failure (1) at or before day 4–5 as either failure to achieve substantial improvement in symptoms or worsening of otoscopic signs and (2) at or before day 10–12 as the persistence of otoscopic signs of acute infection, without regard to the persistence or resolution of middle-ear effusion. We found that using the AOM-SOS, results were modestly more favorable
among children who received amoxicillin-clavulanate than among those who received placebo. More substantial, however, were differences favoring the amoxicillinclavulanate group in the rates of clinical failure. This difference pointed to a need to consider whether the AOM-SOS scale could be refined further. In approaching this task, we also took into account data from an earlier study in which we examined the degree to which individual scale items correlated with parental assessment of ear pain. In that study, we found that the symptoms parents considered most likely to indicate ear pain were ear tugging and irritability, and that those least likely to do so were diminished activity and diminished appetite [4]. METHODS Measuring Responsiveness of Individual AOM-SOS Symptoms
Given that the median time to resolution for most symptoms was 1–2 days, we focused on the change from the baseline version 3 AOM-SOS score to the score on the afternoon or evening of day 2, a span, on average, of 32 hours. To measure responsiveness of individual items we used the standardized response mean (SRM), calculated by dividing the change in score by the standard deviation of the change, in the subgroup of children who did not experience clinical failure, ie, in children who were considered improved by study investigators [5, 6]. An SRM >0.5 is generally considered adequate to detect change on a day-to-day basis [7].
Journal of the Pediatric Infectious Diseases Society pp. 1–3, 2014. DOI:10.1093/jpids/piu062 © The Author 2014. Published by Oxford University Press on behalf of the Pediatric Infectious Diseases Society. All rights reserved. For Permissions, please e-mail: [email protected].
2
Shaikh et al
Measuring Longitudinal Construct Validity of Individual AOM-SOS Symptoms
For a scale intended to assess symptom trajectories over time to be valid, it needs to discern groups with clearly different symptom trajectories. Based on studies to date [3, 8–11], one would expect children treated with amoxicillin-clavulanate to have fewer symptoms over time attributable to AOM than children treated with placebo. The same is perforce the case with children deemed to have had a successful clinical outcome compared with those categorized as having failed treatment. Using generalized estimating equations, for each item of the version 3 AOM-SOS scale, we examined whether the distribution of scores during the initial 7 days of treatment differed as between the 2 treatment groups (amoxicillin-clavulanate vs placebo) and as between the 2 outcome groups (clinical success vs clinical failure). Comparison of Results Using Version 3 and Version 4 of the AOM-SOS Scale
We combined findings of this report to consider whether any modification of version 3 appeared to show promise, and if so, how using that modified version in the aforementioned amoxicillin-clavulanate vs placebo trial [3] in place of version 3 might have affected the trial’s results. Namely, we compared the results using the 2 versions on the following 3 outcomes: (1) the proportion of children resolving, with resolution defined as reaching an AOM-SOS score of 0 or 1; (2) the proportion of children resolving with resolution, defined by 2 successive AOM-SOS scores of 0 or 1; and (3) mean AOM-SOS score over the first 7 days. For the analyses of time to resolution of symptoms, we used tests of equal hazard functions using a proportional hazards model, and for the mean score we used and generalized estimating equations. RESULTS Responsiveness of Individual AOM-SOS Items
The respective SRM values of each of the 7 version 3 AOM-SOS scale items, in decreasing order of magnitude,
were 0.79, 0.76, 0.74, 0.68, 0.62, 0.48, and 0.38 for fussiness, sleep disturbance, fever, crying, tugging, diminished activity, and diminished appetite. Longitudinal Construct Validity of Individual AOM-SOS Items
The distribution of scores during the initial 7 days of treatment differed significantly as between the 2 treatment groups (amoxicillin-clavulanate vs placebo) for difficulty sleeping (P = .03) and fever (P = .005); differences regarding ear tugging (P = .06) and irritability (P = .09) were borderline; and differences regarding crying (P = .24), diminished activity (P = .26) , and diminished appetite (P = .55) were not significant. The distribution of scores during the initial 7 days of treatment differed significantly as between the 2 outcome groups (clinical success vs clinical failure), irrespective of treatment group, for ear tugging (P = .005), crying (P = .04), irritability (P = .004), and difficulty sleeping (P = .04); the difference regarding diminished activity (P = .09) was borderline; differences regarding diminished appetite (P = .11) and fever (P = .15) were not significant. Combined Analyses Regarding the Version 3 AOM-SOS Scale, and Comparison With Results Using a Proposed Revised Scale (Version 4)
Table 1 summarizes the results of these analyses regarding individual symptoms, together with results of the aforementioned earlier analysis of parental opinions about those symptoms in relation to ear pain [4]. When the findings from the various analyses were taken as a whole, diminished activity and diminished appetite emerged as the symptoms that appeared weakest psychometrically, as indicated by the heavy-lined rectangle in the table. Given these findings, we deleted those 2 symptoms from version 3 of the AOM-SOS scale to form a revised, 5-item version 4. We then compared results applying version 4 to the data on symptomatic response in the aforementioned clinical trial with the results that had been obtained in the trial using version 3 as described previously [3]. For each of
Table 1. Summary of Psychometric Properties of Individual Version 3 AOM-SOS Scale Items* Responsive?
Valid in Differentiating Treatment Groups?
Valid in Differentiating Outcome Groups?
Indicative of Otalgia (according to parents) [4]?
Ear tugging Crying Irritability Difficulty sleeping
Yes Yes Yes Yes
Borderline No Borderline Yes
Yes Yes Yes Yes
Yes Yes Yes Yes
Diminished activity Diminished appetite
Borderline No
No No
Borderline No
Borderline Borderline
Fever
Yes
Yes
No
Yes
Abbreviation: AOM-SOS, Acute Otitis Media Severity of Symptoms. *Heavy-lined rectangle identifies the 2 items with the weakest psychometric properties.
Modification of the AOM-SOS Scale
these analyses, results applying version 4 closely paralleled results that had been obtained using version 3, the main difference being a lesser degree of variability in the results when using version 4, with a corresponding increase in the levels of confidence (lower P values) concerning the trial’s results. The P values corresponding to analyses 1, 2, and 3 described in the methods using the AOM-SOS version 4 were .06, .01, and .005, respectively. The corresponding values for version 3 were .14, .04, and .02. DISCUSSION In considering how the AOM-SOS scale might be further refined, we used a 3-pronged approach. First, in examining the responsiveness of each item in the scale, we attempted to identify items that exhibited little change in patients who were clearly improving in other important clinical respects. Such unresponsive items, even if they might seem important to parents or might be valid measures of AOM symptoms, would impede the ability of the scale as a whole to measure change. Second, because no reference standard exists for measuring improvement or worsening of symptoms, comparing symptoms in groups expected to differ from each other clinically constitutes one of the best methods of assessing the symptoms’ construct validity. Thus, we sought to determine, for the 7 symptoms of the version 3 AOM-SOS scale, the degree to which each showed differences over time as between the 2 treatment groups (amoxicillin-clavulanate vs placebo) and as between the 2 outcome groups (clinical success vs clinical failure). Third, we examined parental views regarding the importance of each scale item in regard to the parents’ estimates of their children’s ear pain. From the various analyses described above and summarized in Table 1, it appeared that eliminating the “diminished activity” and “diminished appetite” items from the 7-item version 3 of the AOM-SOS scale might improve the scale. What we found was that clinical trial results applying the reduced, 5-item version 4 were virtually indistinguishable from the results that had been obtained using version 3. That finding notwithstanding, the parsimony gained by moving from a 7-item to a 5-item scale, and the reduction in variability that we found in the clinical trial’s results when applying the 5-item scale, with a
corresponding increase in the levels of confidence concerning those results, suggest that change to the 5-item scale in subsequent clinical trials would be advantageous. However, before considering adoption of this change in the scale, confirmation of our findings will be required in other study populations, preferably in other locales, through analyses of the type described in the present report. Acknowledgments Potential conflicts of interest. The University of Pittsburgh owns the copyrights of the AOM-SOS scale. N. S., A. H., and J. L. P. receive remuneration if the scale is licensed to a for-profit entity. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.
References 1. Shaikh N, Hoberman A, Paradise JL, et al. Responsiveness and construct validity of a symptom scale for acute otitis media. Pediatr Infect Dis J 2009; 28:9–12. 2. Shaikh N, Hoberman A, Paradise JL, et al. Development and preliminary evaluation of a parent-reported outcome instrument for clinical trials in acute otitis media. Pediatr Infect Dis J 2009; 28: 5–8. 3. Hoberman A, Paradise JL, Rockette HE, et al. Treatment of acute otitis media in children under 2 years of age. N Engl J Med 2011; 364:105–15. 4. Shaikh N, Kearney DH, Colborn DK, et al. How do parents of preverbal children with acute otitis media determine how much ear pain their child is having? J Pain 2010; 11:1291–4. 5. Beaton DE, Bombardier C, Katz JN, Wright JG. A taxonomy for responsiveness. J Clin Epidemiol 2001; 54:1204–17. 6. Terwee CB, Dekker FW, Wiersinga WM, et al. On assessing responsiveness of health-related quality of life instruments: guidelines for instrument evaluation. Qual Life Res 2003; 12:349–62. 7. Husted JA, Cook RJ, Farewell VT, Gladman DD. Methods for assessing responsiveness: a critical review and recommendations. J Clin Epidemiol 2000; 53:459–68. 8. Le Saux N, Gaboury I, Baird M, et al. A randomized, doubleblind, placebo-controlled noninferiority trial of amoxicillin for clinically diagnosed acute otitis media in children 6 months to 5 years of age. Can Med Assoc J 2005; 172:335–41. 9. Little P, Gould C, Williamson I, et al. Pragmatic randomised controlled trial of two prescribing strategies for childhood acute otitis media. BMJ 2001; 322:336–42. 10. Sanders SL, Glasziou PP, Del Mar CB, Rovers MM. Antibiotics for acute otitis media in children. Cochrane database of systematic reviews (Online) 2004:CD000219. 11. Tähtinen PA, Laine MK, Huovinen P, et al. A placebo-controlled trial of antimicrobial treatment for acute otitis media. N Engl J Med 2011; 364:116–26.
3
