Uncertainty in clinical prediction rules: the value of credible intervals.

[

clinical commentary

]

Journal of Orthopaedic & Sports Physical Therapy® Downloaded from www.jospt.org at University of Pittsburgh on June 2, 2014. For personal use only. No other uses without permission. Copyright © 2014 Journal of Orthopaedic & Sports Physical Therapy®. All rights reserved.

ROBIN HASKINS, BPhty (Hons)1 • PETER G. OSMOTHERLY, PhD1 • FRANK TUYL, PhD2 • DARREN A. RIVETT, PhD1

Uncertainty in Clinical Prediction Rules: The Value of Credible Intervals

D

ecision making in health care is increasingly informed by the incorporation of knowledge considered within a probabilistic framework.23,29 New information derived from the history, physical examination, and other investigations is used to revise prior beliefs about the likelihood of a given diagnosis or outcome by a magnitude proportional to the relative strength of that information.10,14 The application of such methods, particularly within a diagnostic context, has been termed probabilistic reasoning.11,31 Within a probabilistic framework, perfect predictions are not anticipated and error is knowingly accepted.12 The goal of probabilistic reasoning is therefore not to predict outcomes with certainty but, rather, to generate predictions that are more often “less TTSYNOPSIS: Decision making in physical therapy is increasingly informed by evidence in the form of probabilities. Prior beliefs concerning diagnoses, prognoses, and treatment effects are quantitatively revised by the integration of new information derived from the history, physical examination, and other investigations in a well-recognized application of Bayes’ theorem. Clinical prediction rule development studies commonly employ such methodology to produce quantified estimates of the likelihood of patients having certain diagnoses or achieving given outcomes. To date, the physical therapy literature has been limited to the discussion and calculation of the point estimate of such probabilities. The degree of precision associated with the construction of posterior probabilities, which requires consideration of both uncertainty associated with pretest probability and uncertainty associated with test accuracy, remains largely unrecognized and unreported. This paper provides an introduction to the calculation of the uncer-

wrong” than those generated by other methods. Prior beliefs (prevalence of a diagnosis or outcome), also known as “pretest probability,” and new information (outcome of test with known sensitivity and specificity) may be mathematically integrated to produce the quantified probability of a tainty interval, known as a credible interval, around posterior probability estimates. The method for calculating the credible interval is detailed and illustrated with example data from 2 clinical prediction rule development studies. Two relatively quick and simple methods for approximating the credible interval are also outlined. It is anticipated that knowledge of the credible interval will have practical implications for the incorporation of probabilistic evidence in clinical practice. Consistent with reporting standards for interventional and diagnostic studies, it is equally appropriate that studies reporting posterior probabilities calculate and report the level of precision associated with these point estimates. J Orthop Sports Phys Ther 2014;44(2):85-91. Epub 30 October 2013. doi:10.2519/jospt.2014.4877

TTKEY WORDS: Bayes’ theorem, clinical

prediction rules, decision making, probability

given diagnosis or outcome (known as a “posterior probability” or “posttest probability”) using a well-known application of Bayes’ theorem.2

(1) P (Diagnosis or Outcome) = Prevalence ≈ Sensitivity Prevalence ≈ Sensitivity + (1 – Prevalence) ≈ (1 – Specificity) Clinical prediction rules (CPRs) are a common application of probabilistic reasoning in health care and function to produce estimates of the likelihood of a target diagnosis, prognosis, or treatment outcome that, in turn, inform clinical decision making.16,25 For example, Hicks et al17 derived a CPR that functions to identify patients experiencing low back pain with a higher likelihood of achieving success from an 8-week stabilization-exercise program. They found that when 3 or more factors were present at baseline, including a positive prone instability test, aberrant movements, average straight leg raise greater than 91°, and age greater than 40 years, the probability of achieving a successful treatment outcome increased from 33% to 67%. A second example from the physical therapy literature is a CPR designed to identify patients with low back pain more likely to benefit from lumbopelvic thrust manipulation. Flynn et al13 identified that the probability of success from this intervention increased from 45% to 95% when 4 or more factors were present, including

School of Health Sciences, The University of Newcastle, Callaghan, Australia. 2School of Mathematical and Physical Sciences, The University of Newcastle, Callaghan, Australia. The authors certify that they have no affiliations with or financial involvement in any organization or entity with a direct financial interest in the subject matter or materials discussed in the article. Address correspondence to Robin Haskins, School of Health Sciences, The Hunter Building, The University of Newcastle, Callaghan, NSW 2308 Australia. E-mail: [email protected] t Copyright ©2014 Journal of Orthopaedic & Sports Physical Therapy® 1

journal of orthopaedic & sports physical therapy | volume 44 | number 2 | february 2014 | 85

44-02 Haskins.indd 85

1/17/2014 5:07:28 PM

[

clinical commentary

Uncertainty in Clinical Prediction: Credible Intervals To date, the physical therapy literature has been limited to the discussion and calculation of single values (point estimates) of posterior probabilities. When uncertainty is considered, it is frequently limited to test accuracy (sensitivity, specificity, likelihood ratio), with rare consideration of the precision of the pretest probability. The degree of precision associated with the construction of the posterior probability that is frequently reported in many CPR development studies remains largely unrecognized and unreported. This stands in contrast to the near-uniform reporting of confidence intervals around estimates of treatment effect and diagnostic test accuracy seen in the modern scientific literature.7,32 While it is accepted that confidence intervals reported in interventional studies have important implications for clinical practice,34 no such considerations have been given to the need for uncertainty intervals for posterior probability estimates published in the physical therapy CPR literature. The precision of the posterior probability estimate requires consideration of the uncertainty associated with prevalence, as well as the uncertainty associated with test accuracy.1,4 The resultant uncertainty interval within a Bayesian framework is known as a credible interval (CrI) and gives the range in which the “true” likelihood of a given diagnosis or outcome lies for a specified probability level.26 Conceptually, it is the probability of a probability. For instance, the range of values expressed in a 95% CrI has a 95% chance of containing the true probability of a given outcome based on the information available. FIGURE 1 illustrates the relationship

100%

+LR = 50 +LR = 10

80%

Posttest Probability


duration of symptoms less than 16 days, at least 1 hip with more than 35° of internal rotation, lumbar hypomobility, no symptoms below the knee, and a score of less than 19 on the work subscale of the Fear-Avoidance Beliefs Questionnaire. 37

]

+LR = 2

60%

40%

–LR = 0.5

–LR = 0.1

20%

–LR = 0.02 0% 0%

20%

40%

60%

80%

100%

Pretest Probability

FIGURE 1. The relationship between pretest probability and posttest probability for specified likelihood ratios. The slope of each curve illustrates the degree to which variation in the pretest probability influences the posttest probability. Abbreviation: LR, likelihood ratio.

between pretest probability and posttest probability for 6 likelihood ratio values. The slope of each curve and, therefore, the influence of a 1% change in pretest probability (x-axis) on posttest probability (y-axis) is defined by the following formula: (2) Slope = Likelihood Ratio [Pretest Probability ≈ (Likelihood Ratio – 1) + 1]2 Where the slope of this illustrated relationship is steep, small variations in the pretest probability have a large influence on the posttest probability. Conversely, where the slope is relatively flat, consideration of uncertainty of the pretest probability will only have a relatively small impact on the posterior probability uncertainty interval.4 The degree to which uncertainty in the likelihood ratio impacts the posterior probability is dependent on the magnitude of the pretest probability. Therefore, incorporation of uncertainty of both the pretest probabil-

ity and the likelihood ratio is required to calculate the degree of uncertainty of the posterior probability.

Calculating the CrI An appropriate method to calculate the CrI for a posterior probability estimate of a binary outcome is the objective Bayesian method using Monte Carlo simulation. This method takes into consideration uncertainty related to both pretest probability and test accuracy. Mossman and Berger26 identified this approach as having superior performance properties compared to other methods of uncertainty-interval calculation. Detailed below is a step-by-step guide as to how clinicians and researchers can perform this calculation using Microsoft Excel 2010 (Microsoft Corporation, Redmond, WA) and the statistical freeware R (http://www.r-project.org/). Each calculation step is illustrated using 2 separate examples from the physical therapy CPR literature. Of note, we have substituted the use of the Jeffreys prior used in Mossman and Berger’s26 original method with

86 | february 2014 | volume 44 | number 2 | journal of orthopaedic & sports physical therapy


1/17/2014 5:07:29 PM

TABLE 1

Two-by-Two Table to Obtain Prevalence, Sensitivity, and 1 – Specificity* Outcome Present

Outcome Not Present

Test positive

A

B

Test negative

C

D


*Prevalence = (A + C)/(A + B + C + D). Sensitivity = A/(A + C). 1 – specificity = B/(B + D).

TABLE 2

Two-by-Two Contingency Table Derived From the Data by Hicks et al 17 Treatment Successful (50% or Greater Improvement*)

CPR positive (3 or more variables present†) CPR negative (fewer than 3 variables present)

Treatment Not Successful (Less Than 50% Improvement)

10

5

8

31

Abbreviation: CPR, clinical prediction rule. *Percentage change in baseline and 8-week Oswestry Low Back Pain Disability Questionnaire score. † Age less than 40 years, average straight leg raise greater than 91°, aberrant movement present, positive prone instability test.

TABLE 3

Two-by-Two Contingency Table Derived From the Data by Flynn et al 13 Treatment Successful (Greater Than 50% Improvement*)

Treatment Not Successful (50% or Less Improvement)

CPR positive (4 or more variables present†)

20

1

CPR negative (fewer than 4 variables present)

12

38

Abbreviation: CPR, clinical prediction rule. *Percentage change in Oswestry Low Back Pain Disability Questionnaire score over 3 sessions. † Symptom duration less than 16 days, at least 1 hip with greater than 35° of internal rotation, hypomobility with lumbar spring testing, no symptoms distal to the knee, Fear-Avoidance Beliefs Questionnaire work subscale score less than 19.

the uniform Bayes–Laplace prior, due to its additional ability to perform well in instances where there are no false positives.36 There are essentially 6 steps. Step 1 Identify the numerator (x) and denominator (n) for prevalence (P0), sensitivity (P1), and 1 – specificity (P2). In many circumstances, it may be appropriate to source prevalence and test accuracy data from separate studies. In the case of a single study, these data may often be derived from a standard 2-by-2 contingency table (TABLE 1). If a contingency table has not been provided in the published paper, it can sometimes be calculated and constructed by the reader, as in the 2 examples below.

• Example 1. Based on the published results of Hicks et al17 for determining the likelihood of success for a stabilization exercise program for low back pain for when 3 or more variables were present at baseline, a 2-by-2 contingency table may be derived (TABLE 2). The following calculations may be derived from this table: P0 = x0/n0 = 18/54 (33%) P1 = x1/n1 = 10/18 (56%) P2 = x2/n2 = 5/36 (14%) • Example 2. Using the results of Flynn et al13 for identifying the likelihood of treatment success from lumbopelvic

manipulation for low back pain for when 4 or more variables are present, a 2-by-2 contingency table may be derived (TABLE 3) that enables the following calculations of prevalence (P0), sensitivity (P1), and 1 – specificity (P2): P0 = x0/n0 = 32/71 (45%) P1 = x1/n1 = 20/32 (63%) P2 = x2/n2 = 1/39 (3%) Step 2 The beta distribution is a type

of continuous probability distribution that is employed in Bayesian analysis. The shape parameters a and b that will be used to construct beta distributions (β[a, b]) for prevalence, sensitivity, and 1 – specificity are calculated using the following: β(a, b) = β(xi + 1, ni – xi +1). • Example 1. Continuing our example for stabilization exercises for low back pain: Prevalence: β(a, b) = β(18 + 1, 54 – 18 + 1) = β(19, 37) Sensitivity: β(a, b) = β(10 + 1, 18 – 10 + 1) = β(11, 9) 1 – specificity: β(a, b) = β(5 + 1, 36 – 5 + 1) = β(6, 32) • Example 2. Using the data for lumbopelvic thrust manipulation for low back pain: Prevalence: β(a, b) = β(32 + 1, 71 – 32 + 1) = β(33, 40) Sensitivity: β(a, b) = β(20 + 1, 32 – 20 + 1) = β(21, 13) 1 – specificity: β(a, b) = β(1 + 1, 39 – 1 + 1) = β(2, 39) Step 3 Use a random number genera-

tor to select a large number of values (N) (Mossman and Berger26 suggest 10 000) from each beta distribution. This can be performed using the statistical freeware R, with the following command to facilitate simple importing into Microsoft Excel (replace a and b with the relevant shape parameters calculated in step 2): write.csv(rbeta(10000, a, b)). Step 4 Combine each series of randomly



1/17/2014 5:07:31 PM

[

clinical commentary

]

drawn values for prevalence (P0), sensitivity (P1), and 1 – specificity (P2) using the following formula:


(3)

P0 ≈ P1 P0 ≈ P1 + (1 – P0) ≈ P2

Note that using the point estimates of P0, P1, and P2 in the above will provide the point estimate of the posterior probability. • Example 1. For the stabilization-exercise example: 0.33 ≈ 0.56 = 67% 0.33 ≈ 0.56 + (1 – 0.33) ≈ 0.14 • Example 2: for the lumbopelvic thrust manipulation example: 0.45 ≈ 0.63 = 95% 0.45 ≈ 0.63 + (1 – 0.45) ≈ 0.03 Step 5 Next, sort the resultant array of

values in ascending order. Step 6 To determine the lower and upper boundaries of a 2-tailed 95% CrI, find the values corresponding to N × 0.025 and N × 0.975, respectively, in the sorted array. • Example 1. For the stabilization-exercise example, the 95% CrI is 41% to 85%. • Example 2. For the lumbopelvic thrust manipulation example, the 95% CrI is 77% to 99%. Thus, for the stabilization-exercise example data on the likelihood of success of this intervention for low back pain, the corresponding 2-tailed 95% CrI is 41% to 85%. This means that, given the data, we can be 95% certain that the true probability of success from this exercise program for a patient presenting with 3 or more of the identified baseline predictors is between 41% and 85%. For the lumbopelvic thrust manipulation CPR example, the 95% CrI corresponding to positive status on this rule is 77% to 99%. For those more experienced in using R, steps 3 to 6 of the method outlined above can be performed entirely within R using the syntax commands listed below. Computationally, this method is very

FIGURE 2. R syntax commands and output for the calculation of the posterior probability 95% credible interval using the objective Bayesian method with Monte Carlo simulation. (A) Stabilization-exercise CPR example data. (B) Lumbopelvic thrust manipulation CPR example data. Abbreviation: CPR, clinical prediction rule.

quick and permits the use of an extremely large value for N (eg, 1 million) to produce even more precise estimates of the posterior uncertainty interval, as the calculation remains in vector form and does not require exporting to a spreadsheet. N = 1 000 000 P0 = rbeta(N, x0 + 1, n0 – x0 + 1) P1 = rbeta(N, x1 + 1, n1 – x1 + 1) P2 = rbeta(N, x2 + 1, n2 – x2 + 1) Post = P0 × P1/(P0 × P1 + (1 – P0) × P2) Quantile(Post, c(0.025, 0.975)) FIGURE 2 illustrates the appropriate syntax commands and output for this method in R using the data from the CPR examples provided above.

Quick and Simple Approximations In situations where it may be appropriate to gather prevalence, sensitivity, and specificity data from within a single study (this excludes case-control designs), 2 methods may provide quick and relatively simple approximations of the uncertainty interval of the posterior probability that may be sufficiently accurate for clinical purposes in the majority of instances. Tuyl35 proposed that adequate intervals may be constructed by using the beta(true positive + 1, false positive + 1) distribution. To calculate the 95% interval using this method, the following formulae may be used in Microsoft Excel:

Lower boundary: =beta.inv(0.025, true positive + 1, false positive + 1) Upper boundary: =beta.inv(0.975, true positive + 1, false positive + 1) A useful adjunct to this method is that the mode of this distribution will correspond to the point estimate of the posterior probability. The intervals generated by this method for both the stabilization-exercise example data and the lumbopelvic thrust manipulation example data are almost identical (less than 1% different) to those generated by the objective Bayesian method using Monte Carlo simulation. In the instance of zero false positives, a 1-sided interval appears preferable, such that the point estimate (100%) will be included within the uncertainty interval. To construct a 1-tailed 95% CrI in Microsoft Excel in the case of zero false positives, the following formulae may be applied: Lower boundary: =beta.inv(0.05, True Positive + 1, 1) Upper boundary: =1 An alternative method is to calculate the binomial proportion confidence interval of the positive predictive value: [True Positives/(True Positives + False Positives)]. The following command in the statistical freeware R will produce the relevant 95% interval using the ClopperPearson method9:



1/17/2014 5:07:32 PM


binom.test(True Positives, True Positives + False Positives) This method produces uncertainty intervals that tend to be more conservative than the aforementioned methods28 but may nevertheless be adequate for clinical decision-making purposes. For the stabilization-exercise example data, this method provides an interval of 38% to 88%, and for the thrust manipulation example data, an interval of 76% to 100%.

A Brief Discussion on Selected Alternative Methods It may seem intuitive to many to calculate the boundaries of the uncertainty interval of the posterior probability by combining the lower boundary of the pretest probability confidence interval with the lower boundary of the likelihood ratio confidence interval, and repeating this procedure for the upper boundaries. This method, however, produces uncertainty intervals that are generally much more conservative than the aforementioned procedures. For the stabilization-exercise example data, this method produces an interval of 30% to 90%, and for the lumbopelvic thrust manipulation example data, an interval of 63% to 100%. Given the computational steps involved in completing this method, it is also unlikely to be considered simpler than the approximation methods previously outlined. As already discussed, in some instances, uncertainty in the pretest probability will only have a very small influence on the uncertainty of the posttest probability. In such cases, simply combining the point estimate of the pretest probability with the upper and lower boundaries of the likelihood ratio confidence interval will produce an uncertainty interval similar to that produced by the methods already outlined. For the stabilizationexercise example data, this method produces an interval of 45% to 83%, and for the lumbopelvic thrust manipulation example data, an interval of 74% to 99%. This method, however, may only give a suitable approximation when small

variations in the pretest probability do not greatly influence the posttest probability across any portion of the pretest probability uncertainty interval. The influence of the pretest probability on the postprobability at a given likelihood ratio value may be calculated at its upper and lower uncertainty intervals using formula 2 presented above. Alternatively, it can be approximated by considering the slope of the corresponding parts of the likelihood ratio curves presented in FIGURE 1. Given the calculations required to check the appropriateness of this approximation method, it will be quicker and simpler in most cases to use the methods previously recommended.

Future Directions The additional knowledge of the CrI surrounding a posterior probability point estimate has important implications concerning the appropriate clinical application of probabilistic evidence into practice. Rather than sole reliance on the point estimate, consideration of its precision, as highlighted by the corresponding CrI, provides substantially greater depth of information, which will assist decision making. Analogous to the use of confidence intervals reported for treatment effect sizes,34 the CrI of a posterior probability may be used to inform clinical decision making by considering its position relative to a “threshold level of certainty”14 required by a clinician to make a decision, such as commencing a particular treatment program or confirming or negating a diagnostic hypothesis. Such thresholds are dependent on the relative risks and benefits associated with that decision. In the context of treatment decision making, a clinician may have a relatively low threshold of certainty required for a treatment that is low cost and low risk, and may have a higher threshold of certainty required for a treatment that is associated with a higher risk of an adverse outcome and is more expensive and time consuming. In the stabilization-exercise CPR example provided, it is plausible that the

lower boundary of the uncertainty interval (41%) may be below a “treatment threshold”14 many clinicians and patients would consider sufficient for an 8-week treatment program that requires 16 supervised sessions and daily home exercises. Within a probabilistic framework, further confirmatory evidence from the history and physical examination would be required for clinicians and patients to have confidence in the benefit that may be achieved from this intervention. To their credit, Hicks et al17 explicitly stated that the results of their study about the likelihood of success from a program of stabilization exercises represent the preliminary step in the development of a CPR, and they have not recommended that the derived tool be implemented in clinical practice. We have used the data from their derivation study to illustrate how clinical decisions informed by posterior probabilities may be influenced by the additional knowledge of the CrI. In our second CPR example, concerning the probability of treatment success from lumbopelvic thrust manipulation, the published results of Flynn et al13 enable the calculation of a CrI of 77% to 99%. In contrast to the stabilizationexercise program example, it is plausible that many clinicians and patients may perceive that the lower boundary of this uncertainty interval (77%) is above a suitable treatment threshold14 for this intervention, given its relatively low cost, short time frame, and low risk of serious adverse events.33 The primary method of calculating the CrI outlined in this paper (objective Bayesian method using Monte Carlo simulation) enables the incorporation of data from more than 1 study. Given that several independent studies are required in the development of a CPR,25 meta-analysis of such data may be helpful in the construction of more precise estimates of the posterior probability interval. For example, a recent systematic review5 of a CPR designed to help identify patients at risk of falling used a meta-analysis of the included studies to calculate more



1/17/2014 5:07:34 PM


[

clinical commentary

precise estimates of the tool’s sensitivity and specificity. These data were then used to help calculate more precise estimates of the posttest probability of a fall in patients who were either positive or negative on that CPR. To the best of our knowledge, published guidelines specific to the meta-analysis of CPR studies do not yet exist; however, many considerations concerning the appropriateness of pooling CPR accuracy data may be plausibly generalized from guidelines on the meta-analysis of diagnostic tests. Such considerations include the methodological rigor of included studies, as well as between-study variations in study design, study participants, CPR application, and the assessment of the reference standard.6,18,19,24,30

4.

5.

6.

7.

8.

CONCLUSION 9.

T

his commentary focuses on the precision of posterior probability estimates in CPR development studies. This is a necessary, but not sufficient, consideration in the application of such evidence into clinical practice. Other factors beyond the scope of this commentary, including study design, methodological quality, validation, and impact analysis, require equally due consideration and have been well described in the existing physical therapy literature.3,8,15,20-22,27 Consistent with reporting standards for interventional32 and diagnostic7 studies, we believe that it is equally appropriate that studies reporting posterior probabilities, as commonly practiced in CPR development studies, calculate and report the level of precision associated with these point estimates. t

10.

11. 12.

13.

14.

15.

16.

REFERENCES 1. B aron JA. Uncertainty in Bayes. Med Decis Making. 1994;14:46-51. 2. Bayes TR. An essay towards solving a problem in the doctrine of chances. Phil Trans. 1763;53:370-418. http://dx.doi.org/10.1098/ rstl.1763.0053 3. Beattie P, Nelson R. Clinical prediction rules:

17.

what are they and what do they tell us? Aust J Physiother. 2006;52:157-163. Bianchi MT, Alexander BM, Cash SS. Incorporating uncertainty into medical decision making: an approach to unexpected test results. Med Decis Making. 2009;29:116-124. http://dx.doi. org/10.1177/0272989X08323620 Billington J, Fahey T, Galvin R. Diagnostic accuracy of the STRATIFY clinical prediction rule for falls: a systematic review and meta-analysis. BMC Fam Pract. 2012;13:76. http://dx.doi. org/10.1186/1471-2296-13-76 Bossuyt PM, Leeflang MM. Chapter 6: developing criteria for including studies. In: Deeks JJ, Bossuyt PM, Gatsonis C, eds. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy. Oxford, UK: The Cochrane Collaboration; 2008. Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. BMJ. 2003;326:41-44. Childs JD, Cleland JA. Development and application of clinical prediction rules to improve decision making in physical therapist practice. Phys Ther. 2006;86:122-131. Clopper CJ, Pearson ES. The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika. 1934;26:404-413. http:// dx.doi.org/10.1093/biomet/26.4.404 Davidson M. The interpretation of diagnostic tests: a primer for physiotherapists. Aust J Physiother. 2002;48:227-232. Doust J. Diagnosis in general practice. Using probabilistic reasoning. BMJ. 2009;339:b3823. Einhorn HJ. Accepting error to make less error. J Pers Assess. 1986;50:387-395. http://dx.doi. org/10.1207/s15327752jpa5003_8 Flynn T, Fritz J, Whitman J, et al. A clinical prediction rule for classifying patients with low back pain who demonstrate short-term improvement with spinal manipulation. Spine (Phila Pa 1976). 2002;27:2835-2843. http://dx.doi. org/10.1097/01.BRS.0000035681.33747.8D Fritz JM, Wainner RS. Examining diagnostic tests: an evidence-based perspective. Phys Ther. 2001;81:1546-1564. Hancock M, Herbert RD, Maher CG. A guide to interpretation of studies investigating subgroups of responders to physical therapy interventions. Phys Ther. 2009;89:698-704. http:// dx.doi.org/10.2522/ptj.20080351 Haskins R, Rivett DA, Osmotherly PG. Clinical prediction rules in the physiotherapy management of low back pain: a systematic review. Man Ther. 2012;17:9-21. http://dx.doi.org/10.1016/j. math.2011.05.001 Hicks GE, Fritz JM, Delitto A, McGill SM. Preliminary development of a clinical prediction rule for determining which patients with low back pain will respond to a stabilization exercise program. Arch Phys Med Rehabil. 2005;86:1753-1762. http://dx.doi.org/10.1016/j. apmr.2005.03.033

] 18. Irwig L, Tosteson AN, Gatsonis C, et al. Guidelines for meta-analyses evaluating diagnostic tests. Ann Intern Med. 1994;120:667-676. http://dx.doi.org/10.7326/0003-4819-120-8199404150-00008 19. Jones CM, Athanasiou T. Diagnostic accuracy meta-analysis: review of an important tool in radiological research and decision making. Br J Radiol. 2009;82:441-446. http://dx.doi. org/10.1259/bjr/28171084 20. Kamper SJ, Maher CG, Hancock MJ, Koes BW, Croft PR, Hay E. Treatment-based subgroups of low back pain: a guide to appraisal of research studies and a summary of current evidence. Best Pract Res Clin Rheumatol. 2010;24:181-191. http://dx.doi.org/10.1016/j. berh.2009.11.003 21. Kent P, Hancock M, Petersen DH, Mjøsund HL. Clinimetrics corner: choosing appropriate study designs for particular questions about treatment subgroups. J Man Manip Ther. 2010;18:147-152. http://dx.doi.org/10.1179/1066 98110X12640740712419 22. Kent P, Keating JL, Leboeuf-Yde C. Research methods for subgrouping low back pain. BMC Med Res Methodol. 2010;10:62. http://dx.doi. org/10.1186/1471-2288-10-62 23. Ledley RS, Lusted LB. Reasoning foundations of medical diagnosis; symbolic logic, probability, and value theory aid our understanding of how physicians reason. Science. 1959;130:9-21. 24. Macaskill P, Gatsonis C, Deeks JJ, Harbord RM, Takwoingi Y. Chapter 10: analysing and presenting results. In: Deeks JJ, Bossuyt PM, Gatsonis C, eds. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy. Oxford, UK: The Cochrane Collaboration; 2010. 25. McGinn T, Wyer P, Wisnivesky J, et al. Advanced topics in diagnosis: clinical prediction rules. In: Guyatt G, Rennie D, Meade MO, Cook DJ, eds. Users’ Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. 2nd ed. New York, NY: McGraw-Hill Medical; 2008:491-505. 26. Mossman D, Berger JO. Intervals for posttest probabilities: a comparison of 5 methods. Med Decis Making. 2001;21:498-507. 27. Nee RJ, Coppieters MW. Interpreting research on clinical prediction rules for physiotherapy treatments. Man Ther. 2011;16:105-108. http:// dx.doi.org/10.1016/j.math.2010.10.005 28. Newcombe RG. Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med. 1998;17:857-872. http://dx.doi.org/10.1002/ (SICI)1097-0258(19980430)17:83.0.CO;2-E 29. O’Connor GT, Sox HC, Jr. Bayesian reasoning in medicine: the contributions of Lee B. Lusted, MD. Med Decis Making. 1991;11:107-111. 30. Reitsma JB, Rutjes AWS, Whiting P, Vlassov VV, Leeflang MMG, Deeks JJ. Chapter 9: assessing methodological quality. In: Deeks JJ, Bossuyt PM, Gatsonis C, eds. Cochrane Handbook for



1/17/2014 5:07:34 PM


Systematic Reviews of Diagnostic Test Accuracy. Oxford, UK: The Cochrane Collaboration; 2009. 31. Richardson WS, Wilson MC. Chapter 14: the process of diagnosis. In: Guyatt G, Rennie D, Meade MO, Cook DJ, eds. Users’ Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. 2nd ed. New York, NY: McGraw-Hill Medical; 2008:399-406. 32. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials. Ann Intern Med. 2010;152:726-732. http://dx.doi. org/10.7326/0003-4819-152-11-201006010-

00232 33. S hekelle PG, Adams AH, Chassin MR, Hurwitz EL, Brook RH. Spinal manipulation for low-back pain. Ann Intern Med. 1992;117:590-598. 34. Stratford PW. The added value of confidence intervals. Phys Ther. 2010;90:333-335. http:// dx.doi.org/10.2522/ptj.2010.90.3.333 35. Tuyl F. Diagnostic testing: a simple approximate interval for the positive predictive value. Proceedings of the Third Annual ASEARC Research Conference. Newcastle, Australia: Applied Statistics Education and Research Collaboration; 2009. 36. Tuyl F, Gerlach R, Mengersen K. A com-

parison of Bayes–Laplace, Jeffreys, and other priors: the case of zero events. Am Statistician. 2008;62:40-44. http://dx.doi. org/10.1198/000313008X267839 37. Waddell G, Newton M, Henderson I, Somerville D, Main CJ. A Fear-Avoidance Beliefs Questionnaire (FABQ) and the role of fear-avoidance beliefs in chronic low back pain and disability. Pain. 1993;52:157-168.

@

MORE INFORMATION WWW.JOSPT.ORG

GO GREEN By Opting Out of the Print Journal JOSPT subscribers and APTA members of the Orthopaedic and Sports Physical Therapy Sections can help the environment by “opting out” of receiving JOSPT in print each month as follows. If you are: • A JOSPT subscriber: Email your request to [email protected] or call the JOSPT office toll-free at 1-877-766-3450 and provide your name and subscriber number. • APTA Orthopaedic or Sports Section member: Go to http://www.apta.org/, log in, and select My Profile. Next click on Email Management/GoGreen. Toward the bottom of the list, you will find the Publications options and may opt out of receiving the print JOSPT. Please save this preference. Subscribers and members alike will continue to have access to JOSPT online and can retrieve current and archived issues anytime and anywhere you have Internet access.



1/17/2014 5:07:35 PM

Healthcare provider perceptions of clinical prediction rules.

Deriving clinical prediction rules from stroke outcome research.

Why do authors derive new cardiovascular clinical prediction rules in the presence of existing rules? A mixed methods study.

Clinical prediction rules in practice: review of clinical guidelines and survey of GPs.

A systematic review of studies comparing diagnostic clinical prediction rules with clinical judgment.

Assessment of Reliable Change Using 95% Credible Intervals for the Differences in Proportions: A Statistical Analysis for Case-Study Methodology.

Diagnosing malignant melanoma in ambulatory care: a systematic review of clinical prediction rules.

Kidney cancer: Quilling uncertainty in the prediction of RCC outcomes.

Design Characteristics Influence Performance of Clinical Prediction Rules in Validation: A Meta-Epidemiological Study.

Methods for calculating confidence and credible intervals for the residual between-study variance in random effects meta-regression models.

The vulnerability of rules in complex work environments: dynamism and uncertainty pose problems for cognition.

Clinical prediction rules in community-acquired pneumonia: lies, damn lies and statistics.

Pediatric Emergency Care Applied Research Network head injury clinical prediction rules are reliable in practice.

Translation of clinical prediction rules for febrile children to primary care practice: an observational cohort study.

Diagnostic clinical prediction rules for specific subtypes of low back pain: a systematic review.

Health practitioners' perceptions of adopting clinical prediction rules in the management of musculoskeletal pain: a qualitative study in Australia.

Uncertainty during pain anticipation: the adaptive value of preparatory processes.

The uncertainty principle in clinical data.

Atypical Brain Mechanisms of Prediction According to Uncertainty in Autism.

Improving Personalized Clinical Risk Prediction Based on Causality-Based Association Rules.

Physiotherapists' knowledge, attitudes and practices regarding clinical prediction rules for low back pain.

Potential Clinical Value of Multiparametric PET in the Prediction of Alzheimer's Disease Progression.

The Practical Application of Clinical Prediction Rules: A Commentary Using Case Examples in Surgical Patients with Degenerative Cervical Myelopathy.

Plea for routinely presenting prediction intervals in meta-analysis.