Home and Community Care Digest
Methods: Methodological flaws are identified as a flaw in study design, analysis, or reporting of results.
Findings: The common flaws in study design include (1) omitting important costs or benefits, and (2) selecting inappropriate alternatives for comparison. It is therefore important that the decision-maker ask whether, given the perspective of the analysis (e.g., health care system, societal, government), it is likely that omitted costs and/or benefits will have a significant impact on study results. It is also important to consider whether the selected comparator is the best option, or whether better alternatives are available.
Common flaws in data collection and analysis include (1) inadequate representation of the effectiveness data, (2) inappropriate extrapolation beyond the period observed in clinical studies, (3) excessive use of assumptions rather than data, and (4) inadequate characterization of uncertainty. Health care decisionmakers should question whether all available and relevant clinical studies are used in the costeffectiveness study, and, whether there is sufficient justification for the excluded studies. Decisionmakers should further question whether the costs and effects were costed over the appropriate length of time (i.e., is the time horizon of the study justified?). Lastly, with respect to data collection and analysis, decision-makers should note whether key assumptions used in the analysis were tested and supported by other studies.
Decision-makers should also be aware of flaws in the interpretation and reporting of results. Common flaws include (1) inappropriate aggregation of results, (2) reporting average cost-effectiveness ratios, (3) lack of consideration of generalizability issues (i.e., will the findings be useful to other jurisdictions or decision-makers?), and (4) selective reporting of findings. Decision-makers should note whether all component parts of the cost-effectiveness ratio are included, and whether comparisons between alternatives are expressed in incremental form (i.e., is the cost per additional hospital day reported?). Decision makers are also advised to beware of descriptive words like "substantial" and "minor" in the reporting of key results. It is important to question whether potentially negative results are downplayed while positive results are unduly emphasized.
Conclusions: Economic evaluations are increasingly utilized when making funding and reimbursement decisions. While it is important for health care decision-makers that make funding decisions to be aware of methodological flaws that may impact the decision outcome, the authors note the importance of recognizing that most decisions must be made with imperfect information, and that the value of an economic evaluation should therefore be determined by the extent to which it enhances the decisionmaking process. The guidance provided on how to detect and mitigate key study flaws will assist health care decision-makers make informed judgments about whether the economic evaluations presented to them are of sufficient quality to contribute to, and enhance, their decision-making process.
Reference: Drummond M, Sculpher M. "Common methodological flaws in economic evaluations." Medical Care, 2005; 43(7) suppl, II5-II14.
Be the first to comment on this!
Personal Subscriber? Sign In
Note: Please enter a display name. Your email address will not be publically displayed