Policies on assessing risk and quality

  1. Assessments of risk of bias and quality of the evidence are required for all new Cochrane breast cancer reviews

  2. Methods recommended for assessing the risk of bias in randomised controlled trials and non-randomised studies, and judging the quality of the evidence overall are:

    (i) Cochrane's Risk of Bias tool for randomised controlled trials (chapter 8 in the Cochrane Handbook for Systematic Reviews of Interventions) and Cochrane's Risk of Bias tool for non-randomised studies (ACROBAT-NRSI). The Risk of Bias table is automatically generated in Review Manager software as an extension of the characteristics of included studies table and enables judgements to be made for each of the 7 domains. Some tinkering is required with the Risk of Bias tables depending on the outcomes being assessed and also if the Review will include non-randomised studies

    (ii) GRADEpro GDT for assessing the quality of the evidence. The GRADE process brings together risk of bias assessments, indirectness (how well do the included studies address the review question?), imprecision (reflected in the confidence intervals), inconsistency (how consistent are the results of all of the studies?) and publication bias (is there a high probability of publication bias?). Based on these 5 factors, a judgement of the overall quality of evidence for the main outcomes can be made. GRADEpro GDT generates a summary of the quality of the evidence in the form of a Summary of Findings (SoF) table

  3. Authors are required to state in their protocol how risk of bias will be assessed and any action to be taken based on risk of bias. For example, will trials be excluded on the basis of their risk of bias? Will a sensitivity analysis be undertaken? A sensitivity analysis will involve analyses where studies of poorer quality are included and then excluded, and any differences discussed

  4. Given the nature of the interventions used in the management of breast cancer it will be neither possible, nor practical, to expect blinding of the intervention or outcome assessment in the majority of reviews

  5. Wherever possible, two people should independently assess each potentially eligible trial for risk of bias and outcomes for quality of the evidence

  6. A third person should be used to resolve any discrepancies