| Citation: |
SHI Qianling, LIU Hui, WANG Zijun, LUO Xufei, WANG Bingyi, YANG Nan, MENG Wenbo, CHEN Yaolong. How to Correctly Understand and Use the Low-quality Evidence to Formulate Recommendations in Guidelines[J]. Medical Journal of Peking Union Medical College Hospital, 2024, 15(3): 676-685. DOI: 10.12290/xhyxzz.2024-0278
|
The essence of clinical practice guidelines lies in their recommendations. It is common to find strong recommendations supported by low-quality evidence in current published guidelines. There is a typical misunderstanding among medical professionals that without high-quality evidence, it is impossible to develop high-quality guidelines or only expert consensus can be developed. Based on the GRADE approach, this paper explains the concept and clinical significance of low-quality evidence, and introduces the methods for formulating recommendations based on low-quality evidence in guidelines, with the aim to provide reference for guideline developers and users in China.
| [1] |
Institute of Medicine. Clinical practice guidelines we can trust[M]. Washington, D.C.: The National Academies Press, 2011.
|
| [2] |
Mc Allister M, Florez I D, Stoker S, et al. Advancing guideline quality through country-wide and regional quality assessment of CPGs using AGREE: a scoping review[J]. BMC Med Res Methodol, 2023, 23(1): 283. DOI: 10.1186/s12874-023-02101-5
|
| [3] |
Zhou Q, Wang Z J, Shi Q L, et al. Clinical epidemiology in China series. Paper 4: the reporting and methodological quality of Chinese clinical practice guidelines published between 2014 and 2018: a systematic review[J]. J Clin Epidemiol, 2021, 140: 189-199. DOI: 10.1016/j.jclinepi.2021.08.013
|
| [4] |
Guyatt G H, Oxman A D, Vist G E, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations[J]. BMJ, 2008, 336(7650): 924-926. DOI: 10.1136/bmj.39489.470347.AD
|
| [5] |
周奇, 董冲亚, 王业明, 等. 循证医学理念在指导临床研究与实践中的作用: 基于对抗病毒药物治疗新型冠状病毒肺炎的思考[J]. 中国循证医学杂志, 2022, 22(4): 373-379. https://www.cnki.com.cn/Article/CJFDTOTAL-ZZXZ202204001.htm
Zhou Q, Dong C Y, Wang Y M, et al. The role of evidence-based medicine in guiding clinical research and practice: expert opinion based on studies on antiviral drugs for the treatment of COVID-19[J]. Chin J Evid-based Med, 2022, 22(4): 373-379. https://www.cnki.com.cn/Article/CJFDTOTAL-ZZXZ202204001.htm
|
| [6] |
Guyatt G, Oxman A D, Akl E A, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables[J]. J Clin Epidemiol, 2011, 64(4): 383-394. DOI: 10.1016/j.jclinepi.2010.04.026
|
| [7] |
Andrews J, Guyatt G, Oxman A D, et al. GRADE guidelines: 14. Going from evidence to recommendations: the significance and presentation of recommendations[J]. J Clin Epidemiol, 2013, 66(7): 719-725. DOI: 10.1016/j.jclinepi.2012.03.013
|
| [8] |
Guyatt G H, Oxman A D, Vist G, et al. GRADE guidelines: 4. Rating the quality of evidence--study limitations (risk of bias)[J]. J Clin Epidemiol, 2011, 64(4): 407-415. DOI: 10.1016/j.jclinepi.2010.07.017
|
| [9] |
Amstutz A, Speich B, Mentré F, et al. Effects of remdesivir in patients hospitalised with COVID-19: a systematic review and individual patient data meta-analysis of randomised controlled trials[J]. Lancet Respir Med, 2023, 11(5): 453-464. DOI: 10.1016/S2213-2600(22)00528-8
|
| [10] |
Beigel J H, Tomashek K M, Dodd L E, et al. Remdesivir for the treatment of covid-19 - final report[J]. N Engl J Med, 2020, 383(19): 1813-1826. DOI: 10.1056/NEJMoa2007764
|
| [11] |
Wang Y M, Zhang D Y, Du G H, et al. Remdesivir in adults with severe COVID-19: a randomised, double-blind, placebo-controlled, multicentre trial[J]. Lancet, 2020, 395(10236): 1569-1578. DOI: 10.1016/S0140-6736(20)31022-9
|
| [12] |
Jacobson R M, Pignolo R J, Lazaridis K N. Clinical trials for special populations: children, older adults, and rare diseases[J]. Mayo Clin Proc, 2024, 99(2): 318-335. DOI: 10.1016/j.mayocp.2023.03.003
|
| [13] |
Hanlon P, Hannigan L, Rodriguez-Perez J, et al. Representation of people with comorbidity and multimorbidity in clinical trials of novel drug therapies: an individual-level participant data analysis[J]. BMC Med, 2019, 17(1): 201. DOI: 10.1186/s12916-019-1427-1
|
| [14] |
陈耀龙, 杨克虎. 正确理解、制订和使用临床实践指南[J]. 协和医学杂志, 2018, 9(4): 367-373. DOI: 10.3969/j.issn.1674-9081.2018.04.015
Chen Y L, Yang K H. How to correctly understand, develop, and apply clinical practice guidelines[J]. Med J PUMCH, 2018, 9(4): 367-373. DOI: 10.3969/j.issn.1674-9081.2018.04.015
|
| [15] |
Liu E M, Smyth R L, Luo Z X, et al. Rapid advice guidelines for management of children with COVID-19[J]. Ann Transl Med, 2020, 8(10): 617. DOI: 10.21037/atm-20-3754
|
| [16] |
Lamontagne F, Fowler R A, Adhikari N K, et al. Evidence-based guidelines for supportive care of patients with Ebola virus disease[J]. Lancet, 2018, 391(10121): 700-708. DOI: 10.1016/S0140-6736(17)31795-6
|
| [17] |
Ou J Y, Li J Y, Liu Y, et al. Quality appraisal of clinical guidelines for Helicobacter pylori infection and systematic analysis of the level of evidence for recommendations[J]. PLoS One, 2024, 19(4): e0301006. DOI: 10.1371/journal.pone.0301006
|
| [18] |
Murayama A, Kamamoto S, Murata N, et al. Evaluation of financial conflicts of interest and quality of evidence in Japanese gastroenterology clinical practice guidelines[J]. J Gastroenterol Hepatol, 2023, 38(4): 565-573. DOI: 10.1111/jgh.16089
|
| [19] |
Weissman S, Goldowsky A, Aziz M, et al. Colorectal cancer screening guidelines are primarily based on low-moderate-quality evidence[J]. Dig Dis Sci, 2021, 66(12): 4208-4219. DOI: 10.1007/s10620-020-06755-9
|
| [20] |
Chong M C, Sharp M K, Smith S M, et al. Strong recommendations from low certainty evidence: a cross-sectional analysis of a suite of national guidelines[J]. BMC Med Res Methodol, 2023, 23(1): 68. DOI: 10.1186/s12874-023-01895-8
|
| [21] |
Alexander P E, Bero L, Montori V M, et al. World Health Organization recommendations are often strong based on low confidence in effect estimates[J]. J Clin Epidemiol, 2014, 67(6): 629-634. DOI: 10.1016/j.jclinepi.2013.09.020
|
| [22] |
Zhang Z H, Hong Y C, Liu N. Scientific evidence underlying the recommendations of critical care clinical practice guidelines: a lack of high level evidence[J]. Intensive Care Med, 2018, 44(7): 1189-1191. DOI: 10.1007/s00134-018-5142-8
|
| [23] |
周奇, 王玲, 杨楠, 等. 基于GRADE康复临床实践指南证据质量与推荐强度研究[J]. 中国康复理论与实践, 2020, 26(2): 156-160. https://www.cnki.com.cn/Article/CJFDTOTAL-ZKLS202002006.htm
Zhou Q, Wang L, Yang N, et al. Quality of evidence and strength of recommendations of clinical practice guidelines of rehabilitation based on GRADE[J]. Chin J Rehabil Theory Pract, 2020, 26(2): 156-160. https://www.cnki.com.cn/Article/CJFDTOTAL-ZKLS202002006.htm
|
| [24] |
Miles K E, Rodriguez R, Gross A E, et al. Strength of recommendation and quality of evidence for recommendations in current Infectious Diseases Society of America guidelines[J]. Open Forum Infect Dis, 2021, 8(2): ofab033. DOI: 10.1093/ofid/ofab033
|
| [25] |
Mamada H, Murayama A, Kamamoto S, et al. Evaluation of financial and nonfinancial conflicts of interest and quality of evidence underlying psoriatic arthritis clinical practice guidelines: analysis of personal payments from pharmaceutical companies and authors' self-citation rate in Japan and the United States[J]. Arthritis Care Res (Hoboken), 2023, 75(6): 1278-1286. DOI: 10.1002/acr.25032
|
| [26] |
Fleming P S, Koletsi D, Ioannidis J P A, et al. High quality of the evidence for medical and other health-related interventions was uncommon in Cochrane systematic reviews[J]. J Clin Epidemiol, 2016, 78: 34-42. DOI: 10.1016/j.jclinepi.2016.03.012
|
| [27] |
Howick J, Koletsi D, Ioannidis J P A, et al. Most healthcare interventions tested in Cochrane reviews are not effective according to high quality evidence: a systematic review and meta-analysis[J]. J Clin Epidemiol, 2022, 148: 160-169. DOI: 10.1016/j.jclinepi.2022.04.017
|
| [28] |
Conway A, Conway Z, Soalheira K, et al. High quality of evidence is uncommon in Cochrane systematic reviews in anaesthesia, critical care and emergency medicine[J]. Eur J Anaesthesiol, 2017, 34(12): 808-813. DOI: 10.1097/EJA.0000000000000691
|
| [29] |
Xun Y Q, Guo Q Q, Ren M J, et al. Characteristics of the sources, evaluation, and grading of the certainty of evidence in systematic reviews in public health: a methodological study[J]. Front Public Health, 2023, 11: 998588. DOI: 10.3389/fpubh.2023.998588
|
| [30] |
Kane R L, Butler M, Ng W. Examining the quality of evidence to support the effectiveness of interventions: an analysis of systematic reviews[J]. BMJ Open, 2016, 6(5): e011051. DOI: 10.1136/bmjopen-2016-011051
|
| [31] |
Shi H S, Deng P, Dong C D, et al. Quality of evidence supporting the role of tripterygium glycosides for the treatment of diabetic kidney disease: an overview of systematic reviews and meta-analyses[J]. Drug Des Devel Ther, 2022, 16: 1647-1665. DOI: 10.2147/DDDT.S367624
|
| [32] |
Pandis N, Fleming P S, Worthington H, et al. The quality of the evidence according to GRADE is predominantly low or very low in oral health systematic reviews[J]. PLoS One, 2015, 10(7): e0131644. DOI: 10.1371/journal.pone.0131644
|
| [33] |
Peng L F, Fan M X, Li J H, et al. Evidence quality assessment of sodium tanshinone IIA sulfonate injection intervention coronary heart disease angina pectoris: an overview of systematic reviews and meta-analyses[J]. Medicine (Baltimore), 2023, 102(44): e35509. DOI: 10.1097/MD.0000000000035509
|
| [34] |
Zhang J, Yu Q Y, Peng L, et al. Benefits and safety of Chinese herbal medicine in treating psoriasis: an overview of systematic reviews[J]. Front Pharmacol, 2021, 12: 680172. DOI: 10.3389/fphar.2021.680172
|
| [35] |
Jin Y H, Wang G H, Sun Y R, et al. A critical appraisal of the methodology and quality of evidence of systematic reviews and meta-analyses of traditional Chinese medical nursing interventions: a systematic review of reviews[J]. BMJ Open, 2016, 6(11): e011514. DOI: 10.1136/bmjopen-2016-011514
|
| [36] |
Kong Y K, Wei X Q, Duan L, et al. Rating the quality of evidence: the GRADE system in systematic reviews/meta-analyses of AKI[J]. Ren Fail, 2015, 37(7): 1089-1093.
|
| [37] |
Fu G J, Fan X M, Liang X, et al. An overview of systematic reviews of Chinese herbal medicine in the treatment of migraines[J]. Front Pharmacol, 2022, 13: 924994. DOI: 10.3389/fphar.2022.924994
|
| [38] |
Wang L, Du X Y, Hu P, et al. Quality of evidence supporting the role of acupuncture for breast cancer-related lymphoedema: an overview of systematic reviews and meta-analyses[J]. J Cancer Res Clin Oncol, 2023, 149(18): 16669-16678. DOI: 10.1007/s00432-023-05419-1
|
| [39] |
Brouwers M C, Kho M E, Browman G P, et al. AGREE Ⅱ: advancing guideline development, reporting and evaluation in health care[J]. CMAJ, 2010, 182(18): E839-E842. DOI: 10.1503/cmaj.090449
|
| [40] |
Chen Y L, Yang K H, Maru ic A, et al. A reporting tool for practice guidelines in health care: the RIGHT statement[J]. Ann Intern Med, 2017, 166(2): 128-132. DOI: 10.7326/M16-1565
|
| [41] |
Yang N, Liu H, Zhao W, et al. Development of the scientific, transparent and applicable rankings (STAR) tool for clinical practice guidelines[J]. Chin Med J (Engl), 2023, 136(12): 1430-1438. DOI: 10.1097/CM9.0000000000002713
|
| [42] |
中华医学会杂志社指南与标准研究中心, 中国医学科学院循证评价与指南研究创新单元(2021RU017), 世界卫生组织指南实施与知识转化合作中心, 等. 2022年医学期刊发表中国指南和共识的科学性、透明性和适用性的评级[J]. 中华医学杂志, 2023, 103(37): 2912-2920. DOI: 10.3760/cma.j.cn112137-20230724-00076
Guidelines and Standards Research Center Chinese Medical Association Publishing House, Research Unit of Evidence-Based Evaluation and Guidelines (2021RU017), Chinese Academy of Medical Sciences, WHO Collaborating Centre for Guideline Implementation and Knowledge Translation, et al. Evaluation and ranking for scientific, transparent and applicable of Chinese guidelines and consensus published in the medical journals in 2022[J]. Natl Med J China, 2023, 103(37): 2912-2920. DOI: 10.3760/cma.j.cn112137-20230724-00076
|
| [43] |
Barker T H, Dias M, Stern C, et al. Guidelines rarely used GRADE and applied methods inconsistently: a methodo-logical study of Australian guidelines[J]. J Clin Epidemiol, 2021, 130: 125-134. DOI: 10.1016/j.jclinepi.2020.10.017
|
| [44] |
World Health Organization. WHO handbook for guideline development[M]. 2nd ed. Geneva: WHO Press, 2014.
|
| [45] |
National Institute for Health and Care Excellence. Developing NICE guidelines: the manual[M/OL ]. London: National Institute for Health and Care Excellence, 2024(2024-01-17)[2024-04-02]. https://www.nice.org.uk/process/pmg20/resources/developing-nice-guidelines-the-manual-pdf-72286708700869.
|
| [46] |
Guidelines International Network, McMaster University. GIN-McMaster guideline development checklist[M/OL]. [2024-04-02 ]. https://macgrade.mcmaster.ca/resources/gin-mcmaster-guideline-development-checklist/.
|
| [47] |
Scottish Intercollegiate Guidelines Network (SIGN). A guideline developer's handbook[M/OL]. [2024-04-02]. https://www.sign.ac.uk.
|
| [48] |
陈耀龙, 杨克虎, 王小钦, 等. 中国制订/修订临床诊疗指南的指导原则(2022版)[J]. 中华医学杂志, 2022, 102(10): 697-703. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGUD202210051.htm
Chen Y L, Yang K H, Wang X Q, et al. Guiding principles for developing and updating Chinese diagnosis and treatment guidelines(version 2022)[J]. Natl Med J China, 2022, 102(10): 697-703. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGUD202210051.htm
|
| [49] |
Alonso-Coello P, Schünemann H J, Moberg J, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction[J]. BMJ, 2016, 353: i2016.
|
| [50] |
Guyatt G H, Oxman A D, Kunz R, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes[J]. J Clin Epidemiol, 2011, 64(4): 395-400. DOI: 10.1016/j.jclinepi.2010.09.012
|
| [51] |
Guyatt G, Oxman A D, Sultan S, et al. GRADE guidelines: 11. Making an overall rating of confidence in effect estimates for a single outcome and for all outcomes[J]. J Clin Epidemiol, 2013, 66(2): 151-157. DOI: 10.1016/j.jclinepi.2012.01.006
|
| [52] |
Guyatt G H, Oxman A D, Kunz R, et al. What is "quality of evidence" and why is it important to clinicians?[J]. BMJ, 2008, 336(7651): 995-998. DOI: 10.1136/bmj.39490.551019.BE
|
| [53] |
Karanicolas P J, Davies E, Kunz R, et al. The pylorus: take it or leave it? Systematic review and meta-analysis of pylorus-preserving versus standard whipple pancreaticoduodenectomy for pancreatic or periampullary cancer[J]. Ann Surg Oncol, 2007, 14(6): 1825-1834. DOI: 10.1245/s10434-006-9330-3
|
| [54] |
Guyatt G H, Oxman A D, Schünemann H J, et al. GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology[J]. J Clin Epidemiol, 2011, 64(4): 380-382. DOI: 10.1016/j.jclinepi.2010.09.011
|
| [55] |
杨克虎, 陈耀龙, 孙凤, 等. GRADE在系统评价和实践指南中的应用[M]. 第2版. 北京: 中国协和医科大学出版社, 2021: 3.
Yang K H, Chen Y L, Sun F, et al. The application of GRADE in systematic reviews and practice guidelines[M]. 2nd ed. Beijing: Peking Union Medical College Press, 2021: 3.
|
| [56] |
Andrews J C, Schünemann H J, Oxman A D, et al. GRADE guidelines: 15. Going from evidence to recommendation-determinants of a recommendation's direction and strength[J]. J Clin Epidemiol, 2013, 66(7): 726-735. DOI: 10.1016/j.jclinepi.2013.02.003
|
| [57] |
Bautista-Orduno K G, Dorsey-Trevino E G, Gonzalez-Gonzalez J G, et al. American Thyroid Association guidelines are inconsistent with grading of recommendations assess-ment, development, and evaluations-a meta-epidemiologic study[J]. J Clin Epidemiol, 2020, 123: 180-188. e2. DOI: 10.1016/j.jclinepi.2020.02.010
|
| [58] |
谢方瑜, 孙秀杰, 芦秀燕, 等. 低质量证据强推荐意见在中国护理实践指南中的现状分析[J]. 护理学报, 2021, 28(20): 39-43. https://www.cnki.com.cn/Article/CJFDTOTAL-NFHL202120007.htm
Xie F Y, Sun X J, Lu X Y, et al. Current status of recommendation of low-quality evidence in Chinese nursing practice guidelines[J]. J Nurs, 2021, 28(20): 39-43. https://www.cnki.com.cn/Article/CJFDTOTAL-NFHL202120007.htm
|
| [59] |
Alexander P E, Brito J P, Neumann I, et al. World Health Organization strong recommendations based on low-quality evidence (study quality) are frequent and often inconsistent with GRADE guidance[J]. J Clin Epidemiol, 2016, 72: 98-106. DOI: 10.1016/j.jclinepi.2014.10.011
|
| [60] |
Yao L, Ahmed M M, Guyatt G H, et al. Discordant and inappropriate discordant recommendations in consensus and evidence based guidelines: empirical analysis[J]. BMJ, 2021, 375: e066045.
|
| [61] |
Yao L, Guyatt G H, Djulbegovic B. Can we trust strong recommendations based on low quality evidence?[J]. BMJ, 2021, 375: n2833.
|
| [62] |
Meneses-Echavez J F, Bidonde J, Montesinos-Guevara C, et al. Using evidence to decision frameworks led to guidelines of better quality and more credible and transparent recommendations[J]. J Clin Epidemiol, 2023, 162: 38-46. DOI: 10.1016/j.jclinepi.2023.07.013
|
| [63] |
Lund H, Brunnhuber K, Juhl C, et al. Towards evidence based research[J]. BMJ, 2016, 355: i5440.
|
| [64] |
刘辉, 兰慧, 赵思雅, 等. 2019年期刊公开发表的中国临床实践指南文献调查与评价: 研究空白[J]. 协和医学杂志, 2022, 13(3): 498-505. https://www.cnki.com.cn/Article/CJFDTOTAL-XHYX202203022.htm
Liu H, Lan H, Zhao S Y, et al. Investigation and evaluation of Chinese clinical practice guidelines published in medical journals in 2019: research gaps[J]. Med J PUMCH, 2022, 13(3): 498-505. https://www.cnki.com.cn/Article/CJFDTOTAL-XHYX202203022.htm
|
| [65] |
WHO. WHO guideline for complementary feeding of infants and young children 6-23 months of age[M]. Geneva: World Health Organization, 2023.
|
Supported by: Beijing Renhe Information Technology Co., Ltd. 
Submit
Review
Office
Email Alert
RSS
DownLoad: