Investigation and Evaluation of Systematic Reviews of Prediction Models Published in Chinese Journals: Methodological and Reporting Quality

WANG Ziyi; LU Cuncun; HUANG Jiayi; ZHANG Jinglei; SHANG Wenru; CUI Lu; LIU Wendi; DENG Xiuxiu; ZHAO Xiaoxiao; YANG Kehu; LI Xiuxia

doi:10.12290/xhyxzz.2023-0418

Volume 15 Issue 4

Jul. 2024

Turn off MathJax

Article Contents

Abstract

References

Medical Journal of Peking Union Medical College Hospital > 2024 > 15(4): 927-935. > DOI: 10.12290/xhyxzz.2023-0418

WANG Ziyi, LU Cuncun, HUANG Jiayi, ZHANG Jinglei, SHANG Wenru, CUI Lu, LIU Wendi, DENG Xiuxiu, ZHAO Xiaoxiao, YANG Kehu, LI Xiuxia. Investigation and Evaluation of Systematic Reviews of Prediction Models Published in Chinese Journals: Methodological and Reporting Quality[J]. Medical Journal of Peking Union Medical College Hospital, 2024, 15(4): 927-935. DOI: 10.12290/xhyxzz.2023-0418

Citation:

PDF (3721 KB)

Investigation and Evaluation of Systematic Reviews of Prediction Models Published in Chinese Journals: Methodological and Reporting Quality

WANG Ziyi^{1, 2},
LU Cuncun^3, ,,
HUANG Jiayi^{1, 2},
ZHANG Jinglei⁴,
SHANG Wenru²,
CUI Lu^{1, 2},
LIU Wendi^{1, 2},
DENG Xiuxiu⁵,
ZHAO Xiaoxiao³,
YANG Kehu²,
LI Xiuxia^{1, 2, ,}

1.
Evidence-based Social Science Research Center/Health Technology Assessment Center, School of Public Health, Lanzhou University, Lanzhou 730000, China
2.
Evidence-based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, China
3.
Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing 100700, China
4.
School of Health Sciences, Wuhan University, Wuhan 430062, China
5.
Department of Gastroenterology, Chengdu Pidu District Hospital of Traditional Chinese Medicine, Chengdu 611730, China

Funds:

Fundamental Research Funds for the Central Universities lzujbky-2021-ct06

Fundamental Research Funds for the Central Universities lzujbky-2021-kb22

More Information

Corresponding author:
LU Cuncun, E-mail: cuncunlu2017@163.com

LI Xiuxia, E-mail: lixiuxia@lzu.edu.cn
Received Date: September 04, 2023
Accepted Date: October 17, 2023
Available Online: November 21, 2023
Publish Date: November 20, 2023
Issue Publish Date: July 29, 2024

Graphical Abstract

Abstract

Abstract

Objective
To analyze the methodological and reporting quality of systematic reviews of prediction models published in Chinese journals, with the aim of providing reference for enhancing the overall quality of Chinese systematic reviews of prediction models.
Methods
We searched the CNKI, WanFang Data, CBM, and VIP databases for Chinese systematic reviews of prediction models from inception to July 20, 2023. After two independent reviewers screened literature and extracted data, the AMSTAR(A Measurement Tool to Assess Systematic Reviews) and PRISMA 2020(Preferred Reporting Items for Systematic reviews and Meta-Analyses 2020) tools were used to assess the methodological and reporting quality of the included reviews.
Results
A total of 55 systematic reviews published between 2015 and 2023 were included, 12 of which were meta-analysis. The reviews covered various topics, mainly including cardiovascular diseases, stroke, and diabetes. The identified systematic reviews exhibited obvious deficiencies: items 1, 4, 5, 6, and 10 of AMSTAR showed poor methodological quality, and items 7, 10a, 12, 13a-f, 14, 15, 16a-b, 17, 20b-d, 21, 22, 23d, 24a-c, 25 and 26 of PRISMA 2020 needed improvement in reporting quality. Furthermore, a moderate positive correlation (r=0.58, P < 0.001) was observed between the methodological and reporting quality. Multiple linear regression analysis revealed that a greater number of pages, more recent publications, and funding support were associated with higher methodological quality (P < 0.05). Similarly, a greater number of pages, more recent publications, qualitative systematic reviews, and funding support were associated with higher reporting quality, but the number of authors showed a negative association (P < 0.05).
Conclusion
The methodological and reporting quality of existing systematic reviews of prediction models published in Chinese journals is relatively poor and demands improvement.
- prediction models,
- systematic reviews,
- reporting quality,
- methodological quality

FullText(HTML)

References (30)

References

[1]	Hendriksen J M T, Geersing G J, Moons K G M, et al. Diagnostic and prognostic prediction models[J]. J Thromb Haemost, 2013, 11(Suppl 1): 129-141.
[2]	Meehan A J, Lewis S J, Fazel S, et al. Clinical prediction models in psychiatry: a systematic review of two decades of progress and challenges[J]. Mol Psychiatry, 2022, 27(6): 2700-2708. DOI: 10.1038/s41380-022-01528-4
[3]	王昊玥, 王俊峰. 临床预测模型研究中的常见误区[J]. 中国卒中杂志, 2023, 18(7): 758-769. DOI: 10.3969/j.issn.1673-5765.2023.07.004 Wang H Y, Wang J F. Common misconception in clinical prediction model research[J]. Chin J Stroke, 2023, 18(7): 758-769. DOI: 10.3969/j.issn.1673-5765.2023.07.004
[4]	Hueting T A, Van Maaren M C, Hendriks M P, et al. The majority of 922 prediction models supporting breast cancer decision-making are at high risk of bias[J]. J Clin Epidemiol, 2022, 152: 238-247. DOI: 10.1016/j.jclinepi.2022.10.016
[5]	Kaiser I, Diehl K, Heppt M V, et al. Reporting quality of studies developing and validating melanoma prediction models: an assessment based on the TRIPOD statement[J]. Healthcare (Basel), 2022, 10(2): 238.
[6]	Murad M H, Montori V M, Ioannidis J P A, et al. How to read a systematic review and meta-analysis and apply the results to patient care: users' guides to the medical literature[J]. JAMA, 2014, 312(2): 171-179. DOI: 10.1001/jama.2014.5559
[7]	Hoffmann T C, Oxman A D, Ioannidis J P, et al. Enhancing the usability of systematic reviews by improving the consideration and description of interventions[J]. BMJ, 2017, 358: j2998.
[8]	Shea B J, Grimshaw J M, Wells G A, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews[J]. BMC Med Res Methodol, 2007, 7: 10. DOI: 10.1186/1471-2288-7-10
[9]	丁舒, 吴瑛, 陶香君, 等. 院内患者谵妄风险预测模型的系统评价[J]. 中华护理杂志, 2015, 50(5): 613-619. DOI: 10.3761/j.issn.0254-1769.2015.05.022 Ding S, Wu Y, Tao X J, et al. Systematic review of delirium risk prediction models in hospitalized patients[J]. Chin J Nurs, 2015, 50(5): 613-619. DOI: 10.3761/j.issn.0254-1769.2015.05.022
[10]	靳育静, 高鹰, 张卿. Gail模型在我国女性乳腺癌发病风险预测中的研究: 系统综述与Meta分析[J]. 现代肿瘤医学, 2022, 30(6): 1017-1023. DOI: 10.3969/j.issn.1672-4992.2022.06.013 Jin Y J, Gao Y, Zhang Q. Assessment of performance of the Gail model for predicting breast cancer risk in Chinese women: systematic review and Meta-analysis[J]. J Mod Oncol, 2022, 30(6): 1017-1023. DOI: 10.3969/j.issn.1672-4992.2022.06.013
[11]	梁诗雨, 李晨阳, 邵乐文. ICU后综合征风险预测模型的系统评价[J]. 中国护理管理, 2023, 23(3): 431-437. https://www.cnki.com.cn/Article/CJFDTOTAL-GLHL202303020.htm Liang S Y, Li C Y, Shao L W. Prediction models for Post-intensive Care Syndrome: a systematic review[J]. Chin Nurs Manag, 2023, 23(3): 431-437. https://www.cnki.com.cn/Article/CJFDTOTAL-GLHL202303020.htm
[12]	詹立睿, 张紫薇, 宋萍, 等. 2型糖尿病患者低血糖风险预测模型的系统评价[J]. 中华糖尿病杂志, 2023, 15(3): 244-251. DOI: 10.3760/cma.j.cn115791-20220720-00347 Zhan L R, Zhang Z W, Song P, et al. A systematic review of hypoglycemia risk prediction models in patients with type 2 diabetes mellitus[J]. Chin J Diabetes Mellitus, 2023, 15(3): 244-251. DOI: 10.3760/cma.j.cn115791-20220720-00347
[13]	Zeraatkar D, Bhasin A, Morassut R E, et al. Characteristics and quality of systematic reviews and meta-analyses of observational nutritional epidemiology: a cross-sectional study[J]. Am J Clin Nutr, 2021, 113(6): 1578-1592. DOI: 10.1093/ajcn/nqab002
[14]	Grammatopoulos T, Hunter J W S, Munn Z, et al. Reporting quality and risk of bias in JBI systematic reviews evaluating the effectiveness of interventions: a methodological review protocol[J]. JBI Evid Synth, 2023, 21(3): 584-591. DOI: 10.11124/JBIES-22-00317
[15]	Shea B J, Hamel C, Wells G A, et al. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews[J]. J Clin Epidemiol, 2009, 62(10): 1013-1020. DOI: 10.1016/j.jclinepi.2008.10.009
[16]	Page M J, McKenzie J E, Bossuyt P M, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews[J]. BMJ, 2021, 372: n71.
[17]	Page M J, Moher D, Bossuyt P M, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews[J]. BMJ, 2021, 372: n160.
[18]	徐俊峰, 安妮, 周为文, 等. 《中国循证医学杂志》发表的干预类系统评价/Meta分析方法学质量评价[J]. 中国循证医学杂志, 2013, 13(5): 605-611. https://www.cnki.com.cn/Article/CJFDTOTAL-ZZXZ201305023.htm Xu J F, An N, Zhou W W, et al. Methodological quality assessment of systematic reviews or Meta-analyses of intervention published in the Chinese Journal of Evidence-Based Medicine[J]. Chin J Evid-Based Med, 2013, 13(5): 605-611. https://www.cnki.com.cn/Article/CJFDTOTAL-ZZXZ201305023.htm
[19]	卢存存, 柯立鑫, 汤昊, 等. 中文期刊发表的氨甲环酸Meta分析的报告质量与方法学质量评价[J]. 中国循证医学杂志, 2021, 21(11): 1332-1338. https://www.cnki.com.cn/Article/CJFDTOTAL-ZZXZ202111013.htm Lu C C, Ke L X, Tang H, et al. Reporting and methodological quality of tranexamic acid Meta-analyses published in Chinese journals[J]. Chin J Evid-Based Med, 2021, 21(11): 1332-1338. https://www.cnki.com.cn/Article/CJFDTOTAL-ZZXZ202111013.htm
[20]	Li J Y, Tang L Y, Tang H, et al. Reporting quality of systematic review protocols of interventions for knee osteoarthritis: a systematic review protocol[J]. Med Data Min, 2021, 4(3): 12.
[21]	Schober P, Boer C, Schwarte L A. Correlation coefficients: appropriate use and interpretation[J]. Anesth Analg, 2018, 126(5): 1763-1768. DOI: 10.1213/ANE.0000000000002864
[22]	Villa M, Le Pera M, Cassina T, et al. Reporting quality of abstracts from randomised controlled trials published in leading critical care nursing journals: a methodological quality review[J]. BMJ Open, 2023, 13(3): e070639. DOI: 10.1136/bmjopen-2022-070639
[23]	Paez A. Gray literature: an important resource in systematic reviews[J]. J Evid Based Med, 2017, 10(3): 233-240. DOI: 10.1111/jebm.12266
[24]	Brozek J L, Canelo-Aybar C, Akl E A, et al. GRADE guidelines 30: the GRADE approach to assessing the certainty of modeled evidence—an overview in the context of health decision-making[J]. J Clin Epidemiol, 2021, 129: 138-150. DOI: 10.1016/j.jclinepi.2020.09.018
[25]	陈汐敏. 我国医学期刊论文发表偏倚的调查和思考[J]. 中国科技期刊研究, 2019, 30(7): 715-720. https://www.cnki.com.cn/Article/CJFDTOTAL-JYKQ201907004.htm Chen X M. Investigation and reflection on publication bias of medical journals in China[J]. Chin J Sci Tech Period, 2019, 30(7): 715-720. https://www.cnki.com.cn/Article/CJFDTOTAL-JYKQ201907004.htm
[26]	王佳琳, 闫佰荟, 王璐, 等. 中文肿瘤学干预类Meta分析论文摘要报告质量评价分析[J]. 中华医学图书情报杂志, 2020, 29(10): 24-31. DOI: 10.3969/j.issn.1671-3982.2020.10.004 Wang J L, Yan B H, Wang L, et al. Assessment and analysis of reporting quality of abstracts in Meta-analysis papers on intervention published in Chinese oncology journals[J]. Chin J Med Libr Inf Sci, 2020, 29(10): 24-31. DOI: 10.3969/j.issn.1671-3982.2020.10.004
[27]	王朝宏, 张卫国, 杜媛鲲, 等. 中华超声影像学杂志2010—2012年基金项目论文统计分析与评价[J]. 中华超声影像学杂志, 2013, 22(5): 458-459. DOI: 10.3760/cma.j.issn.1004-4477.2013.05.030 Wang C H, Zhang W G, Du Y K, et al. Statistical analysis and evaluation of funding project papers in the Chinese Journal of Ultrasonography from 2010 to 2012[J]. Chin J Ultrasonogr, 2013, 22(5): 458-459. DOI: 10.3760/cma.j.issn.1004-4477.2013.05.030
[28]	王国豪, 靳英辉, 张磊, 等. 国内中医护理系统评价/Meta分析的报告质量与方法学质量评价[J]. 护理学杂志, 2016, 31(1): 98-102. DOI: 10.3870/j.issn.1001-4152.2016.01.098 Wang G H, Jin Y H, Zhang L, et al. Reporting and methodological quality of systematic reviews and Meta-analyses in TCM nursing[J]. J Nurs Sci, 2016, 31(1): 98-102. DOI: 10.3870/j.issn.1001-4152.2016.01.098
[29]	王子怡, 卢存存, 张晶磊, 等. 个体预后与诊断多变量预测模型系统评价/Meta分析报告规范(TRIPOD-SRMA)解读[J]. 中国循证医学杂志, 2024, 24(2): 202-210. https://www.cnki.com.cn/Article/CJFDTOTAL-ZZXZ202402008.htm Wang Z Y, Lu C C, Zhang J L, et al. Interpretation of checklist for transparent reporting of multivariable prediction models for individual prognosis or diagnosis tailored for systematic reviews and meta-analyses(TRIPOD-SRMA)[J]. J Evid Based Med, 2024, 24(2): 202-210. https://www.cnki.com.cn/Article/CJFDTOTAL-ZZXZ202402008.htm
[30]	Shea B J, Reeves B C, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both[J]. BMJ, 2017, 358: j4008.

[1]	GAO Yicheng, LIU Zhihan, TAO Liyuan, CAO Rui, XIA Ruyu, FENG Yuting, FANG Rui, DENG Yingjie, XIANG Wenyuan, FEI Yutong. Considerations and Methodological Suggestions on Health Equity in Clinical Practice Guideline Development[J]. Medical Journal of Peking Union Medical College Hospital, 2023, 14(6): 1319-1324. DOI: 10.12290/xhyxzz.2023-0096
[2]	SU Renfeng, YU Xuan, SHI Qianling, LUO Xufei, SUN Yajia, LAN Hui, REN Mengjuan, WU Shouyuan, WANG Ping, WANG Ling, ZHAO Junxian, CHEN Yaolong. Current Situation and Progress of Evidence Synthesis Methodology[J]. Medical Journal of Peking Union Medical College Hospital, 2023, 14(6): 1301-1309. DOI: 10.12290/xhyxzz.2023-0062
[3]	LIU Hui, SU Renfeng, SHI Anya, YAO Yuanyuan, ZHOU Qi, WANG Zijun, TIE Rui, LU Mengying, ZHANG Hao, SUN Haonan, ZHAO Junxian, ZHAO Siya, YANG Kehu, LIU Xingrong, CHEN Yaolong. Methodological Quality Analysis of Systematic Review/Meta-analysis Published in Chinese Science Citation Database Journals[J]. Medical Journal of Peking Union Medical College Hospital, 2023, 14(2): 390-398. DOI: 10.12290/xhyxzz.2022-0317
[4]	SUN Yajia, SHI Qianling, YANG Nan, GUO Qiangqiang, SU Renfeng, CHEN Ze, ZHANG Guangxin, CAO Xichao, CHEN Yaolong. Rethinking and Exploring the Comprehensive Evaluation of Clinical Practice Guidelines[J]. Medical Journal of Peking Union Medical College Hospital, 2023, 14(1): 22-30. DOI: 10.12290/xhyxzz.2022-0612
[5]	CHEN Yaolong, SUN Yajia, LUO Xufei, YU Xuan. The Core Methods and Key Models in Evidence-based Medicine[J]. Medical Journal of Peking Union Medical College Hospital, 2023, 14(1): 1-8. DOI: 10.12290/xhyxzz.2022-0686
[6]	LI Zhenghong, QUAN Meiying. Quality Improvement Project in the Healthcare Field and Standards for Quality Improvement Reporting Excellence[J]. Medical Journal of Peking Union Medical College Hospital, 2022, 13(6): 1074-1080. DOI: 10.12290/xhyxzz.2022-0210
[7]	REN Mengjuan, YANG Nan, LIU Yunlan, SUN Yajia, SHI Qianling, ZHOU Qi, WANG Zijun, CHEN Yaolong. Investigation and Evaluation of Chinese Clinical Practice Guidelines Published in Medical Journals in 2019:Reporting Status of Abstract[J]. Medical Journal of Peking Union Medical College Hospital, 2022, 13(5): 880-887. DOI: 10.12290/xhyxzz.2022-0190
[8]	LIU Hui, LAN Hui, ZHAO Siya, WANG Zijun, SHI Qianling, LIU Xiao, ZHOU Qi, WANG Jianjian, LYU Meng, LIU Yunlan, YANG Nan, XUN Yangqin, LI Qinyuan, PEI Hang, LIU Xingrong, CHEN Yaolong. Investigation and Evaluation of Chinese Clinical Practice Guidelines Published in Medical Journals in 2019:Research Gaps[J]. Medical Journal of Peking Union Medical College Hospital, 2022, 13(3): 498-505. DOI: 10.12290/xhyxzz.2022-0029
[9]	LIU Yunlan, ZHANG Jingyi, SHI Qianling, YANG Nan, WANG Zijun, LUO Xufei, REN Mengjuan, XUN Yangqin, ZHOU Qi, LIU Hui, LYU Meng, CHEN Yaolong. Investigation and Evaluation of Chinese Clinical Practice Guidelines Published in Medical Journals in 2019:Methodological and Reporting Quality[J]. Medical Journal of Peking Union Medical College Hospital, 2022, 13(2): 324-331. DOI: 10.12290/xhyxzz.2022-0027
[10]	WANG Zi-jun, ZHOU Qi, XING Dan, YANG Nan, LUO Xu-fei, ZHANG Jing-yi, SHI Qian-ling, ZHAO Si-ya, LIU Hui, LIU Xiao, LI Qin-yuan, DU Liang, YANG Ke-hu, CHEN Yao-long. Reporting Quality of Literature Interpreting Clinical Practice Guidelines/Consensus: A Cross-sectional Study[J]. Medical Journal of Peking Union Medical College Hospital, 2021, 12(2): 260-267. DOI: 10.12290/xhyxzz.20200250

Cited By

Cited by

Periodical cited type(1)

孙雅佳，史乾灵，杨楠，郭强强，苏仁凤，陈泽，张广新，曹锡超，陈耀龙. 临床实践指南综合评价的思考与探索. 协和医学杂志. 2023(01): 22-30 .

本站查看

Other cited types(1)

Get Citation

PDF

XML

Article Metrics

Article views (301) PDF downloads (40) Cited by(2)

Address：	Room 302,Third Floor,Building 7, No. 1 Shuaifuyuan, Wangfujing, Dongcheng District, Beijing 100730, China
Tel：	010-69154261/4262
E - mail：	medj@pumch.cn mjpumch@126.com
Copyright of website：	Editorial Office of Medical Journal of Peking Union Medical College Hospital

Investigation and Evaluation of Systematic Reviews of Prediction Models Published in Chinese Journals: Methodological and Reporting Quality