Developing the RIGHT-COI&F extension for the reporting conflicts of interest and funding in practice guidelines: study protocol
Introduction
Practice guidelines are intended to inform diagnosis and treatment decisions, narrow practice gaps, improve the quality of health care, and possibly reduce medical expenses (1-6). Interests, both financial and non-financial (related to, e.g., academic career progress or social status) may affect the objectivity and independence of those involved in guideline development, thus constituting conflicts of interest (COI) (1,7,8). COI and the involvement of funders may cause bias during the development process of guidelines and subsequently affect their trustworthiness (1,9,10).
COIs are commonly encountered in guideline development (11,12), and may in some cases negatively affect the trustworthiness of the guidelines. In 2009, five experts who associated with pharmaceutical companies participated in developing the guidelines for the management of influenza A (H1N1) of the World Health Organization (WHO) (13), but none of their COIs had been properly declared and managed in advance. In 2020, WHO retracts opioid guidelines after accepting that industry had an influence due to undue influence of opioid manufacturers in guideline development process (14). Above two examples on negative consequences of the lack of transparency or incomplete disclosure of COI on guidelines have led to the credibility of the WHO and the trust in the global public health system. Studies of COI in guideline development have identified deficiencies in different aspects, such as COI disclosure, and across diseases and fields, including oncology, acute ischemic stroke, endocrinology, and heart disease (15-21). Meanwhile, the reporting rate of COIs and funding in both Chinese and international guidelines has been shown to be low and the level of detail in the disclosure varies greatly (22-26). Because inappropriate handling of COI and funding may undermine guidelines’ trustworthiness (1,11,27-29), guideline developers have strived to address this issue.
Guideline-producing organizations such as WHO and the American College of Physicians have developed processes for the declaration and management of COI and require the sources of funding to be clearly reported (30-32). Such processes, if fully implemented, can optimize the credibility of the guidelines (33,34). Although handbooks for guideline development typically provide general guidance for declaring and managing the funding and COI, they commonly lack specific guidance. Adequately reporting of the basic information of COIs and funding helps the readers to accurately comprehend the nature and extent of the role of COI and funding in the guidelines, thus enhancing the credibility of the guidelines.
To improve the completeness and quality of guideline reporting, the Reporting Items of Practice Guidelines in Healthcare (RIGHT) Working Group developed the reporting standard for healthcare practice guidelines in 2017 (35). To date, investigators have developed or are developing multiple extensions to the RIGHT statement, such as for guideline adaptations (36), patients/public version of guidelines (37), traditional Chinese medicine (TCM) guidelines (38), and acupuncture guidelines (39). Such reporting checklists and extensions enable guideline developers to better report their guidelines and guideline users to access the key information easily and quickly. Although the RIGHT checklist (includes 22 items) has two items (items 18 and 19) related to COIs and funding (35), further refinement of the standard is needed (e.g., describe how users of the guideline obtain the COIs related statement or describe the effect of funding on the recommendations of this guideline, etc.). We decided to develop an extension to the RIGHT statement to clarify areas of COIs and funding in guidelines with the support of the RIGHT Working Group. This extension aims to standardize the reporting of COIs and funding to improve guideline development and uptake.
Objective
The aim of this project is to develop the essential reporting items for reporting COIs and funding in practice guidelines (RIGHT-COI&F) as an extension of the RIGHT Statement, to meet the requirements of complete and transparent reporting of these critical issues.
Methods
We will refer to and adapt the toolkit for developing a reporting guideline recommend by the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network (40) (https://www.equator-network.org/) and to the methods used in the RIGHT statement and extensions to develop our RIGHT-COI&F (36-39). We plan to complete the project in approximately 2 years (i.e., by July 2023) in the following six stages: (I) identify the need for the extension; (II) register the project and establish working groups; (III) collect the initial items; (IV) reach consensus on the items to be included; (V) revision and formulation of the final checklist; and (VI) dissemination and implementation. The detailed development process is illustrated in Figure 1, and the proposed schedule is shown in Figure 2. Because this project will not involve collection of primary data from humans or animals, ethical approval and patient consent are not required.
Identify the need for the checklist
To identify the need for developing this checklist, we searched the published literature and documents for COI and funding, and found only a few specific checklists that are not widely accepted internationally. We will identify the handbooks and manuals of guideline development that cover the principles of COI and funding as well as survey on the reporting of COI or funding: the results of these investigations will serve as the main source of the initial pool of items.
Obtain funding
This project will be funded by The Fundamental Research Funds for the Central Universities, Lanzhou University (lzujbky-2021-ey13). The funder will have no role in the study design, data collection and analysis, writing of the article, or the decision to submit it for publication.
Drafting the protocol and registering the project
To enhance the transparency and quality of the RIGHT-COI&F development, we drafted a project protocol and registered it on the EQUATOR network (https://www.equator-network.org/library/reporting-guidelines-under-development/reporting-guidelines-under-development-for-other-study-designs/#RIGHT-COI).
Establishing working groups
RIGHT-COI&F working groups will be composed of experts from different disciplines (statisticians, methodologists, journal editors, and professional medical writers) and multi-country (Switzerland, Canada, UK, Lebanon, Japan, USA, Korea, Spain, China, etc.), and with different fields of expertise (relevant experience in clinical guideline development and COI and/or in systematic reviews/guidelines research methodology). We will also invite members of the RIGHT working group and representatives of patients and the public to join the following working groups.
Advisory group
The advisory group will consist of 3–5 skilled experts who have rich experience in the development of clinical practice guidelines and reporting checklists. The group will review and provide expert advice during the different steps of the RIGHT-COI&F development process. The advisory group will approve the final checklist and accompanying guidance.
Coordination team
The coordination team will lead and coordinate the development process of RIGHT-COI&F and ensure its completion according to the established timeline. The Coordination Team is responsible for forming the initial pool of items for the reporting checklist, responding to the suggestions and opinions of experts in the consensus conference and revising the list. In addition, they will collect feedback, support the other groups, and consult with external parties throughout the development process.
Delphi panel
The Delphi panel will be composed of 20–30 international experts representing a broad range of disciplines. We will invite experts specializing in the management of COIs, as well as members of the Guidelines International Network (GIN), the RIGHT working group, the Appraisal of Guidelines for Research and Evaluation (AGREE) cooperative organization, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group, and other relevant organizations and guideline stakeholders. The Delphi panel will participate in Delphi surveys and/or consensus conferences, vote and agree on items during the checklist formation process, and approve the final version of the RIGHT-COI&F checklist.
Conduct systematic surveys
The coordination team will conduct systematic surveys, including a systematic survey of the reporting of COI and funding in guidelines, and a review of handbooks or manuals of guideline development for recommended policies related to COI and funding. The literature screening and data abstraction will be done by two independent investigators, and discrepancies will be solved by discussion. The coordination team will also conduct a survey among guideline stakeholders for their preferences regarding the reporting of COI and funding in guidelines.
Generation of the initial checklist
The coordination team will formulate an initial pool of items for RIGHT-COI&F based on the systematic surveys during regular online or face-to-face meetings. The coordination team will record and review the results of the meetings and draft a report that includes recommendations on the initial version of the reporting checklist. The coordination team will discuss each of the items to refine them if necessary, and generate the first version of the checklist based on the feedback to be evaluated in the Delphi process. We will use Microsoft Excel 2016 software to collect and manage all data and items.
Statistical analysis
To achieve consensus on which items from the list developed in step 6 should be included in the final reporting tool, we will conduct either two or three rounds of a modified Delphi survey, using a 7-point Likert scale for expressing agreement with each potential reporting item (35,41-43). We will use SurveyMonkey (https://www.surveymonkey.com/) to collect and summarize the results. We will determine the agreement with each item according to the median score that the item receives among the panelists (Table 1). Items that achieve ‘agreement’ are removed from subsequent rounds and included in the final version; items with ‘disagreement’ are removed from subsequent rounds and excluded from the final version; and items rated as ‘ambivalent’ or where no consensus is reached will be modified to reflect points raised by the Delphi panel and included in the next Delphi round.
Table 1
Score | Meaning | Median score | Definition of consensus |
---|---|---|---|
1 | Strongly disagree | Median score of 1–3 points without substantial comments | Disagreement with the item; exclude without further evaluation |
2 | Disagree | ||
3 | Somewhat disagree | ||
4 | Neither agree nor disagree (neutral) | Median score of 4–5 points, or ≤3 or ≥6 with substantial comments requesting a major revision of the item | Ambivalence about an item; further evaluation to either retain, modify or exclude the item |
5 | Somewhat agree | ||
6 | Agree | Median score of 6–7 points without substantial comments | Agreement with the item; retain without further evaluation |
7 | Strongly agree |
Modified Delphi process
During the first round of Delphi surveys, the Delphi panel will have the opportunity to propose items not included in the initial list. In each round, we will include a free text box to suggest changes to the checklist or provide comments. In order to minimize potential bias, the answers will be analyzed anonymously by a medical statistician who is not a member of the Delphi panel. After two or three rounds of Delphi surveys, we will generate a list of reporting items of COI and funding in guidelines, which will be discussed in the teleconference consensus meeting described next.
Consensus meeting
We will organize a consensus meeting as a teleconference. Representatives of the coordination team and Delphi panel will be invited to participate and present and discuss the results of the Delphi process to refine the list of items. All suggestions and comments will be recorded and archived by the coordination team, and feedback will be provided via email to form the first draft of the reporting checklist.
External review
When a draft of the list in step 8 is completed, it will be externally reviewed by individuals with extensive experience and expertise in the development of reporting guidelines and who have not participated in the Delphi survey or the consensus meeting. The reviewers will be invited to comment on the usability, integrity and formulation of items. We will refine the checklist accordingly.
Formulating the draft checklist
Based on the results of the Delphi surveys, consensus meeting and external review, the coordination team will help the advisory group to draft the final checklist and send it to the Delphi panel for their review for accuracy and correctness.
Pilot test and examination of validity
The checklist produced in step 10 will be applied to report the COI and funding of three ongoing guidelines. The developers of these guidelines will be invited to examine the checklist. We will design a questionnaire for this survey and solicit the opinions of these guideline developers to improve the checklist. Two investigators will also use the checklist to assess the reporting status on COIs and funding in a sample of 10 guidelines published in the past 2 years.
Development of the guidance statement and publication strategy (finalized RIGHT-COI&F checklist)
We will draft a statement based on the pilot test, and submit it to a peer-reviewed journal for publication.
Development of an explanatory document
The Coordination Team will develop a detailed explanatory and elaborative document for the basic reporting items to inform and guide users and facilitate the implementation of the checklist. The Delphi panel will be invited to review the document and provide suggestions.
Encourage endorsement and adherence
We will disseminate the statement as follows: (I) we will submit the reports and the checklist for endorsement by EQUATOR, the RIGHT website (http://www.right-statement.org/), and other relevant publicly accessible websites; (II) we will present the results at international academic conferences, such as the GIN Annual Meeting, and the Cochrane Colloquium; (III) we will distribute the statement to interested guideline developers and users worldwide; (IV) we will conduct lectures and training for guideline developers and users on how to use the RIGHT-COI&F checklist.
Translating and adapting the checklist
We welcome any initiatives to translate or adapt the checklist by and in collaboration with guideline developers throughout the world to suit their country-specific settings of use.
Updating the checklist
We will review the reporting checklist every 3 years and revise it as needed, including feedback from users of the checklist and new information from scientific publications.
Patient and public involvement
We will not directly involve patients in our study, but will analyze the survey from patient and public viewpoints and preferences to inform potential projects. They will provide their comments on the reporting items.
Discussion
The main output of this project is an extension of the RIGHT checklist for reporting COI and funding, as well as an explanatory document. To ensure the quality and smooth progress of this project, we plan to record each step of the process in detail and post it on the RIGHT website. We will disseminate the RIGHT-COI&F list by publishing it in peer-reviewed journals, introducing it to relevant stakeholders and translating it into different languages. Meanwhile, we will continue to seek feedback from guideline developers, users and stakeholders, and update the checklist according to the latest research evidence and feedback.
Whether a guideline is reported in a standardized manner or not is one of the key factors that determine whether it can be promoted efficiently and implemented smoothly (44). Since the publication of the RIGHT statement in 2017, guideline developers have increasingly used it to increase the transparency and clarity of their guidelines (36-39).
It is important to note that the original RIGHT statement includes two items (four sub-items) related to funding and COI: (I) declaration of specific funding sources for each phase of guideline development; (II) reporting the role of funders in guideline development, dissemination, and implementation; (III) declaring the types of COI associated with the guideline (financial and non-financial); and (IV) reporting how the COIs were evaluated and managed and how users of the guideline can access the declarations of interest. These items, however, do not address details such as how COI and funding should be reported, who should manage them, how they should be managed, and how the results of COI and funding are related to the final recommendations of the guideline. We will address these issues with the RIGHT-COI&F extension, a more detailed reporting checklist focusing specifically on COI and funding.
The RIGHT-COI&F statement will help guideline developers with the reporting of COIs and funding, and improve the reporting quality of the guidelines. Journal editors, guideline users and evaluators will particularly benefit from more complete and transparent information on COIs and funding.
Acknowledgments
Funding: This study was supported by the Fundamental Research Funds for the Central Universities (No. lzujbky-2021-ey13). The funders had no role in the study design, data collection and analysis, interpretation of data and writing of the article, or decisions to submit it for publication.
Footnote
Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://atm.amegroups.com/article/view/10.21037/atm-22-2123/coif). YX is funded by China Scholarship Council (No. 202106180043). EAA declares that he has contributed to the development of methods of guideline adaptation, the RIGHT statement, and methodological studies in the field. YC declares that he is the co-founder of the RIGHT Working Group. JE declares that he has contributed to the development of the RIGHT statement. The other authors have no conflicts of interest to declare.
Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. Because this project will not involve collection of primary data from humans or animals, ethical approval and patient consent are not required.
Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.
References
- Institute of Medicine (US). Clinical Practice Guidelines We Can Trust. Washington (DC): The National Academies Press, 2011.
- Grimshaw J, Russell I. Achieving health gain through clinical guidelines. I: Developing scientifically valid guidelines. Qual Health Care 1993;2:243-8. [Crossref] [PubMed]
- Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet 1993;342:1317-22. [Crossref] [PubMed]
- Graham RP, James PA, Cowan TM. Are clinical practice guidelines valid for primary care? J Clin Epidemiol 2000;53:949-54. [Crossref] [PubMed]
- Oh SW, Lee HJ, Chin HJ, et al. Adherence to clinical practice guidelines and outcomes in diabetic patients. Int J Qual Health Care 2011;23:413-9. [Crossref] [PubMed]
- Murad MH. Clinical Practice Guidelines: A Primer on Development and Dissemination. Mayo Clin Proc 2017;92:423-33. [Crossref] [PubMed]
- World Health Organization. WHO handbook for guideline development. World Health Organization, 2014.
- Schünemann HJ, Osborne M, Moss J, et al. An official American Thoracic Society Policy statement: managing conflict of interest in professional societies. Am J Respir Crit Care Med 2009;180:564-80. [Crossref] [PubMed]
- Iannone P, Montano N, Minardi M, et al. Wrong guidelines: why and how often they occur. Evid Based Med 2017;22:1-3. [Crossref] [PubMed]
- Norris SL, Holmer HK, Ogden LA, et al. Conflict of interest disclosures for clinical practice guidelines in the national guideline clearinghouse. PLoS One 2012;7:e47343. [Crossref] [PubMed]
- The Lancet. Managing conflicts of interests in clinical guidelines. Lancet 2019;394:710. [Crossref]
- Muth CC. Conflict of Interest in Medicine. JAMA 2017;317:1812. [Crossref] [PubMed]
- Cohen D, Carter P. Conflicts of interest. WHO and the pandemic flu "conspiracies". BMJ 2010;340:c2912. [Crossref] [PubMed]
- Dyer O. WHO retracts opioid guidelines after accepting that industry had an influence. BMJ 2020;368:m105. [Crossref] [PubMed]
- Tabatabavakili S, Khan R, Scaffidi MA, et al. Financial Conflicts of Interest in Clinical Practice Guidelines: A Systematic Review. Mayo Clin Proc Innov Qual Outcomes 2021;5:466-75. [Crossref] [PubMed]
- Holloway RG, Mooney CJ, Getchius TS, et al. Invited Article: Conflicts of interest for authors of American Academy of Neurology clinical practice guidelines. Neurology 2008;71:57-63. [Crossref] [PubMed]
- Saito H, Ozaki A, Sawano T, et al. Evaluation of Pharmaceutical Company Payments and Conflict of Interest Disclosures Among Oncology Clinical Practice Guideline Authors in Japan. JAMA Netw Open 2019;2:e192834. [Crossref] [PubMed]
- Checketts JX, Sims MT, Vassar M. Evaluating Industry Payments Among Dermatology Clinical Practice Guidelines Authors. JAMA Dermatol 2017;153:1229-35. [Crossref] [PubMed]
- Campbell EG, Rao SR, DesRoches CM, et al. Physician professionalism and changes in physician-industry relationships from 2004 to 2009. Arch Intern Med 2010;170:1820-6. [Crossref] [PubMed]
- Chen Y, Wang J, Zhan S, et al. How to address conflicts of interest in clinical practice guidelines. Medical Journal of Peking Union Medical College Hospital 2019;10:685-91.
- Chen YL, Yao L, Xiao XJ, et al. Quality assessment of clinical guidelines in China: 1993 - 2010. Chin Med J (Engl) 2012;125:3660-4. [PubMed]
- Saleh RR, Majeed H, Tibau A, et al. Undisclosed financial conflicts of interest among authors of American Society of Clinical Oncology clinical practice guidelines. Cancer 2019;125:4069-75. [Crossref] [PubMed]
- Andreatos N, Zacharioudakis IM, Zervou FN, et al. Discrepancy between financial disclosures of authors of clinical practice guidelines and reports by industry. Medicine (Baltimore) 2017;96:e5711. [Crossref] [PubMed]
- Wang Q, Huang C, Li J, et al. A survey of reporting status of conflict of interest and economic evidence in Chinese clinical practice guidelines published in 2017. Chinese Journal of Evidence-Based Medicine 2018;18:379-87.
- Wang Y, Wang Q, Zhao M, et al. Conflict of interest and economic evidence in Chinese clinical practice guidelines published in 2018: A survey. New Medicine 2020;30:139-58.
- Hu J, Chen R, Wu S, et al. The quality of clinical practice guidelines in China: a systematic assessment. J Eval Clin Pract 2013;19:961-7. [PubMed]
- Nejstgaard CH, Bero L, Hróbjartsson A, et al. Association between conflicts of interest and favourable recommendations in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews: systematic review. BMJ 2020;371:m4234. [PubMed]
- Fineberg HV. Conflict of Interest: Why Does It Matter? JAMA 2017;317:1717-8. [Crossref] [PubMed]
- Cosgrove L, Bursztajn HJ, Krimsky S. Developing unbiased diagnostic and treatment guidelines in psychiatry. N Engl J Med 2009;360:2035-6. [Crossref] [PubMed]
- Jiang Z, Zhan S, Jia X, et al. The basic methods and process for developing/revising clinical guidelines. National Medical Journal of China 2016;96:250-3.
- Morciano C, Basevi V, Faralli C, et al. Policies on Conflicts of Interest in Health Care Guideline Development: A Cross-Sectional Analysis. PLoS One 2016;11:e0166485. [Crossref] [PubMed]
- Cosgrove L, Krimsky S, Wheeler EE, et al. Conflict of Interest Policies and Industry Relationships of Guideline Development Group Members: A Cross-Sectional Study of Clinical Practice Guidelines for Depression. Account Res 2017;24:99-115. [Crossref] [PubMed]
- Sinclair D, Isba R, Kredo T, et al. World Health Organization guideline development: an evaluation. PLoS One 2013;8:e63715. [Crossref] [PubMed]
- Schünemann HJ, Al-Ansary LA, Forland F, et al. Guidelines International Network: Principles for Disclosure of Interests and Management of Conflicts in Guidelines. Ann Intern Med 2015;163:548-53. [Crossref] [PubMed]
- Chen Y, Yang K, Marušic A, et al. A Reporting Tool for Practice Guidelines in Health Care: The RIGHT Statement. Ann Intern Med 2017;166:128-32. [Crossref] [PubMed]
- Song Y, Alonso-Coello P, Ballesteros M, et al. A Reporting Tool for Adapted Guidelines in Health Care: The RIGHT-Ad@pt Checklist. Ann Intern Med 2022;175:710-9. [Crossref] [PubMed]
- Wang X, Chen Y, Akl EA, et al. The reporting checklist for public versions of guidelines: RIGHT-PVG. Implement Sci 2021;16:10. [Crossref] [PubMed]
- Xie R, Xia Y, Chen Y, et al. The RIGHT Extension Statement for Traditional Chinese Medicine: Development, Recommendations, and Explanation. Pharmacol Res 2020;160:105178. [Crossref] [PubMed]
- Tang C, Duan Y, Zhang Y, et al. RIGHT for acupuncture: An extension of the RIGHT statement for clinical practice guidelines on acupuncture. J Clin Epidemiol 2021;139:330-9. [Crossref] [PubMed]
- Moher D, Schulz KF, Simera I, et al. Guidance for developers of health research reporting guidelines. PLoS Med 2010;7:e1000217. [Crossref] [PubMed]
- Linstone HA, Turoff M. editors. The Delphi method. Reading, MA: Addison-Wesley, 1975:3-12.
- Vernooij RW, Alonso-Coello P, Brouwers M, et al. Reporting Items for Updated Clinical Guidelines: Checklist for the Reporting of Updated Guidelines (CheckUp). PLoS Med 2017;14:e1002207. [Crossref] [PubMed]
- The 7 point Likert Scale. Available online: https://www.formpl.us/blog/point-likert-scale
- Ding H, Yang N, Deng W, et al. Principles and Methods of WHO Guidelines Development. Chinese Journal of Evidence-Based Medicine 2016;16:471-7.
(English Language Editor: K. Brown)