Methodological and reporting quality of pediatric clinical practice guidelines: a systematic review

Introduction

Clinical practice guidelines (CPGs) are one of the most important sources of evidence-based medical practice. According to the definition of the Institute of Medicine (IOM) in 1990, clinical guidelines are syntheses of the best available evidence that support clinicians, managers and policy makers in decision making about the organization and delivery of health care (1). In 2011, IOM redefined their definition as follows: CPGs are statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options (2). Clear, explicit and transparent CPGs can enable health care practitioners, health administrators, program managers and the public to understand and implement recommendations that can positively affect the health patients and population group (3). Good CPGs in pediatrics can thus help pediatric professionals make clinical decisions and integrate the best available evidence into patient care.

In recent years, the number of available CPGs had grown rapidly. However, some studies have suggested that many existing guidelines are of poor quality. Guidelines on the same topic may make different or even contradictory recommendations (4,5). For example, there are conflicting recommendations in the currently valid guidelines for breast cancer and prostate cancer. An organization dedicated to reducing harm from cancer may place greater value on intensive cancer screening interventions, even though such interventions may prove to be extremely costly for the magnitude of the benefit they provide. Another organization, whose purpose is to promote the overall health of society, may view the same evidence differently, preferring to concentrate on other interventions with greater impact on overall public health (6,7). In fact, many guidelines published in the past have been proven to be unreliable and some even had recommendation with serious errors in the accuracy of clinical test (8-11). The adoption of low-quality guidelines may result in widespread use of ineffective treatments, inefficient practices and ultimately harm the patients (5,12). Therefore, identifying and promoting high-quality guidelines for physicians is of great importance (13).

At present, with the rapid increase in the number of pediatric CPGs, an increasing amount of quality assessment studies for pediatric CPGs have also risen. Appraisal of Guidelines Research and Evaluation (AGREE) and Reporting Items for Practice Guidelines in Health care (RIGHT) are two authoritative tools to evaluate the quality of clinical guidelines. The AGREE was established in 2003 by an international group of researchers from 13 countries (14). The improved version, AGREE II, was officially released in 2009 (15). The purpose of the AGREE II tool is to provide a framework for evaluating the quality of guidelines and a strategy for developing guidelines (16,17). The RIGHT standard can assist developers in reporting guidelines, support journal editors and peer reviewers in considering guideline reports and help health-care practitioners understand and implement guidelines. RIGHT standard was established by a multidisciplinary international team and officially launched in 2017 (18). The RIGHT checklist highlights the importance of reporting PICO (population, intervention, comparisons, outcomes) questions and the quality of the evidence. The RIGHT explanation and elaboration statement (Supplement) provides detailed information and examples, which are lacking in the AGREE tool (18,19).

However, few studies have systematically evaluated the methodological and reporting quality of pediatric CPGs. The quality assessments of CPGs in the field of pediatrics have so far been limited to guidelines included in professional guideline database such as National Guideline Clearinghouse (NGC), and there have been no studies that comprehensively assess the quality of pediatric CPGs published in the common literature databases. But the main channel for pediatricians to obtain CPGs is to search common databases such as MEDLINE via PubMed, and they are unable to identify the quality of pediatric CPGs from these databases. Therefore, this study selected published pediatric CPGs from common literature search databases (MEDLINE via PubMed), and used AGREE II tool and RIGHT standard to evaluate and analyze the methodological quality and reporting quality of these CPGs. We aimed to provide reference for clinical decision-making, and to provide suggestions for the standardized development and evidence-based formulation of pediatric CPGs.

We present the following article in accordance with the PRISMA reporting checklist (available at https://dx.doi.org/10.21037/atm-21-2686).

Methods

Search strategy and selection criteria

We searched MEDLINE via PubMed (1 Jan 1990 to 31 Dec 2019) and selected guideline documents published in the 10 pedicatric journals with the highest impact factors in 2018 and 4 general medical journals [The New England Journal of Medicine (NEJM), The Lancet, The Journal of the American Medical Association (JAMA) and The BMJ]. These journals have the highest impact factors among the journals that are most likely to publish pediatric CPGs. It is therefore likely that the pediatric CPGs published in these journals are among those most cited, and will be the primary choices of readers and users when selecting recommendations to guide their clinical practice. The full search strategy is shown in Appendix 1.

Guideline documents meeting all of following criteria were included: (I) the document accorded with the definition of guidelines issued by the Institute of Medicine (IOM); (II) the language of publication was English; (III) the focus was only on children or adolescents aged <18 years (the target group is only children); and (IV) the focus was on topics related to clinical practice. The following types of documents were excluded: (I) documents interpreting existing guidelines; (II) consensus statements; and (III) documents that could not be obtained after contacting the journal and the developer institution.

Data extraction

Prior to data extraction, two investigators (Lei Wu, Yang Wang) reviewed the RIGHT statement and AGREE-II instrument manuals to become familiar with the checklist items. Data extraction forms were designed for both tools. Two investigators (Lei Wu, Yang Wang) extracted data for all items of both tools independently while masked to each other’s decisions. Basic information such as publication year, country, organization and clinical discipline were also extracted. Any discrepancies in data extraction were resolved through discussion.

The quality assessment tools

Methodological quality

AGREE II instrument was used to evaluate the methodological quality of the included pediatric CPGs. The AGREE II instrument is a 23-item tool that addresses six guideline quality-related domains, and it is available on the website for the AGREE II Research Trust (http://www.agreetrust.org) (20). The two investigators had to respond to 23 questions for each guideline using a scale of 1, ‘strongly disagree’, to 7, ‘strongly agree’, based on examples and instructions described in the AGREE II manual. The reviewers were not allowed to communicate with each other during the evaluation process. In accordance with the AGREE II manual, we summed up the scores of items within each domain, and calculated a scaled domain score for each domain for each CPG as follows: (Obtained Score – Minimum Score)/(Maximum Score – Minimum Score). And the average score of six domain scores was calculated as the overall mean score.

Reporting quality

The RIGHT checklist is divided into seven domains, which include 22 items divided further into 35 sub items (18). The two investigators independently assessed the compliance of the guidelines to the RIGHT checklist after assessing the content reported in the guideline. For each sub item, we assigned a dichotomous score of Yes (compliant) or No (not compliant).

Data analysis

We performed an intra-class correlation coefficient (ICC) consistency analysis to calculate the Kappa value for the two evaluations. Kappa value >0.7 indicates good consistency, and <0.4 indicates poor consistency. We conducted subgroup analysis of AGREE-II and RIGHT evaluation results by journal and time period using the rank sum test. We divided the publication years into three categories according too the release years of the AGREE and AGREE II instruments: 1991–2003, 2004–2009 and 2010–2019. Finally, Spearman rank correlation analysis was used to evaluate the correlation between AGREE II and RIGHT evaluation results. SPSS 22.0 software was used to analyze the data.

Results

Literature Search

A total of 246 records were initially identified. Thirteen documents were excluded as duplicates, and after screening titles and abstracts, 33 guideline documents were excluded because their topics were not relevant to clinical practice. After screening the full texts of the remaining 200 records, 41 documents were excluded because their target population were not restricted to children and/or the full text was not available. Finally, 159 guideline documents were included (Figure 1).

Figure 1 Flow chart detailing the selection process.

Basic characteristics

A summary of the characteristics of the included CPGs is presented in Table 1. Three types of guideline articles were included, 85 documents were identified as guidelines, 59 documents were identified as recommendations, and 15 documents were identified as statements. Most guidelines were published in the journal Pediatrics (n=119, 74.8%). The included guidelines were developed by 42 different organizations, including AAP, USPSTF, AHA, NICE, International Liaison Committee on Resuscitation, Committee on Practice and Ambulatory Medicine, Cystic Fibrosis Foundation, GLAD-PC Steering Group, American Society of Dentistry for Children, EAACI including government departments, academic groups, foundations, the World Health Organization (WHO) and universities. The CPGs covered 25 clinical disciplines according to the ICD-11 classification, including mental, behavioral or neurodevelopmental disorders, diseases of the respiratory system, diseases of the circulatory system, certain infectious or parasitic diseases, and factors affecting health status or contact with health services.

Table 1 Basic information of the included pediatric CPGs
Full table

Evaluation of the methodological quality (AGREE II)

The ICCs between the two reviewers were high (>0.8) in all six domains of AGREE II instrument, which indicates good overall agreement between the reviewers (Table 2).

Table 2 AGREE II domain scores of the included guidelines
Full table

In the evaluation of methodological quality of pediatric CPGs with the AGREE-II instrument, the overall mean score ± standard deviation (SD) of the six domains was 37.8%±12.4%. The lowest score among the guidelines was 16.0%, and the highest score was 86.5%. Only eight (5.0%) guidelines scored above 60%. The mean ± SD domain scores were the following: domain 1 (Scope and purpose) 74.5%±14.2%, domain 2 (Stakeholder involvement) 42.7%±16.2%, domain 3 (Rigour of development) 18.7%±14.2%, domain 4 (Clarity of presentation) 56.5%±17.0%, domain 5 (Applicability) 8.9%±12.7%, domain 6 (Editorial independence) 25.2%±34.6%. And domain 1 was the only domain with a score above 60% (Table 2). Figure 2 shows the scores of the 23 AGREE items in a rose chart. The mean score of item 1 (The overall purpose of the guideline is clearly stated) was the highest (6.03), and the score of item 20 (The potential resource inputs are considered when applying recommendations) was the lowest (1.21). Ten items [5, 7, 8, 9, 13, 14, 18, 19, 20, 21] had a score below 2. The mean AGREE scores tended to increase over time (Figure 3).

Figure 2 Mean AGREE- II scores of each item in the included pediatric clinical practice guidelines. A description of each item is available at http://right-statement.org/

Figure 3 AGREE- II scores and RIGHT reporting compliance by domain and year of publication in the included clinical practice guidelines. (A) AGREE- II scores by domain and year of publication in the included clinical practice guidelines. Domain 1, Scope and purpose; domain 2, Stakeholder involvement; domain 3, Rigour of development; domain 4, Clarity of presentation; domain 5, Applicability; domain 6, Editorial independence. (B) RIGHT reporting compliance by domain and year of publication in the included clinical practice guidelines. Domain 1, Basic information; domain 2, Background; domain 3, Evidence; domain 4, Recommendations; domain 5, Review and quality assurance; domain 6, Funding and declaration and management of interests; domain 7, Other information.

Table 3 and Figure 4 show the standardized scores of each AGREE II domain of guidelines published in different time periods and different journals. Except for the domain 1 (scope and purpose), the mean AGREE II domain scores in the other five domains significantly increased from the period 1991–2003 to 2010–2019. The improvement was particularly pronounced in the domain 3 (Rigour of development), domain 5 (Applicability), and domain 6 (Editorial independence). In the Pediatrics journal, the mean overall AGREE II score was 36.8%±10.3%, and the overall scores varied broadly across the guidelines published in Pediatrics (16.0–68.9%). Of the four journals with highest number of guidelines (Pediatrics, JAMA, BMJ and The Journal of Pediatrics), BMJ and JAMA are high-ranking general medicine journals. In BMJ and JAMA, the overall mean AGREE II scores were 42.4%±17.3% and 41.5%±16.4% respectively, and in The Journal of Pediatrics, 33.6%±9.3%.

Table 3 AGREE II domain scores across the different subgroups
Full table

Figure 4 Mean AGREE-II scores in the included pediatric clinical practice guidelines by journal. In the scatter diagram, the red dots represent the scores of guidelines published between 1991 and 2003, the blue dots represent the scores of guidelines published between 2004 and 2009, and green dots represent the scores of guidelines published between 2010 and 2019. (A) Standardized AGREE-II scores of different journals in domain 1 (Scope and purpose), (B) Standardized AGREE-II scores of different journals in domain 2 (Stakeholder involvement), (C) Standardized AGREE-II scores of different journals in domain 3 (Rigour of development), (D) Standardized AGREE-II scores of different journals in domain 4 (Clarity of presentation), (E) Standardized AGREE-II scores of different journals in domain 5 (Applicability), (F) Standardized AGREE-II scores of different journals in domain 6 (Editorial independence), (G) Standardized AGREE-II scores of different journals in overall assessment. BMJ, The British Medical Journal; JAMA, The Journal of the American Medical Association; JAMAPED, JAMA Pediatrics; JAACAP, Journal of the American Academy of Child and Adolescent Psychiatry; PED, Pediatrics; PAI, Pediatric Allergy and Immunology; JAH, The Journal of Adolescent Health; ECAP, European Child & Adolescent Psychiatry; JPEDS, The Journal of pediatrics; DMCN, The Developmental Medicine & Child Neurology; PEDDIA, Pediatric Diabetes; PEDDEN, Pediatric Dentistry.

Evaluation of the reporting quality (RIGHT Statement)

The mean overall reporting rate was 46.4%±16.6%, ranging between 13.9% to 91.7% across the guidelines. Thirty-one (19.5%) guidelines had an overall reporting rate above 60%. The mean domain reporting rates were the following: domain 1 (Basic information) 73.6%±14.9%, domain 2 (Background) 67.1%±16.4%, domain 3 (Evidence) 32.7%±27.2%, domain 4 (Recommendations) 32.4%±22.5%, domain 5 (Review and quality assurance) 22.9%±40.4%, domain 6 (Funding and declaration and management of interests) 24.1%±36.3%, domain 7 (Other information) 45.3%±30.1% (Table 4). The compliance to item 7a (The main target population for the implementation of the guideline is described) was the highest (99.4%), and the compliance to item 14c (Whether the factors such as fairness, feasibility and acceptability are also considered in the formation of recommendation) was the lowest (11.3%) among the 36 RIGHT checklist items. Eighteen items [1b, 9a, 10b, 11a, 11b, 12, 13b, 13c, 14a, 14b, 14c, 15, 16, 17, 18a, 18b, 21, 22] had a reporting rate below 30%.

Table 4 Compliance to the RIGHT checklist in the included guidelines
Full table

Table 5 and Figure 5 show the reporting rates of each RIGHT domain of guidelines published in different time periods and different journals. The RIGHT reporting rates in the seven domains increased significantly from the period 1991–2003 to 2010–2019. The improvement was particularly notable in the domains (Recommendations), (Review and quality assurance), (Funding and declaration and management of interests) and (Other information). In the Pediatrics journal, the overall mean ± SD reporting rate was 44.8%±13.1%, and the reporting rates of guidelines published in Pediatrics were also widely distributed (16.7–83.3%). In BMJ and JAMA, the overall reporting rates were 59.0%±26.3% and 37.0%±19.4%, respectively, and in The Journal of Pediatrics, 48.3%±22.6%.

Table 5 Compliance to the RIGHT checklist in the different subgroups
Full table

Figure 5 Compliance to the RIGHT checklist in the included pediatric clinical practice guidelines by journal. In the scatter diagram, the red dots represent the compliance of guidelines published between 1991 and 2003, the blue dots represent the compliance of guidelines published between 2004 and 2009, and green dots represent the compliance of guidelines published between 2010 and 2019. (A) RIGHT reporting rates of different journals in domain 1 (Basic information), (B) RIGHT reporting rates of different journals in domain 2 (Background), (C) RIGHT reporting rates of different journals in domain 3 (Evidence), (D) RIGHT reporting rates of different journals in domain 4 (Recommendations), (E) RIGHT reporting rates of different journals in domain 5 (Review and quality assurance), (F) RIGHT reporting rates of different journals in domain 6 (Funding and declaration and management of interests), (G) RIGHT reporting rates of different journals in domain 7 (Other information). (H) RIGHT reporting rates of different journals in overall assessment. BMJ, The British Medical Journal; JAMA, The Journal of the American Medical Association; JAMAPED, JAMA Pediatrics; JAACAP, Journal of the American Academy of Child and Adolescent Psychiatry; PED, Pediatrics; PAI, Pediatric Allergy and Immunology; JAH, The Journal of Adolescent Health; ECAP, European child & Adolescent Psychiatry; JPEDS, The Journal of Pediatrics; DMCN, The Developmental Medicine & Child Neurology; PEDDIA, Pediatric Diabetes; PEDDEN, Pediatric Dentistry.

Correlation of AGREE II and RIGHT scores

There was a high positive correlation between AGREE II scores and RIGHT reporting rates (r=0.645, P<0.001). This correlation is also clear in the scatter plot (Figure 6). The results showed that there was a high correlation between AGREE II and RIGHT, and pediatric CPGs with good methodological quality also had good reporting quality.

Figure 6 Scatter plot of AGREE-II overall scores versus overall compliance to the RIGHT checklist. The dots represent the overall scores of AGREE-II and the overall compliance to RIGHT for each guideline.

Discussion

The quality of the pediatric CPGs included in our study was lower when compared with adult CPGs. Wayant et al. used AGREE-II to evaluate 48 adult CPGs for cancer treatment in the NCCN guidelines library (21). As all guidelines from the study of Wayant were published after 2009, we could compare these findings with the results from the last time period (2010–2019) of our study. The scores of pediatric CPGs in Rigour of development (23.4% vs. 62.4%, P<0.001), Clarity of presentation (67.6% vs. 84.4%, P<0.001), Applicability (16.3% vs. 57.5%, P<0.001), and Editorial independence (53.4% vs. 94.4%, P<0.001) were significantly lower than the scores of adult cancer CPGs.

The quality of the pediatric CPGs included in our study may be also lower than those indexed in professional guideline libraries such as RIGHT. Sun et al. assessed the methodological quality of 50 pediatric CPGs in NGC using the AGREE-II instrument (22). According to their findings, only one domain (domain 5, Applicability, 54.6%) had a mean AGREE II score below 60%. In another study of the NGC guidelines, the average scores of six AGREE II domains were 84%, 42%, 54%, 78%, 19%, 40% respectively (23), which were all higher than those observed in our study.

In recent years, although the number of CPGs has increased, the quality of CPGs is uneven, and some researches even showed that the quality of some CPGs is poor (9,13). Since recommendations of poor quality CPGs can cause harm to the patients, it is particularly important to identify and develop high-quality CPGs for clinicians and health care professionals to use. This study found that the methodological and reporting quality of pediatric CPGs published as journal articles tended to increase over time. But as a whole, the methodological and reporting quality was still poor, and further improvement is needed.

Recommendations for improving the quality of pediatric CPGs

The development process of guidelines needs to follow a rigorous systematic methodology. In order to ensure the quality of the guidelines, strict standards need to be developed. WHO, the Scottish Inter-Collegiate Guideline Network (SIGN), NHMRC, and the Council of Europe and NICE have all published standards for guideline development. The key steps are basically the same, which can be divided into subject selection, evidence synthesis, formation of recommendations, peer review, publication, implementation, and update (24-27). With the progress of guideline methodology, the quality of pediatric CPGs has been improved in various domains, especially those we found to have the lowest scores: rigor of development, conflict of interest management, applicability, and editorial independence. There is however still room for improvement:

Methodological design

At the beginning of the development of pediatric CPGs, guideline makers should determine reasonable outcome selection method, such as systematic review, survey or voting. For example, a guideline developed by Halken et al. identified outcome indicators through systematic review and meta-analysis, which were divided into two categories: primary outcome indicators and secondary outcome indicators (28). At the same time, it is necessary to determine the external evaluation scheme of the guideline, including the list of evaluation experts and the treatment method of evaluation opinions. Guideline makers can also refer to the guideline of Halken et al. (28). The guideline was sent to experts in relevant topics for review after the draft was completed. At the same time, the draft guideline was made publicly available on the Internet for the public to give their feedback. Finally, the collected opinions were assessed and contributed to the revision of the guideline.

Sources and evaluation of evidence

The inclusion and exclusion criteria of evidence should be clearly defined and strictly implemented. At the same time, formal tools or methods (such as Jadad scale, GRADE method) should be used to evaluate the strength and limitations of evidence. In addition to ensuring that these practices are strictly implemented in the formulation of guidelines, attention should be paid to the completeness of expression in the formulation of guideline document. For example, for the description of the systematic review, the whole process of reference retrieval should be described in detail, including time period, database, keywords, etc.

The formation method and strength of recommendations

The method of forming recommendations should be determined, such as how to reach a consensus among the members of the guideline development group. In addition to the strict implementation of this method, the guideline document should give a detailed description of the formation process of the recommendations. The guideline document should also describe the strength of recommendations and the quality of evidence. For example, the guideline developed by Halken et al. defined the grade of evidence, recommendation and the strength of recommendation, and described them clearly for each recommendation (28).

Disclosure and management of conflicts of interest

The disclosure of conflicts of interest has an important impact on the publication quality of the guidelines and the promotion and implementation of the guidelines. Therefore, the guideline documents should pay attention to the disclosure and management of conflicts of interest, and collect detailed information of conflicts of interest as far as possible. As an example, the guidelines formulated by AHA and American Academy of Pediatrics require each team member to submit a statement of conflict of interest disclosure, which can be used as a reference for guideline developers. The conflict of interest disclosure questionnaire in AHA’s guidelines includes a declaration of employment, research grants, other research support, and speaker’ bureaus of the reviewer (29).

Promotion and application of the guidelines

At the beginning of the guideline development process, the promotion and application plan of the guideline should be formulated, including the target users, the advantages and disadvantages of the recommendation, and the costs and resources to be invested in the promotion of guideline. For example, a guideline of the American Academy of Pediatrics contains a large number of content items to guide the application of the guideline, including all aspects to be considered in the application of the guideline, different measures to be taken in the application of the guideline in children of different races and genders, and the cost-effectiveness of the application of the guideline is analyzed in detail (30). The guideline by Halken et al. provides a detailed summary of the promoting factors, barrier factors, audit criteria and resource need in the application of the recommendations in a table (28).

Construction of a professional pediatric guideline library

After the formulation of the guidelines, external measures must be relied on to promote the clinical use of guidelines. A series of problems such as that the clinical guidelines did not attracted enough attention from medical societies, the quality of some guidelines is low, or that doctors rarely use guidelines, have brought great challenges to the promotion and implementation of clinical guidelines. At present, there are few public platforms that provide a large number of high-quality CPGs for free.

However, most pediatric clinical staff and researchers generally use the common literature database such as MEDLINE to search and view the pediatric guidelines. But the quality of pediatric guidelines indexed in such database that cover a large part of the worldwide medical literature is uneven, and the quality seems also to be lower than professional guideline library such as NGC.

Our study also found that even the highest-ranking general medicine journals may also publish low-quality guidelines. Therefore, the readers should not select guidelines solely by the impact factor of journal. The highly variable quality affects greatly the application of pediatric guidelines by pediatric clinical staff and researchers, and limit the role of pediatric guidelines in pediatric clinical practice.

Therefore, it is necessary to form a multi-disciplinary joint expert group to evaluate and screen the published pediatric guidelines, establish a professional pediatric guideline library based on evidence-based methods, and make it available to pediatric clinical staff and researchers with the aim of effectively improving the overall quality of pediatric guidelines. Such professional database would help pediatric health care providers to select the highest-quality pediatric guidelines to better guide pediatric clinical practice. The National Clinical Medical Research Center of Children’s Hospital of Chongqing Medical University is at present making effort to build a database of high-quality pediatric evidence-based guideline.

Limitations

This study has some limitations. We only searched Medline, even though several other databases such as the Web of Science, EMBASE, and Cochrane Library could also include relevant articles. In particular, we only included 12 journals with high impact factors. Therefore, the CPGs included in this research are not necessarily fully representative of the pediatric CPGs published worldwide, which may lead to some bias in the results.

Conclusions

There are exist a large number of pediatric guidelines that are accessible through the PubMed search tool. The methodological and reporting quality of these pediatric guidelines seems to have improved over time together with the launch and development of assessment tools, but their overall quality is still poor. Compared with clinical guidelines for adults and pediatric CPGs indexed in professional guideline libraries, the quality of pediatric CPGs published in common databases is clearly lower. Therefore there is an urgent need to establish a guideline library specifically focusing on pediatric guidelines, which can help to select and promote high-quality rigorously developed CPGs. We also found a high correlation between the methodological quality and the reporting quality in the pediatric CPGs. When developing pediatric guidelines, guideline makers should make full use of both ARGEE and RIGHT tools.

Acknowledgments

Funding: This work was supported by the Chongqing Science and Health Joint Technology Innovation and Application Development Project in 2020 [2020MSXM125 to SZ] and the National Key Research and Development Program of China (2019YFC0840703).

Footnote

Reporting Checklist: The authors have completed the PRISMA reporting checklist. Available at https://dx.doi.org/10.21037/atm-21-2686

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://dx.doi.org/10.21037/atm-21-2686). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Institute of Medicine Committee to Advise the Public Health Service on Clinical Practice Guidelines. Clinical Practice Guidelines: Directions for a New Program. Washington (DC): National Academies Press, 1990.
2. Institute of Medicine Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. Clinical Practice Guidelines We Can Trust. Washington (DC): National Academies Press, 2011.
3. Oxman AD, Schünemann HJ, Fretheim A. Improving the use of research evidence in guideline development: 14. Reporting guidelines. Health Res Policy Syst 2006;4:26. [Crossref] [PubMed]
4. Swales JD. Guidelines on guidelines. J Hypertens 1993;11:899-903. [Crossref] [PubMed]
5. Thomson R, McElroy H, Sudlow M. Guidelines on anticoagulant treatment in atrial fibrillation in Great Britain: variation in content and implications for treatment. BMJ 1998;316:509-13. [Crossref] [PubMed]
6. Leitch AM, Dodd GD, Costanza M, et al. American Cancer Society guidelines for the early detection of breast cancer: update 1997. CA Cancer J Clin 1997;47:150-3. [Crossref] [PubMed]
7. . National Institutes of Health Consensus Development Conference Statement: Breast Cancer Screening for Women Ages 40-49, January 21-23, 1997. National Institutes of Health Consensus Development Panel. J Natl Cancer Inst 1997;89:1015-26. [Crossref] [PubMed]
8. Cluzeau FA, Littlejohns P, Grimshaw JM, et al. Development and application of a generic methodology to assess the quality of clinical guidelines. Int J Qual Health Care 1999;11:21-8. [Crossref] [PubMed]
9. Shaneyfelt TM, Mayo-Smith MF, Rothwangl J. Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. JAMA 1999;281:1900-5. [Crossref] [PubMed]
10. Ward JE, Grieco V. Why we need guidelines for guidelines: a study of the quality of clinical practice guidelines in Australia. Med J Aust 1996;165:574-6. [Crossref] [PubMed]
11. Graham ID, Beardall S, Carter AO, et al. What is the quality of drug therapy clinical practice guidelines in Canada? CMAJ 2001;165:157-63. [PubMed]
12. Unwin N, Thomson R, O'Byrne AM, et al. Implications of applying widely accepted cholesterol screening and management guidelines to a British adult population: cross sectional study of cardiovascular disease and risk factors. BMJ 1998;317:1125-30. [Crossref] [PubMed]
13. Grilli R, Magrini N, Penna A, et al. Practice guidelines developed by specialty societies: the need for a critical appraisal. Lancet 2000;355:103-6. [Crossref] [PubMed]
14. AGREE Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care 2003;12:18-23. [Crossref] [PubMed]
15. Brouwers MC, Kho ME, Browman GP, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ 2010;182:E839-42. [Crossref] [PubMed]
16. Oxman AD, Schünemann HJ, Fretheim A. Improving the use of research evidence in guideline development: 16. Evaluation. Health Res Policy Syst 2006;4:28. [Crossref] [PubMed]
17. Wilson KC, Irwin RS, File TM Jr, et al. Reporting and publishing guidelines: article 12 in Integrating and coordinating efforts in COPD guideline development. An official ATS/ERS workshop report. Proc Am Thorac Soc 2012;9:293-7. [Crossref] [PubMed]
18. Chen Y, Yang K, Marušic A, et al. A Reporting Tool for Practice Guidelines in Health Care: The RIGHT Statement. Ann Intern Med 2017;166:128-32. [Crossref] [PubMed]
19. Moher D, Schulz KF, Simera I, et al. Guidance for developers of health research reporting guidelines. PLoS Med 2010;7:e1000217 [Crossref] [PubMed]
20. Advancing the science of practice guidelines. The AGREE II Instrument. 2021. Available online: https://www.agreetrust.org/
21. Wayant C, Cooper C, Turner D, et al. Evaluation of the NCCN guidelines using the RIGHT Statement and AGREE-II instrument: a cross-sectional review. BMJ Evid Based Med 2019;24:219-26. [Crossref] [PubMed]
22. Sun Y, Chen J, Shi SZ, et al. Are paediatric clinical practice guidelines trustworthy? J Paediatr Child Health 2019;55:1044-55. [Crossref] [PubMed]
23. Boluyt N, Lincke CR, Offringa M. Quality of evidence-based pediatric guidelines. Pediatrics 2005;115:1378-91. [Crossref] [PubMed]
24. Council of Europe. Developing a methodology for drawing up guidelines on best medical practices. (Recommendation (2001)13 and explanatory memorandum). Z Arztl Fortbild Qualitatssich 2002;96:5-59. [PubMed]
25. National Health and Medical Research Council. A guide to the development, implementation and evaluation of clinical practice guidelines. 1998.
26. National Institute for Health and Clinical Excellence Short Clinical Guideline. London: National Institute for Health and Care Excellence (UK), 2007.
27. SIGN. Sign 50: A guideline developers´ handbook. Scottish Intercollegiate Guidelines Network. 2004.
28. Halken S, Larenas-Linnemann D, Roberts G, et al. EAACI guidelines on allergen immunotherapy: Prevention of allergy. Pediatr Allergy Immunol 2017;28:728-45. [Crossref] [PubMed]
29. Newburger JW, Takahashi M, Gerber MA, et al. Diagnosis, treatment, and long-term management of Kawasaki disease: a statement for health professionals from the Committee on Rheumatic Fever, Endocarditis, and Kawasaki Disease, Council on Cardiovascular Disease in the Young, American Heart Association. Pediatrics 2004;114:1708-33. [Crossref] [PubMed]
30. American Academy of Pediatrics Committee on Infectious Diseases. Updated guidance for palivizumab prophylaxis among infants and young children at increased risk of hospitalization for respiratory syncytial virus infection. Pediatrics 2014;134:415-20. [Crossref] [PubMed]