Critical appraisal of the quality of clinical practice guidelines for idiopathic pulmonary fibrosis

Introduction

Idiopathic pulmonary fibrosis (IPF), a chronic and progressive fibrotic lung disease, is the most common type of idiopathic interstitial pneumonia (1,2). IPF has a short median survival time, and although it was once considered rare, its incidence and mortality have risen over time (3-5). Its incidence is estimated to range between 3–9 cases/100,000 people per year in Europe and North America, and is lower in East Asia and South America (3). In 2012, the incidence of IPF in the UK was 80% higher than that in 2000 (6). In 2014 alone, approximately 28,000–65,000 people died of IPF clinical syndrome in Europe, and 13,000–17,000 people died of IPF in the USA (4). In 2017, 21/100,000 people died of IPF in the USA; the male mortality rate was approximately 1.26 times than the female mortality rate (7). Acute exacerbations and comorbidities in IPF are major causes of hospitalization, death, and financial burden (8,9). Unfortunately, due to the lack of precise diagnostic techniques and therapeutics, the diagnosis and management of IPF still constitute major challenges with high complexity for clinicians in practice. Therefore, several countries and international organizations have developed and updated the clinical practice guidelines (CPGs) of IPF to improve the efficiency of diagnosis and intervention.

Ideally, CPGs are statements that include recommendations aimed at optimizing patient care through systematic reviews of evidence and assessments of the benefits and harms of alternative care options (10). The practical values of the guidelines are mainly based on rigorous methodology, transparency of development, and quality of reporting (11,12). In general, clinicians and policy-makers now consider CPGs as an essential tool for selecting the most evidenced and cost-effective treatments for their practice (13,14). However, there is a surplus of IPF CPGs with widely variable quality; therefore, it is important to identify the IPF CPGs of high quality. To date, critical appraisal of the quality of existing CPGs for IPF has been limited. Therefore, we thoroughly assessed CPGs on the diagnosis or/and management of IPF using the Appraisal of Guidelines, Research and Evaluation II (AGREE II) and Reporting Items for Practice Guidelines in Healthcare (RIGHT) checklist, with the aim of finding high-quality CPGs for IPF and to highlight their potential for improvement (15,16).

Methods

Study design

We conducted a critical appraisal of the methodology and reporting quality of CPGs for IPF using the AGREE II and RIGHT checklists, respectively (15,16).

Retrieval of guidelines

We performed a systematic database search in PubMed, Embase, and Web of Science to identify CPGs that presented diagnosis or/and treatment of IPF from inception to July 13, 2020. The database search strategy combined the following terms: idiopathic pulmonary fibrosis, pulmonary fibros*, idiopathic, IPF, practice guideline, guidance*, recommendation*, and consensus. We adjusted the search formulae to different databases (a supplementary appendix for this can be found online). For a more comprehensive search, we also searched the websites of The National Institute for Health and Care Excellence (NICE, https://www.nice.org.uk/), Scottish Intercollegiate Guidelines Network (https://www.sign.ac.uk/), and Guidelines International Network (https://g-i-n.net) as supplemental sources. We also performed a search in Google Scholar to obtain CPGs possibly missed by the systematic searches. The searches were independently performed by two reviewers (XLL and XQY). Disagreements were resolved by consulting a third reviewer (YFM).

Selection of guidelines

We imported the acquired records into EndNote X8 software (Clarivate Analytics), and used its command in combination with the manual elimination of duplicates. We then screened the titles and abstracts to eliminate irrelevant records and obtained the full text of prospective content-related CPGs for IPF. All relevant CPGs were examined in accordance with the following inclusion criteria: (I) complete CPG in English version was available; (II) focus on IPF diagnosis or treatment; and (III) included the updated version for the latest CPG. The CPG exclusion criteria were: (I) secondary or multiple publications; and (II) reviews, short summaries, concluding remarks, editorials, interpretations, position papers, and other versions of CPGs still under review. For each filtered guideline, we thoroughly searched for accompanying appendices and supplementary documents to better inform our assessments. Two reviewers (YX and ZZF) selected the guidelines independently and consulted a methodology expert (YLC) to resolve ambiguities.

Data extraction of guidelines

We designed a standard form for data extraction covering the first author, publication year, country, topic, version, developers, grading system, and number of recommendations. To distinguish between guidelines, we used the first author and publication year as the ID of the guideline. For guidelines in which the first author was not mentioned, we used the abbreviation of the development institution plus publication year as the guideline ID. One reviewer (XLL) extracted data from the CPGs, and a second reviewer (XQY) checked the data.

Methodological quality assessment of guidelines

To evaluate whether the filtered CPGs for IPF met our preset inclusion criteria, the AGREE II instrument was adopted. AGREE II comprises 23 items divided into 6 domains: scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, and editorial independence. Each domain identified a unique dimension of the methodology quality of the included CPGs, and each item was assessed with a seven-point score, ranging from 1 (strongly disagree) to 7 (strongly agree). Domain scores were calculated according to the following formula: (obtained score – minimal possible score) / (maximal possible score – minimal possible score) ×100%. Each included guideline was generally given an overall quality assessment score based on the average score for all domains. Because neither the AGREE tool nor previous studies had defined a uniform criterion for overall quality, we considered domains with scores <50% to indicate lower quality, 50–70% as sufficient quality, and >70% as good quality in our study (17-20). Three reviewers (XQY, XLL, and ZZF) independently performed the methodological quality appraisals based on the AGREE II (15) under the guidance of two methodological experts (YFM and YLC). Before formal assessment, a meeting was held to discuss the appraisal criteria according to the AGREE II (15).

Reporting quality assessment of guidelines

To assess whether the reporting quality of each IPF CPG met our preset inclusion criteria, the RIGHT checklist (16), consisting of 22 items divided into 7 domains, was used. Certain requirements were divided into two or three subitems according to their content. Therefore, the checklist covered 35 items in 7 domains, including basic information (6 items), background (8 items), evidence (5 items), recommendations (7 items), review and quality assurance (2 items), funding and conflicts of interest statements and management (4 items), and other information (3 items). We rated items as “reported” (relevant information was fully presented), “partially reported” (relevant information was partly presented), “unreported” (lack of all relevant information), or “not applicable” (not appropriate for evaluating specific guidelines), based on the protocol of a previous study (21). Two reviewers (XLL and ZZF) performed the reporting quality appraisals independently under the training and guidance of the RIGHT checklist initiator (YLC). We completed two rounds of pre-evaluation before formal assessment, and SPSS version 22.0 software was used to calculate the kappa value of the coefficient of internal consistency.

Statistical analysis

We used Excel 2016 software (Microsoft) to document the rates and percentages of reporting of the RIGHT items, as well as the score for the domains of the AGREE II. The means and ranges of the CPG scores for each AGREE II domain were calculated. Agreement among reviewers was measured by intraclass correlation coefficient (ICC) calculated by SPSS version 22.0, according to previous studies (17-20). The ICC level was classified as poor (<0.40), fair (0.40–0.59), good (0.60–0.74), or excellent (0.75–1.00). The relationship between the methodological quality and the reporting quality of included CPGs were analyzed by Spearman’s correlation using SPSS version 22.0 software, according to previous research (22). We made a judgment of “fully reported”, “partially reported”, “unreported”, or “not applicable”, with corresponding scores of 1, 0.5, 0, and 0, respectively. The total scores that could be obtained with the RIGHT checklist and the AGREE II tool were 35 and 161, respectively. The relationship between the methodological quality and the reporting quality is presented in the scatter gram.

Results

Identification of guidelines

The electronic search yielded 2,629 records. Duplicated records were removed, and 1,678 records were excluded according to their titles and abstracts. Eventually, 60 full-text articles were filtered for eligibility, and 12 CPGs (2,23-33) met our preset eligibility criteria and were therefore included in the present study (Figure 1).

Figure 1 CPGs selection process. CPG, clinical practice guidelines.

Characteristics of guidelines

All 12 included CPGs (11 CPGs published in journals and 1 on the NICE website) were published between 2013 and 2020 (2,23-33). All were developed by local or international medical societies. Two CPGs (2,23) were globally developed, and the others came from nine different countries: France, Germany, Spain, Switzerland, Japan, Korea, England, Brazil, and Poland (24-33). Six CPGs (24,26,28,30,31,33) covered the diagnosis and treatment of IPF, five referred to treatment (23,25,27,29,32), and one referred to diagnosis (2). Six CPGs were updated and developed from original guidelines (2,23-25,27,31), three were adapted from other guidelines (26,28,30), and the remaining three were original versions (2,32,33). The ranking systems of evidence quality also varied among these CPGs. The Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) system (34) was adopted to grade evidence quality in eight CPGs (2,23,26,27,29,31-33). The Oxford Centre for Evidence-based Medicine Levels of Evidence (35) were adopted for one of the selected guidelines (25), and the remaining three CPGs did not report the criteria they used to grade evidence (24,28,30). Recommendations for the included CPGs ranged from 7 to 66 (Table 1).

Table 1 The characteristics of eligible CPGs
Full table

Result of methodology quality

Scope and purpose and clarity of presentation were the two domains where CPGs had the highest mean score, with scores of 70.99% (range, 27.8–88.89%) and 68.06% (range, 16.7–85.2%), respectively. The mean score for clarity of presentation had the highest range. CPGs had the lowest mean score for the rigor of development and applicability domains: 50.1% and 34.1% (range, 6.3–70.1% and 4.2–61.1%, respectively). The mean score of the editorial independence domain was 54.1%, with the outlier being 2.9% (28). Overall, ICC scores were >80% in each domain, suggesting considerable agreement between the reviewers (Table 2).

Table 2 AGREE II domain and overall assessment for eligible guidelines
Full table

Additionally, two CPGs scored <50% in each domain (26,27), one guideline scored 29.2% for the rigorous development domain (24), and one guideline in the editorial independence domain scored 2.9% (28). Therefore, these CPGs were not recommended. One guideline was recommended with modifications, as it failed to describe facilitators and barriers to its application, and presented monitoring and/or auditing criteria in the applicability domain, where it only scored 13% (30). The remaining CPGs were recommended because they had a score >50% score in at least 5 domains (2,23,25,29,31-33). In particular, two guidelines (32,33) published in 2020 scored >50% in each domain, indicating high methodological quality (Figure 2).

Figure 2 The scores of eligible guidelines in AGREE II domains. AGREE II, Appraisal of Guidelines for Research and Evaluation Instrument II.

Result of reporting quality

Our pre-evaluation results (kappa=0.831, P<0.001) demonstrated good agreement between the reviewers. Of the seven domains, the basic information domain and other information domain had the highest full reporting rates; however, the full reporting rates of the remaining six domains were <50%. The evidence domain and the review and quality assurance domain had unreported rates of >25%, and the background domain had the lowest unreported rate (8.3%) of all domains. Partial reports existed in all domains, ranging from 12.5% to 34.5% (Figure 3).

Figure 3 The scores of included guidelines in RIGHT checklist domains. RIGHT, Reporting Items for Practice Guidelines in Healthcare.

For specific items, 8 items had a satisfactory level of reporting quality; ≥70% CPGs adhered to the RIGHT checklist: 1a: identify the report as a guideline; 1c: describe the focus of the guideline; 4: identify at least one corresponding author; 5: describe the basic epidemiology of the problem; 7a: describe the primary populations; 13a: provide clear recommendations; 15: describe the processes and approaches to formulate the recommendations; and 20: how is the guideline assessed, respectively. However, 14 items had poor reporting; ≤25% CPGs adhered to the checklist: 7b: give special consideration for subgroups; 8b: describe the settings for which the guideline is intended; 9b: list all necessary information of the individuals involved in developing the guideline; 10: indicate how to select and sort the outcomes; 11b: describe how to identify and assess the system review; 14a: describe whether values and preferences of the target populations were considered; 14b: describe whether cost and resource implications were considered; 14c: describe other factors taken into consideration; 17: indicate whether the guideline was subjected to a quality assurance process; 18a: describe the specific sources of funding for all stages of guideline development; 18b: describe the role of funders in the development of guideline; 19b: describe how conflicts of interest were evaluated and managed; and 22: describe any limitations in the guideline development process (Table 3).

Table 3 The reporting rate in each RIGHT checklist Item of eligible guidelines
Full table

Correlation of methodological and reporting quality

The Spearman correlation analysis showed that R=0.872, P=0.000 (Figure 4).

Figure 4 The correlation of methodological and reporting quality.

Discussion

In this critical appraisal of 12 CPGs for IPF using the validated AGREE II tool and RIGHT checklist, 7 CPGs were considered to be high quality and suitable for recommendation to clinical practitioners and policy-makers. The international guidelines and the guidelines issued in 2020, in particular, were of high methodological and reporting quality (2,23,32,33). Although two CPGs used the GRADE system to classify the quality of evidence, their scores in each domain of AGREE II were <50%, indicating that the methodological quality was insufficient (26,27). One guideline scored 29.2% score in the rigorous development domain, as it failed to explicitly describe the criteria for selecting evidence and making recommendations (24), and one guideline in the editorial independence domain scored only 2.9%, as it failed to explicitly describe the conflicts of interest (28); therefore, these CPGs were not recommended. We recommend that these guidelines (24,26-28) use the GRADE system for grading the quality of evidence, and use the AGREE II tool to improve the methodology quality when it is updated or revised.

The performance of the CPGs across the AGREE II domains was poor; the mean CPG scores were low, and ranges differed greatly between the scope and purpose and the clarity of presentation domains. The CPGs had mean scores of >60%, which was consistent with previous studies (17-19). In the rigor of development domain, the mean score of eligible CPGs was 50.9%, with a range of 6.3–70.1%. This domain was used to define the quality of guidelines because it highlights the difficulties of producing evidence-based recommendations in related research, including whether the methods for guideline development are considered exhaustive enough to produce evidence-based recommendations (36,37). In our study, even recommended CPGs were likely to have lower scores (≤70%) in the rigor of development domain; thus, developers of CPGs should pay particular attention to the methods used for developing guidelines. Similar to other studies, the applicability domain had the lowest mean score of 34.14%, with a range of 4.2–61.1% (17-19,36,38,39). Low applicability could lead to limited compliance and failure to achieve the purpose of the guidelines; therefore, formulating an appropriate implementation strategy is important (40).

The reporting quality of CPGs for IPF published in journals and on guideline developers’ websites from 2013 to 2020 was evaluated using the RIGHT checklist (16). The RIGHT checklist is generally a reliable and useful tool for the assessment of reporting quality of guidelines (21,41). Among its domains, the basic information domain had the highest overall reporting rate, while the other 6 domains had rates of <50%. Eight items (22.9%) had a satisfactory level of full reporting, and 14 items (40%) had a poor level of reporting, according to the RIGHT checklist. The common deficiencies in the reporting of the included IPF guidelines were: (I) a lack of a summary of the recommendations contained in the guideline; (II) a lack of adequate descriptive methods for the selection and classification of clinical outcomes; (III) the funding for the various stages of guideline development were rarely described; and (IV) a lack of description of the limitations of the guideline development process. These deficiencies called for meticulous evaluation on the reporting quality of existing IPF guidelines and the improvement of quality for reporting in practice.

The AGREE II tool was developed for the methodological rigor and transparency of CPGs (42). Meanwhile, the aim of the RIGHT checklist is to assist guideline developers in reporting CPGs, to support journal editors and peer reviewers when considering guideline reports, and to help clinical practitioners understand and implement a guideline; the checklist differs from the AGREE II in many important aspects (16). Our research showed that IPF guidelines with better methodological quality also had greater reporting quality. Therefore, applying the AGREE II tool and RIGHT checklist together to evaluate the quality of CPGs, we were able to identify the possible gaps in the different aspects as well as areas for further improvement.

To the best of our knowledge, the present study was the first to assess the quality of CPGs for IPF by combining the AGREE II tool with the RIGHT checklist. Taken together, our study highlights the areas for further improvement in the methodology and reporting of CPGs for IPF.

Limitations

The present study has some limitations. First, we included only CPGs published in English. Although we attempted to perform systematic searches and reasonable screening of the literature, the smaller scope might still hinder the diversity of regions and settings. Second, evaluating the methodology and reporting quality of CPGs using the AGREE II and RIGHT checklists is a subjective process. We independently evaluated and used ICC scores to examine agreement; however, bias was still inevitable. Third, due to limitations of journal layout, the reporting of some CPGs might be incomplete, resulting in a possible loss of some important information and reduced reporting quality. Despite these shortcomings, our study was conducted under the guidance of a guideline methodologist, and a comprehensive search for CPGs and excellent agreement enhanced the credibility of our findings.

Conclusions

Our critical appraisal of CPGs for IPF found that more than half of the guidelines are of high quality and could be recommended; however, the methodological quality of IPF CPGs varied greatly, and the full reporting rate is low in some domains. In the future, we should focus not only on improving the methodological quality for the development of guidelines, but also the reporting quality of guidelines.

Acknowledgments

Funding: This study was supported by Key Project of the National Natural Science Foundation of China (81830116); Qihuang Scholars Award of the State TCM Academic Leader Program; Central Plains Thousand People Program (No. 194200510018, ZYQR201810159); Training Project for Top-notch Talent of Traditional Chinese Medicine in Henan Province (2019ZYBJ002).

Footnote

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/atm-20-3200). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. Ethics application was not required.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Richeldi L, Collard HR, Jones MG. Idiopathic pulmonary fibrosis. Lancet 2017;389:1941-52. [Crossref] [PubMed]
2. Raghu G, Remy-Jardin M, Myers JL, et al. Diagnosis of Idiopathic Pulmonary Fibrosis. An Official ATS/ERS/JRS/ALAT Clinical Practice Guideline. Am J Respir Crit Care Med 2018;198:e44-68. [Crossref] [PubMed]
3. Hutchinson J, Fogarty A, Hubbard R, et al. Global incidence and mortality of idiopathic pulmonary fibrosis: a systematic review. Eur Respir J 2015;46:795-806. [Crossref] [PubMed]
4. Hutchinson JP, McKeever TM, Fogarty AW, et al. Increasing global mortality from idiopathic pulmonary fibrosis in the twenty-first century. Ann Am Thorac Soc 2014;11:1176-85. [Crossref] [PubMed]
5. Raghu G, Chen SY, Yeh WS, et al. Idiopathic pulmonary fibrosis in US Medicare beneficiaries aged 65 years and older: incidence, prevalence, and survival, 2001-11. Lancet Respir Med 2014;2:566-72. [Crossref] [PubMed]
6. Strongman H, Kausar I, Maher TM. Incidence, Prevalence, and Survival of Patients with Idiopathic Pulmonary Fibrosis in the UK. Adv Ther 2018;35:724-36. [Crossref] [PubMed]
7. Dove EP, Olson AL, Glassberg MK. Trends in Idiopathic Pulmonary Fibrosis-related Mortality in the United States: 2000-2017. Am J Respir Crit Care Med 2019;200:929-31. [Crossref] [PubMed]
8. Diamantopoulos A, Wright E, Vlahopoulou K, et al. The Burden of Illness of Idiopathic Pulmonary Fibrosis: A Comprehensive Evidence Review. Pharmacoeconomics 2018;36:779-807. [Crossref] [PubMed]
9. Yu YF, Wu N, Chuang CC, et al. Patterns and Economic Burden of Hospitalizations and Exacerbations Among Patients Diagnosed with Idiopathic Pulmonary Fibrosis. J Manag Care Spec Pharm 2016;22:414-23. [Crossref] [PubMed]
10. Institute of Medicine (US) Committee on Standards for Developing Trustworthy. Clinical Practice Guidelines. Clinical practice guidelines we can trust. Washington, DC: The National Academies Press, 2011.
11. Murad MH. Clinical Practice Guidelines: A Primer on Development and Dissemination. Mayo Clin Proc 2017;92:423-33. [Crossref] [PubMed]
12. Glasziou P, Altman DG, Bossuyt P, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet 2014;383:267-76. [Crossref] [PubMed]
13. Shekelle PG. Clinical Practice Guidelines: What's Next? JAMA 2018;320:757-58. [Crossref] [PubMed]
14. Steel N, Abdelhamid A, Stokes T, et al. A review of clinical practice guidelines found that they were often based on evidence of uncertain relevance to primary care patients. J Clin Epidemiol 2014;67:1251-57. [Crossref] [PubMed]
15. Brouwers MC, Kho ME, Browman GP, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. J Clin Epidemiol 2010;63:1308-11. [Crossref] [PubMed]
16. Chen Y, Yang K, Marušic A, et al. A Reporting Tool for Practice Guidelines in Health Care: The RIGHT Statement. Ann Intern Med 2017;166:128-32. [Crossref] [PubMed]
17. Andrade R, Pereira R, van Cingel R, et al. How should clinicians rehabilitate patients after ACL reconstruction? A systematic review of clinical practice guidelines (CPGs) with a focus on quality appraisal (AGREE II). Br J Sports Med 2020;54:512-9. [Crossref] [PubMed]
18. Jiang M, Guan WJ, Fang ZF, et al. A Critical Review of the Quality of Cough Clinical Practice Guidelines. Chest 2016;150:777-88. [Crossref] [PubMed]
19. Shen WQ, Yao L, Wang XQ, et al. Quality assessment of cancer cachexia clinical practice guidelines. Cancer Treat Rev 2018;70:9-15. [Crossref] [PubMed]
20. Sigfrid L, Perfect C, Rojek A, et al. A systematic review of clinical guidelines on the management of acute, community-acquired CNS infections. BMC Med 2019;17:170. [Crossref] [PubMed]
21. Wang Q, Duan Y, Liang J, et al. Reporting quality of 2014-2018 clinical practice guidelines on diabetes according to the RIGHT checklist. Endocrine 2019;65:531-41. [Crossref] [PubMed]
22. Yun X, Yaolong C, Zhao Z, et al. Using the RIGHT statement to evaluate the reporting quality of clinical practice guidelines in traditional Chinese medicine. PLoS One 2018;13:e0207580. [Crossref] [PubMed]
23. Raghu G, Rochwerg B, Zhang Y, et al. An Official ATS/ERS/JRS/ALAT Clinical Practice Guideline: Treatment of Idiopathic Pulmonary Fibrosis. An Update of the 2011 Clinical Practice Guideline. Am J Respir Crit Care Med 2015;192:e3-19. [Crossref] [PubMed]
24. Cottin V, Crestani B, Cadranel J, et al. French practical guidelines for the diagnosis and management of idiopathic pulmonary fibrosis - 2017 update. Full-length version. Rev Mal Respir 2017;34:900-68. [Crossref] [PubMed]
25. Behr J, Günther A, Bonella F, et al. German Guideline for Idiopathic Pulmonary Fibrosis - Update on Pharmacological Therapies 2017. Pneumologie 2018;72:155-68. [Crossref] [PubMed]
26. Xaubet A, Ancochea J, Bollo E, et al. Guidelines for the diagnosis and treatment of idiopathic pulmonary fibrosis. Sociedad Española de Neumología y Cirugía Torácica (SEPAR) Research Group on Diffuse Pulmonary Diseases. Arch Bronconeumol 2013;49:343-53. [PubMed]
27. Xaubet A, Molina-Molina M, Acosta O, et al. Guidelines for the medical treatment of idiopathic pulmonary fibrosis. Arch Bronconeumol 2017;53:263-69. [Crossref] [PubMed]
28. Funke-Chambour M, Azzola A, Adler D, et al. Idiopathic Pulmonary Fibrosis in Switzerland: Diagnosis and Treatment. Respiration 2017;93:363-78. [Crossref] [PubMed]
29. Homma S, Bando M, Azuma A, et al. Japanese guideline for the treatment of idiopathic pulmonary fibrosis. Respir Investig 2018;56:268-91. [Crossref] [PubMed]
30. Lee SH, Yeo Y, Kim TH, et al. Korean Guidelines for Diagnosis and Management of Interstitial Lung Diseases: Part 2. Idiopathic Pulmonary Fibrosis. Tuberc Respir Dis (Seoul) 2019;82:102-17. [Crossref] [PubMed]
31. The National Institute for Health and Care Excellence. Idiopathic pulmonary fibrosis in adults: diagnosis and management (Last updated May 2017). Accessed March 1, 2020. Available online: https://www.nice.org.uk/
32. Baddini-Martinez J, Ferreira J, Tanni S, et al. Brazilian guidelines for the pharmacological treatment of idiopathic pulmonary fibrosis. Official document of the Brazilian Thoracic Association based on the GRADE methodology. J Bras Pneumol 2020;46:e20190423. [Crossref] [PubMed]
33. Piotrowski WJ, Bestry I, Białas AJ, et al. Guidelines of the Polish Respiratory Society for diagnosis and treatment of idiopathic pulmonary fibrosis. Adv Respir Med 2020;88:41-93. [Crossref] [PubMed]
34. Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008;336:924-26. [Crossref] [PubMed]
35. Oxford Centre for Evidence-based Medicine. Levels of Evidence (March 2009). Accessed March 1, 2020. Available online: https://www.cebm.net/2009/06/oxford-centre-evidence-based-medicine-levels-evidence-march-2009/
36. Molino CGRC, Leite-Santos NC, Gabriel FC, et al. Factors Associated With High-Quality Guidelines for the Pharmacologic Management of Chronic Diseases in Primary Care: A Systematic Review. JAMA Intern Med 2019;179:553-60. [Crossref] [PubMed]
37. Sox HC. Conflict of Interest in Practice Guidelines Panels. JAMA 2017;317:1739-40. [Crossref] [PubMed]
38. Acuña-Izcaray A, Sánchez-Angarita E, Plaza V, et al. Quality assessment of asthma clinical practice guidelines: a systematic appraisal. Chest 2013;144:390-7. [Crossref] [PubMed]
39. Armstrong JJ, Goldfarb AM, Instrum RS, et al. Improvement evident but still necessary in clinical practice guideline quality: a systematic review. J Clin Epidemiol 2017;81:13-21. [Crossref] [PubMed]
40. Chan WV, Pearson TA, Bennett GC, et al. ACC/AHA Special Report: Clinical Practice Guideline Implementation Strategies: A Summary of Systematic Reviews by the NHLBI Implementation Science Work Group: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Circulation 2017;135:e122-37. [Crossref] [PubMed]
41. Tokalić R, Viđak M, Buljan I, et al. Reporting quality of European and Croatian health practice guidelines according to the RIGHT reporting checklist. Implement Sci 2018;13:135. [Crossref] [PubMed]
42. Burls A. AGREE II-improving the quality of clinical care. Lancet 2010;376:1128-29. [Crossref] [PubMed]