Evaluation of the current guidelines for the management of haemorrhoidal disease using the Appraisal of Guidelines Research and Evaluation II instrument

Introduction

Haemorrhoids are the most common proctological condition in the Western world and their prevalence rate is around 4.5%, Furthermore, the most common affected cohort is those aged between 45 and 65 years (1,2). It has been reported that by the age of 50, half of the general population has experienced symptoms related to haemorrhoids (1-3). The debate over definition of symptomatic haemorrhoids is controversial and ongoing. Often, a variety of diverse symptoms are attributed to haemorrhoids (rightly or wrongly) either by physicians or by the patients themselves making it difficult for clinicians to define symptomatic haemorrhoids and has the consequential effect of either over or undertreatment of the symptoms (1-4). The most common anorectal conditions that overlap with haemorrhoids are fissures, skin tags, abscesses, inflammatory bowel diseases (IBD), and anorectal neoplasms (5,6). Painless rectal bleeding during or immediately after defecation is the most common presentation of haemorrhoids (1-6). Common associated symptoms are itching, soiling, prolapse, swelling and discomfort of the perianal area (1-6). Recurrent bleeding may cause secondary iron deficiency anaemia and rarely overwhelming bleeding may need urgent hospitalisation and blood transfusion (7,8).

In order to tailor an individualised treatment strategy, classifications and scoring systems for haemorrhoids have been developed. The Goligher classification was developed to classify internal haemorrhoids according to the presence and severity of prolapse (9). Principal limitations of the Goligher classification could be considered the lack of consideration of etiopathogenesis, and lack of consideration of associated symptoms and their impact on quality of life and from an anatomical point of view does not consider the difference between circumferential prolapse versus a single prolapsing haemorrhoid (10). To overcome the above limitations newer classification systems have been developed. In particular, the Nyström system is based on the frequency of discomfort, pain, itching, soiling, and need of manual reduction of prolapsed haemorrhoids (11). The system grades the frequency of the symptoms into four grades (I) never, (II) less than one-week, (III) 1–6 times per week and (IV) “every day”; it has been easily validated. However, it is limited by not considering the presence and frequency of cases with prolapse that do not need manual reduction. (11,12). In 2019, Rørvik et al., modified the Nyström scoring system considering how often the patient experiences prolapse of haemorrhoids but again not the need for manual reduction (13). Consequently, some other classifications have been proposed; however, they failed to gain widespread use due to their complexity (14,15). The most recent evidence based on the network meta-analysis by Jin et al. demonstrates that the treatment of haemorrhoids is not necessarily related to their symptomatic classification, but rather on their morphological presentation (16).

Realising the importance of the management of symptomatic haemorrhoids several professional societies have produced guidelines in order to aid effective management of patients with symptomatic haemorrhoids.

The aim of the present study was to evaluate the methodological quality of these guidelines using the Appraisal of Guidelines Research and Evaluation II (AGREE II) instrument. This study was presented in accordance with the PRISMA reporting checklist (available at https://atm.amegroups.com/article/view/10.21037/atm-22-4255/rc).

Methods

AGREE-II is a validated tool for assessment of the methodological quality of guidelines and moreover is supported by the World Health Organisation (WHO) advisory committee on health research and by many guidelines’ development teams (17,18). The AGREE II tool comprises 23 items divided into six domains: scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, and editorial independence. For further details regarding the criteria used to describe and evaluate the 6 domains and the 23 consisted items, please see Figure S1. Detailed information can be found also at www.agreetrust.org (19).

Search strategy and guideline selection

PG and AA independently conducted a systematic literature search into Embase, PubMed, Cochrane library and Google Scholar using the following terms: clinical practice guidelines, hemorrhoidal disease, haemorrhoids or hemorrhoids. The search was limited to guidelines published in the English language. The search strategy was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) (20). After independent evaluation of guidelines by two authors (PG and AA) the following data were extracted: country of origin, year of publication, developers, funding resources, evaluation measures.

Appraisal of guidelines

The AGREE II tool was used to assess the quality of guidelines. Five appraisers (PG, AA, EG, NDA, SDS) evaluated the guidelines. Further details regarding the criteria used to describe and evaluate the 6 domains and the 23 items of the AGREE II tool are described in previous publications (21,22).

As per the AGREE II manual, discrepancies of more than 2 standard deviations (SDs) were resolved through dialogue. Authors had the ability to change their entry after group discussion. Domain scores were calculated with the following formula:

(Obtainedscore−Minimumpossiblescore)(Maximumpossiblescore−Minimumpossiblescore)×100%[1]

Results

AGREE II appraisal

A total of six guidelines were identified including: ACRSI (Association of Colon & Rectal Surgeons of India), ASCRS (American Society of Colon and Rectal Surgeons), ESCP (European Society of Coloproctology), SICCR (Società Italiana di Chirurgia Colorectale, which is Italian Society of Colorectal Surgery), PSG (Portuguese Society of Gastroenterology) and JPG (Japanese Practice Guidelines for anal disorders, chapter hemorrhoids) (23-28) (Figure S2).

Scores across the six domains varied widely; the overall median score across all the guidelines was 73% (Figures 1,2).

Figure 1 Scores for each domain for the individual guidelines. ACRSI, Association of Colon & Rectal Surgeons of India; ASCRS, American Society of Colon and Rectal Surgeons; ESCP, European Society of Coloproctology; JPG, Japanese Practice Guidelines; PSG, Portuguese Society of Gastroenterology; SICCR, Società Italiana di Chirurgia Colorectale (Italian Society of Colorectal Surgery).

Figure 2 Median scores for each domain. Error bars denote the lowest and highest scoring guidelines.

In domain I (Scope and purpose), the median score was 85%. The best score of 98% was observed in both SICCR and PSG and the lowest in ACRSI at 40%.

In domain II (Stakeholder involvement), the median score was 41%. ESCP scored the best at 89% and the lowest score was observed in JPG guidelines at 25%.

In domain III (Rigour of development), the median score was 66%, the highest scored was observed in ASCRS at 81% and the lowest in ACRSI at 19%.

In domain IV (Clarity of presentation), two guidelines ASCRS and ESCP shared the best score with 89% and the lowest score was observed in ACRSI at 68%. The median score of this domain was 81%.

In domain V (Applicability), ESCP scored 72% and the lowest score was observed in the JPG guidelines at 13%. The median score of the domain was 48%.

In domain VI (Editorial independence), the median score was 95%. The highest score was observed in ESCP at 100% and the lowest in the JPG guidelines at 4%.

Domain of overall assessment and recommendation for use of each guideline

Only the SICCR, ASCRS and ESCP guidelines were unanimously voted suitable for use in their present form, whilst the ACRSI guidelines received a score of ‘Yes with Modification’ (Figures 1,2).

Discussion

This study demonstrates that there is considerable variability with regards to the quality of haemorrhoid guidelines. The SICCR, ESCP and ASCRS guidelines scored the highest across almost all of the domains whilst the PSG and ACRSI scored reasonable scores across all the domains too. The JPG scored the lowest scores across all the domains.

Overall, domain VI (editorial independence) scored the highest out of all the domains. This domain assesses the editorial independence by checking whether the developers of the guidelines provided an explicit statement declaring whether they have any competing interests. In this domain, one guideline scored 95%. Five of the six guidelines scored above 95% (Figure 1).

These findings are similar to previous appraisals of guidelines for other colorectal conditions (20,21). This finding demonstrates that the developers explicitly describe the competing interests and guarantees that if there is a funding body this does not influence the content of the guideline and the development of recommendations.

In Domain I (scope and purpose) four (SICCR, PSG, ASCRS, ESCP) out of six guidelines scored above 94%. This finding demonstrates that the overall objectives regarding diagnosis, treatment, prevention, follow-up and the expected health benefits from the guidelines were specific to the health topic and reported clearly. Moreover, the target population, proposed interventions, comparisons and outcomes were reported clearly, particularly for the key recommendations.

Irrespective of healthcare issues or pathology, guidelines almost universally seem to score poorly in Domain II which pertains to the involvement of stakeholders. The Achilles’ heel of this domain is the limited representation of all relevant specialties. Moreover, contemporary patient centred healthcare systems require participation of patients within the target population. Despite the universally accepted modern approach to patient centred healthcare, repeated publications across a wide variety of subspecialties unfortunately indicate that patient groups are not being included in the formulation of guidelines. This is particularly concerning given that a considerable number of guidelines are produced by world renowned institutions and societies. In this domain ESCP, the PSG and SICCR guidelines achieved high scores. The remaining three scored very low and brought the overall score at 41%, the lowest of all the domains (Figure 1).

Scores for Domain III: Rigour of Development assesses the methodology of the guidelines and how robust the creation of the guidelines has been. Generally, the guidelines scored reasonably in this Domain. The highest score was obtained by the ASCRS (81%). Although three more guidelines namely ESCP, PSG and SICCR scored higher than 70%, the median score was 66%, because the lowest by ACRSI at 19% score had a negative impact on the median score. This finding demonstrates that the search of the literature was systematic, the electronic databases where the searched was performed was described and the search terms used precisely. In addition, evidence selection criteria, strengths and limitations of the evidence were described clearly. Moreover, the methods used to formulate the guidelines and the recommendations, and how final decisions were reached were described by the developers precisely.

The results of the domain IV (clarity of presentation) were quite satisfactory. Four (ASCRS, ESCP, PSG and SICCR) obtained scores above 88%. They achieved these results because their recommendations were specific and unambiguous. Moreover, the proposed management options and key recommendations were easily identifiable.

The haemorrhoid guidelines scored poorly in domain V (Applicability) (48%). The guidelines gave no indication or guidance as to how their recommendations should be implemented. This is another domain which has historically scored very poorly. Previous studies have reported this issue, namely that whilst guidelines may be good at presenting and clarifying their recommendations, they are often poor at reporting how clinicians should integrate these recommendations into clinical practice (21,22,29,30).

This historic tendency of guidelines to ignore applicability of their recommendations is very concerning as without a clear road map of how clinical practise could be improved, a unique opportunity may be missed to disseminate effective guideline implementation tools and strategies. Examples of how guideline recommendations should be implemented and not just what to implement, can go a long way in ensuring that hospitals and services learn from the best performing units around the world rather than each hospital trying to tackle the challenges of guideline implementation (of which there are many). Furthermore, a guideline that is able to consider the human and material resources required to implement its recommendations and how shortfalls in such resources could be overcome would undoubtedly be of greater value than one which simply lists recommendations.

There are several limitations to our study, most of which are inherent to the limitations of the AGREE II checklist itself. Whilst the AGREE II tool is a validated and universally accepted appraisal tool for the assessment of guidelines and is thought to be comprehensive in its six-domain assessment, it does have some restrictions. For example, AGREE II gives equal weighting to all of its Domains, implying that they are all equally as important as each other in formulating an overall impression of a guideline. However, this may not necessarily be the case. A 2017 study demonstrated that most assessors are heavily influenced by how guidelines perform in domain III (Rigour of Development) and domain V (Applicability) (31). In short, any guideline that scores well in these two areas is likely to have a favourable overall score. Arguably this phenomenon raises the question as to whether any future iteration of the AGREE checklist should have weighted rather than equivocal domains.

In any study such as this, there is also a concern for potential bias across the assessors. Although the assessors were all blinded to one another’s answers, loyalty to home/regional societies or familiarity with certain guidelines could have swayed an assessor’s score. Furthermore, despite the AGREE II website going to painstaking lengths to provide explanations as to what would constitute a certain grade/mark in each section, there is an element of subjectivity and interpretation that is hard to eliminate. In our study, the rate of concordance between the different assessors was high across the board and no particular Domain was highlighted as at ‘High Risk’ of discrepancy.

Conclusions

In line with previously published reports, guidelines in haemorrhoid surgery are generally good at outlining who they have formulated their recommendations for and what actions should be undertaken for particular clinical scenarios. Unfortunately, they also suffer from the same problems that many other guidelines experience and that is, they have been largely unable to instruct clinicians as to how they should implement these recommendations. The opinions of patients or their advocates have also sadly been lacking from the majority of the guidelines, highlighting a missed opportunity to elicit opinions from patients themselves as to what constitutes excellent care. During the updating of these guidelines, particular attention must be paid to certain Domains and the AGREE II checklist should be used as a template on how to improve future guidelines.

Acknowledgments

Funding: None.

Footnote

Reporting Checklist: The authors have completed the PRISMA reporting checklist. Available at https://atm.amegroups.com/article/view/10.21037/atm-22-4255/rc

Peer Review File: available at https://atm.amegroups.com/article/view/10.21037/atm-22-4255/prf

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://atm.amegroups.com/article/view/10.21037/atm-22-4255/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Gallo G, Sacco R, Sammarco G. Epidemiology of hemorrhoidal disease. In: Ratto C, Parello A, Litta F. editors. Hemorrhoids Coloproctology. vol 2. Springer, Cham; 2018:3-7.
2. Lohsiriwat V. Hemorrhoids: from basic pathophysiology to clinical management. World J Gastroenterol 2012;18:2009-17. [Crossref] [PubMed]
3. Riss S, Weiser FA, Schwameis K, et al. The prevalence of hemorrhoids in adults. Int J Colorectal Dis 2012;27:215-20. [Crossref] [PubMed]
4. Johanson JF, Sonnenberg A. The prevalence of hemorrhoids and chronic constipation. An epidemiologic study. Gastroenterology 1990;98:380-6. [Crossref] [PubMed]
5. Idrees JJ, Clapp M, Brady JT, et al. Evaluating the Accuracy of Hemorrhoids: Comparison Among Specialties and Symptoms. Dis Colon Rectum 2019;62:867-71. [Crossref] [PubMed]
6. Sengupta N, Tapper EB, Feuerstein JD. Early Versus Delayed Colonoscopy in Hospitalized Patients With Lower Gastrointestinal Bleeding: A Meta-Analysis. J Clin Gastroenterol 2017;51:352-9. [Crossref] [PubMed]
7. Gralnek IM, Neeman Z, Strate LL. Acute Lower Gastrointestinal Bleeding. N Engl J Med 2017;376:1054-63. [Crossref] [PubMed]
8. Aoki T, Hirata Y, Yamada A, et al. Initial management for acute lower gastrointestinal bleeding. World J Gastroenterol 2019;25:69-84. [Crossref] [PubMed]
9. Goligher JC. Surgery of the anus rectum and colon. In: Goligher JC, Duthie HL, Nixon H. editors. Injuries of the Rectum and Colon. 5th Ed. London: Bailliere Tindall; 1984:1133-4.
10. Gallo G, Martellucci J, Sturiale A, et al. Consensus statement of the Italian society of colorectal surgery (SICCR): management and treatment of hemorrhoidal disease. Tech Coloproctol 2020;24:145-64. [Crossref] [PubMed]
11. Nyström PO, Qvist N, Raahave D, et al. Randomized clinical trial of symptom control after stapled anopexy or diathermy excision for haemorrhoid prolapse. Br J Surg 2010;97:167-76. [Crossref] [PubMed]
12. Lee MJ, Morgan J, Watson AJM, et al. A validated severity score for haemorrhoids as an essential prerequisite for future haemorrhoid trials. Tech Coloproctol 2019;23:33-41. [Crossref] [PubMed]
13. Rørvik HD, Styr K, Ilum L, et al. Hemorrhoidal Disease Symptom Score and Short Health ScaleHD: New Tools to Evaluate Symptoms and Health-Related Quality of Life in Hemorrhoidal Disease. Dis Colon Rectum 2019;62:333-42. [Crossref] [PubMed]
14. Elbetti C, Giani I, Novelli E, et al. The single pile classification: a new tool for the classification of haemorrhoidal disease and the comparison of treatment results. Updates Surg 2015;67:421-6. [Crossref] [PubMed]
15. Gaj F, Trecca A. New "PATE 2006" system for classifying hemorrhoidal disease: advantages resulting from revision of "PATE 2000 Sorrento". Chir Ital 2007;59:521-6. [PubMed]
16. Jin JZ, Bhat S, Lee KT, et al. Interventional treatments for prolapsing haemorrhoids: network meta-analysis. BJS Open 2021;5:zrab091. [Crossref] [PubMed]
17. Brouwers MC, Kho ME, Browman GP, et al. AGREE II: advancing guideline development, reporting, and evaluation in health care. Prev Med 2010;51:421-4. [Crossref] [PubMed]
18. Alonso-Coello P, Irfan A, Solà I, et al. The quality of clinical practice guidelines over the last two decades: a systematic review of guideline appraisal studies. Qual Saf Health Care 2010;19:e58. [PubMed]
19. Appraisal of Guidelines for Research and Evaluation II2013. Available online: https://www.agreetrust.org/
20. Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009;6:e1000097. [Crossref] [PubMed]
21. Gavriilidis P, Askari A, Gavriilidis E, et al. Appraisal of the current guidelines for the management of diverticular disease using the Appraisal of Guidelines Research and Evaluation II (AGREE II) instrument. Ann R Coll Surg Engl 2021;103:471-7. [Crossref] [PubMed]
22. Gavriilidis P, Askari A, de'Angelis N, et al. Appraisal of the Current Guidelines for Management of Malignant Left-Sided Colonic Obstruction Using the Appraisal of Guidelines Research and Evaluation II Instrument. Dig Surg 2021;38:177-85. [Crossref] [PubMed]
23. Agarwal N, Singh K, Sheikh P, et al. Executive Summary - The Association of Colon & Rectal Surgeons of India (ACRSI) Practice Guidelines for the Management of Haemorrhoids-2016. Indian J Surg 2017;79:58-61. [Crossref] [PubMed]
24. Davis BR, Lee-Kong SA, Migaly J, et al. The American Society of Colon and Rectal Surgeons Clinical Practice Guidelines for the Management of Hemorrhoids. Dis Colon Rectum 2018;61:284-92. [Crossref] [PubMed]
25. van Tol RR, Kleijnen J, Watson AJM, et al. European Society of ColoProctology: guideline for haemorrhoidal disease. Colorectal Dis 2020;22:650-62. [Crossref] [PubMed]
26. Yamana T. Japanese Practice Guidelines for Anal Disorders I. Hemorrhoids. J Anus Rectum Colon 2017;1:89-99. [Crossref] [PubMed]
27. Altomare DF, Roveran A, Pecorella G, Gaj F, Stortini E. The treatment of hemorrhoids: guidelines of the Italian Society of Colorectal Surgery. Tech Coloproctol 2006;10:181-6. [Crossref] [PubMed]
28. Salgueiro P, Caetano AC, Oliveira AM, et al. Portuguese Society of Gastroenterology Consensus on the Diagnosis and Management of Hemorrhoidal Disease. GE Port J Gastroenterol 2020;27:90-102. [Crossref] [PubMed]
29. Shallwani SM, King J, Thomas R, et al. Methodological quality of clinical practice guidelines with physical activity recommendations for people diagnosed with cancer: A systematic critical appraisal using the AGREE II tool. PLoS One 2019;14:e0214846. [Crossref] [PubMed]
30. Gavriilidis P, Askari A, Roberts KJ, et al. Appraisal of the current guidelines for management of cholangiocarcinoma-using the Appraisal of Guidelines Research and Evaluation II (AGREE II) Instrument. Hepatobiliary Surg Nutr 2020;9:126-35. [Crossref] [PubMed]
31. Hoffmann-Eßer W, Siering U, Neugebauer EA, et al. Guideline appraisal with AGREE II: Systematic review of the current evidence on how users handle the 2 overall assessments. PLoS One 2017;12:e0174831. [Crossref] [PubMed]