Ophthalmology foundation Ophthalmic Surgical Competency Assessment Rubric (OSCAR) for pterygium surgery
Highlight box
Key findings
• An internationally validated tool for teaching and assessing pterygium surgery has been created.
What is known and what is new?
• Pterygia are a very common eye condition that often require surgical intervention. All ophthalmologists are trained to perform this surgery. International standards for teaching and assessing this procedure do not exist.
• This report details the development of an internationally standardized tool to facilitate teaching and assessing of the commonly performed procedure.
What is the implication, and what should change now?
• Ophthalmic educators can use this ophthalmology surgical competency assessment rubric (OSCAR) to facilitate their teaching and to help assure competency in pterygium surgery.
Introduction
We have witnessed graduate medical education around the world evolving from an apprenticeship model to a competency-based model on the recommendations of external sources including public and professional regulatory bodies and the general public. Teaching and assessing tools are needed to assure competence in trainees. Medical knowledge has traditionally been assessed by multiple choice question (MCQ) and short answer question (SAQ) tests. Objective structured clinical examination (OSCE) and direct observation can be used for clinical skills. Teaching and assessment of surgical ability has been one of the most challenging tasks for the trainers. Demonstration of competence by the resident in procedural skills should be verified in the operating room under real life situations. This requires substantial human resources to assess all residents in the training program. In order to overcome this hurdle a valid and reliable rubric has been shown to be a very effective tool (1). A variety of internationally validated rubrics, Ophthalmology Surgical Competency Assessment Rubrics (OSCARs), have been developed (2-12).
Pterygium is a common ocular pathology with a reported prevalence of 10.2% (13). In a large epidemiological study from India, pterygium was found to be responsible for one-third of all cases of corneal opacities (14). Thus, it is a frequently performed surgery by ophthalmology residents. Pterygium surgery has evolved over time and now includes many techniques to prevent recurrence after better understanding of the pathophysiology of the disease (15). Although it has been thought to be a minor surgery, many intricate steps are required to achieve good results. The Ophthalmology Foundation’s Mission is to work with eye care professionals, societies and organizations to enhance and provide ophthalmic education, with special focus on low-resource and underserved countries. We commit to strengthening ophthalmic education, with the ultimate goal of improving eyecare and advancing the preservation and restoration of vision for all. To achieve this mission, we organized a group of authors and panelists to develop a rubric with content and face validity to teach and assess the competence of ophthalmology residents in pterygium surgery.
Methods
A panel of 5 specialists consisting of members from different regions of the world including Pakistan, United States, Iran, Mongolia and Peru was formed to create a rubric for teaching and assessment of pterygium surgery by ophthalmology residents. The rubric was designed in concordance with previously published OSCARs (2). The rubric consisted of a four-point rating scale using a modified Dreyfus model of skill acquisition (16). As this rubric was designed to assess competence of residents, the expert skill level was excluded. Description of the required skill for each level of competence was included to decrease rater subjectivity.
The rubric draft was then sent to an international faculty of 10 cornea specialists from around the World (China, Peru, Egypt, Australia, Canada, United States, and the United Kingdom). The panel of experts included consultants with at least 10 years of experience in teaching and performing pterygium surgery. The authors reviewed comments from all reviewers and appropriate changes were made in the rubric to improve validity. Panel suggestions included amendments in steps of surgery and improvements in the skill descriptions.
No statistical analysis was needed for this type of study.
Results
The authors refined the content and face validity of the rubric by reviewing the panel of experts comments and making modifications. Googledocs was used to allow continuous commenting, revision and discussion by the authors until the authors agreed on a final rubric. Consensus was made to assess this surgery using 13 essential steps and 5 global indices (Table 1). Each step was divided into four sections to grade the behavioral responses of the surgeon. These included novice, beginner, advanced beginner and competent. The minimal skills required for each level as decided by the authors and panel of experts were defined for each level of each surgical step. As this rubric was only designed to check surgical competence no steps were included for complications. Global indices were added to check the handling of tissues, use of microscope and suturing techniques.
Table 1
| No. | Components of procedure | Novice (score =2) | Beginner (score =3) | Advanced beginner (score =4) | Competent (score =5) | Not applicable. Done by preceptor (score =0) |
|---|---|---|---|---|---|---|
| Essential steps | ||||||
| 1 | Placement of speculum and draping | Difficult to place the speculum and application of drape. Requires assistance. Leaves most of the lashes exposed | Places the speculum and applies drape with minimal verbal instruction. Incomplete lash coverage | Places the speculum and applies drape without difficulty with minimally obstructing view. Lashes mostly covered | Places the speculum very easily and applies drape not obstructing view. Lashes completely covered and clear of incision site | |
| 2 | Limbal/corneal traction suture (if applicable) | Unable to put limbal/corneal traction suture without help Superficial sutures leading to cheese-wire. Anterior chamber penetration with corneal traction suture |
Limbal/corneal traction suture with minimal verbal instruction but some mistakes (i.e., small bite) | Limbal/corneal traction suture of adequate depth & horizontal bite without help | Limbal/corneal traction suture without complications and with appropriate technique | |
| 3 | Marking the site of conjunctival graft incision with marker | Unable to mark the conjunctiva with calipers or does not check the caliper setting to confirm planned action. Does not understand the discrepancy between the size of defect and of conjunctival flap/graft | Able to mark the conjunctiva with calipers but the shape does not fit defects. Understands the discrepancy between the size of defect and of conjunctival flap/graft | Able to accurately mark the conjunctiva with calipers but marks fade because not well prepared. Understands the discrepancy between the size of defect and of conjunctival flap/graft | Able to efficiently and accurately mark the conjunctiva with calipers. Understands the discrepancy between the size of defects and of conjunctival flap/graft. (Takes a graft larger than the defect) | |
| 4 | Conjunctival graft dissection using sub-conjunctival fluid injection (if applicable) |
Conjunctival perforation during needle insertion. Unable to insert the needle at the right place leading to dissection of deep conjunctival stroma or Tenon’s capsule | Appropriate sub-conjunctival fluid injection. Conjunctival dissection without perforation using scissors but includes deep conjunctival stroma or Tenon’s capsule | Appropriate sub-conjunctival fluid injection Prepares an intact thin conjunctival flap/graft with some verbal help | Appropriate sub-conjunctival fluid injection. Prepares an intact thin conjunctival flap/graft without help | |
| 5 | Conjunctival resection and dissection of the pterygium body | Cuts with difficulty without correct dissection, leaves ragged edges, tears tissues. Oversized conjunctival resection or too small leaving without removing damaged tissue | Cuts with moderate difficulty, leaves ragged edges. Conjunctival resection still of inadequate size | Cuts with minimal difficulty, leaves regular edges. Proper size of the conjunctival resection | Cuts very easily dissecting smoothly. Proper size of conjunctival resection and complete removal of the pterygium body | |
| 6 | Excision of pterygium head. | Performs partial removal with severe difficulty, dissects pterygium at a wrong plane | Pterygium head almost completely removed. With moderate difficulty. Not confident with the surgical technique | Pterygium head completely removed with little difficulty. Confident with the surgical technique | Pterygium head completely removed without difficulty. Very confident surgical technique | |
| 7 | Superficial keratectomy | Not aware of technique. Incomplete abnormal tissue removal, leaves debris on the cornea and severe irregular surface. Deep stromal damage due to vigorous keratectomy | Aware of technique Completely removes abnormal tissues but retains some superficial scars, leaving some remains in the cornea. Mildly irregular surface |
Technique—near perfect. Minimal to nil bed irregularity or deep stromal damage with minimal residue on the cornea | Completely removes abnormal tissues. Creates a smooth and regular bed with minimal residual opacity | |
| 8 | Dissection of Tenon’s capsule and removal of a 2 mm strip from the conjunctival edge | Is not able to establish difference between the conjunctiva and Tenon´s capsule, and dissects very deeply, distends and break tissues | Has moderate difficulty to establish difference between conjunctiva and Tenon’s capsule. Damage to conjunctiva, Tenon’s removal may be insufficient or excessive | Dissects Tenon’s capsule properly with minimal difficulty. Tenon’s removal is enough | Dissects and removes Tenon’s capsule easily, in enough quantity without injuring the conjunctiva or other adjacent structure. | |
| 9 | Scleral shaving and cauterization | Unable to successfully shave sclera underlying pterygium. Cauterization insufficient or excessive both in intensity and localization | Shaves underlying pterygium with difficulty and hesitation. Cauterization slightly insufficient or excessive both in intensity and localization |
Achieves good scleral smoothness. Adequate cauterization in intensity & localization | Precisely and deftly prepares sclera. Appropriate and precise cauterization | |
| 10 | Mitomycin-C application (if applicable) | Not aware of indication, concentration of the drug, or duration, method and site of application (without taking care of adjacent structures, does not record time and performs insufficient washout) | Aware of indication. Poor management of drug concentrations or duration, method and site of application (no time to register, no care with adjacent structures and performs moderate washout) | Aware of indication. Some degree of difficulty in drug preparation, or duration, method and site of application (records the time, takes care of adjacent structures, and profuse washout) | Aware of indication. Good management of drug concentrations, and duration, method and site of application (records time, special care with adjacent structures and profuse washout) | |
| 11 | Conjunctival flap/graft preparation and dissection | Not sure of size & location of conjunctival excision Perforates conjunctiva during dissection, Includes Tenon’s capsule in the conjunctival flap/graft. Reverses the graft placing the epithelium down | Size of conjunctival flap/graft is barely adequate. Poor management of the instruments. Thick conjunctival flap/graft containing too much stroma, tears the conjunctiva, takes care of not to reverse the graft | Size is almost exact. Good management of the instruments Prepares a very thin conjunctival flap/graft, manipulates the conjunctiva smoothly, takes care of not reversing the graft |
Measures the graft size and obtains an adequate size. Excellent management of the instruments, dissects adequately and gets one right size and right thickness graft. Prepares a very thin conjunctival flap/graft | |
| 12 | Placing the graft in the area of pterygium excision | Places the graft over the limbus or leaves more than 2 mm uncovered sclera, does not match the limbal graft area with the limbal area of pterygium excision, places the epithelium graft down | Places the graft at 1 mm from the limbus, does not match the limbal graft area with the limbal area of pterygium excision, places the epithelium graft down | Places the graft at 1 mm from the limbus, does not match the limbal graft area with the limbal area of pterygium excision, places the epithelium graft up | Places the graft at 1 mm from the limbus, matching the limbal graft area with the limbal area of pterygium excision, places the epithelium graft up | |
| 13 | Conjunctival flap/graft suturing (with Scleral fixation) | Unable to pass episcleral suture of the flap/graft (anchoring sutures). Unable to suture the flap/graft edge to conjunctiva. Has great difficulty inserting the needle, tears tissues with forceps double 0.12 fixation teeth. Breaks suture, deforms needle | Conjunctival flap/graft suturing with minimal verbal instruction but some mistakes (i.e., very close or very far from limbus, or inversion of conjunctival edges). Moderate difficulty. Inserts the needle after some failed attempts, may damage tissue | Conjunctival flap/graft suturing without help. Inserts needle and second instrument on first attempt with mild difficulty, no damage to tissue | Conjunctival flap/graft suturing without complications and with appropriate technique. Smoothly inserts instruments without damaging the tissue. Stable wound, good apposition of the donor and host tissue | |
| Global indices | ||||||
| 14 | Tissue handling | Is excessively aggressive or timid in manipulating tissue. Inadvertent tissue damage occurs (including significant corneal epithelium disruption) | Aware of techniques for avoidance of tissue damage and bleeding but needs supervision to accomplish proper handling. Mild corneal epithelium disruption may occur | Tissue handling is safe but sometimes requires multiple attempts to achieve desired manipulation of tissue. Minimal corneal epithelium disruption may occur | Tissue handling is efficient, fluid and almost always achieves desired tissue manipulation on first attempt | |
| 15 | Eye positioned centrally within microscope view | Constantly requires repositioning | Occasional repositioning required | Mild fluctuation in pupil position | The pupil is kept centered during the surgery | |
| 16 | Technique of surgical knot tying | Unable to tie knots | Require multiple extra hand maneuvers to make first throw lay flat and/or loosens first throw while attempting to perform the second throw | Is able to tie a flat surgeon’s knot first throw but second and third throws are inefficient. Does not inadvertently loosen the first throw | Is able to efficiently tie a flat, square surgeon’s knot | |
| 17 | Handling instruments | Instruments not selected according to the procedure, holding improperly, handled roughly and with great difficulty | Instruments properly selected, holding improperly, handled with moderate difficulty | Instruments properly selected, properly held, with mildly difficulty | Instruments properly selected for each procedure, handled properly and skillfully | |
| 18 | Communication with surgical team | Does not know role of surgical team members. Lacks confidence or has too much. Does not establish good rapport with team. Unable to request instruments from scrub nurse using proper instrument and suture names and/or instructions to surgical assistant are vague or nonexistent | Knows role of most surgical team members. Lacks confidence. Has difficulty establishing good rapport with team members. Able to request most instruments from scrub nurse using proper instrument and suture names but instructions to surgical assistant are inadequate to perform procedure safely | Knows role of each surgical team member. Is somewhat confident and usually treats team with respect. Establishes good working relationship. Able to request most instruments from scrub nurse using proper instrument and suture names in correct order. Instructions to surgical assistant are adequate for a skilled assistant but inadequate for an unskilled assistant | Knows role of each surgical team member. Is confident and treats team with respect. Establishes good working relationship. Able to efficiently request instruments from scrub nurse using proper names in correct order. Able to consistently give clear instructions to surgical assistant | |
Discussion
Globally, wide variability exists in teaching and assessing ophthalmic surgical skill. A study by Gogate and associates reported the experience of ophthalmology residents in India and found significant variation in the teaching of clinical and surgical skills in residency programs across the country (17). They concluded that standardization is needed to assure all graduates are competent and render consistent quality of service. Assessment of clinical skills is equally important in a residency program and should have a positive impact on resident training (18). According to the Dreyfus model, human learning usually goes through 5 different levels starting from novice and possibly concluding with expert. Given the relatively short training period of residents, we don’t believe a resident would ever reach the expert level. In many countries, a minimum number of procedures performed by residents are required to graduate (19). However, minimum numbers do not guarantee competence.
In addition, there is variability across the globe as to the numbers of procedures and the methods used to assess level of surgical competence. The Royal College of Ophthalmologists of the UK requires that objective assessment of surgical and technical skills (OSATS) should be done by observation of the behavioral response of the resident using a 4-point rating scale (poor to very good) using a checklist of essential steps (20). This is much better than simply verbal feedback at the end of a procedure but there can be an element of bias as to what constitutes poor to very good. This can be due to different levels of knowledge and surgical ability of the assessor. To address this issue of bias and inter-rater reliability, OSCARs were developed for a variety of surgical procedures including phacoemulsification (2), small incision cataract surgery (3), extracapsular cataract surgery (2), strabismus surgery (4), pediatric cataract surgery (5), trabeculectomy (6), lateral tarsal strip (7), intravitreal injection (8), external dacryocystorhinostomy (9), pan-retinal photocoagulation (10), vitrectomy (11), and ptosis surgery (12). Each OSCAR contains precise descriptions of skills needed to achieve competence. Several OSCARs have also been shown to have good inter-rater reliability (21,22).
Importantly, OSCARs are designed to be a teaching tool as well. The trainee can simply read the “competent” column in advance to learn what is required at each surgical step. At the conclusion of each case, the rubric is discussed with the residents to provide timely formative feedback in surgical skills that require improvement. This provides the trainee with targeted learning goals and they can then concentrate on the steps in which improvement is needed.
The Ophthalmology Foundation determined that pterygium surgery was a very commonly performed procedure and thus an OSCAR should be developed. Specific domains for pterygium surgery include placement of speculum, draping, limbal/corneal traction suture, marking the site of conjunctival graft incision with a marker, conjunctival graft dissection using sub conjunctival fluid injection, conjunctival resection and dissection of the pterygium body, excision of the pterygium head, superficial keratectomy, dissection of tenon’s capsule and removal of 2 mm strip from the conjunctival edge, scleral shaving and cauterization, mitomycin C 0.02% application, conjunctival flap/graft preparation and dissection, placing the graft in the area of pterygium excision and conjunctival flap/graft preparation and dissection, placing the graft in the area of pterygium excision and conjunctival flap/graft suturing with scleral fixation (Table 1). The scoring levels are cumulative; if one achieves a certain level, they would have achieved the preceding lower levels.
As we developed the OSCAR for pterygium, another group published a pterygium surgery rubric (23). Their rubric includes 13 stages of pterygium surgery which are to be assessed. The tool has been developed for conjunctival autograft and amniotic membrane transplantation. It has been further validated for good inter-rater reliability by rating of 20 masked surgical videos of pterygium surgery of 10 residents by 2 independent surgeons. Our rubric is different in that it also assesses general surgical domains in addition to the specific steps of surgery. Moreover, our rubric was created by an international panel of experts to try and assure applicabilty around the world.
Conclusions
Surgical procedures expected to be done by graduating trainees must be taught and the competence of graduates must be assessed. The OSCAR tool for pterygium is an internationally applicable method of teaching and assessing competence in this procedure. Although the OSCAR pterygium has face and content validity further studies are needed to determine the inter-rater reliability.
Acknowledgments
Funding: None.
Footnote
Peer Review File: Available at https://atm.amegroups.com/article/view/10.21037/atm-22-3075/prf
Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://atm.amegroups.com/article/view/10.21037/atm-22-3075/coif). The authors have no conflicts of interest to declare.
Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.
References
- Jonsson A, Svingby G. The use of scoring rubrics: reliability, validity and educational consequences. Educ Res Rev 2007;2:130-44.
- Golnik KC, Beaver H, Gauba V, et al. Cataract surgical skill assessment. Ophthalmology 2011;118:427.e1-5. [Crossref] [PubMed]
- Golnik KC, Haripriya A, Beaver H, et al. Cataract surgery skill assessment. Ophthalmology 2011;118:2094.e2. [Crossref] [PubMed]
- Golnik KC, Motley WW, Atilla H, et al. The ophthalmology surgical competency assessment rubric for strabismus surgery. J AAPOS 2012;16:318-21. [Crossref] [PubMed]
- Swaminathan M, Ramasubramanian S, Pilling R, et al. ICO-OSCAR for pediatric cataract surgical skill assessment. J AAPOS 2016;20:364-5. [Crossref] [PubMed]
- Green CM, Salim S, Edward DP, et al. The Ophthalmology Surgical Competency Assessment Rubric for Trabeculectomy. J Glaucoma 2017;26:805-9. [Crossref] [PubMed]
- Golnik KC, Gauba V, Saleh GM, et al. The ophthalmology surgical competency assessment rubric for lateral tarsal strip surgery. Ophthalmic Plast Reconstr Surg 2012;28:350-4. [Crossref] [PubMed]
- Kiew SY, Yeo IYS, Golnik KC, et al. The Ophthalmology Surgical Competency Assessment Rubric for Intravitreal Injections (ICO-OSCAR:IVI). J Clin Med 2021;10:1476. [Crossref] [PubMed]
- Juniat V, Saleh GM, Naik M, et al. The Ophthalmology Surgical Competency Assessment Rubric for External Dacryocystorhinostomy Surgery. Ophthalmic Plast Reconstr Surg 2021;37:S11-8. [Crossref] [PubMed]
- Law JC, Golnik KC, Cherney EF, et al. The Ophthalmology Surgical Competency Assessment Rubric for Panretinal Photocoagulation. Ophthalmol Retina 2018;2:162-7. [Crossref] [PubMed]
- Golnik KC, Law JC, Ramasamy K, et al. The Ophthalmology Surgical Competency Assessment Rubric for Vitrectomy. Retina 2017;37:1797-804. [Crossref] [PubMed]
- Juniat V, Golnik KC, Bernardini FP, et al. The Ophthalmology Surgical Competency Assessment Rubric (OSCAR) for anterior approach ptosis surgery. Orbit 2018;37:401-4. [Crossref] [PubMed]
- Liu L, Wu J, Geng J, et al. Geographical prevalence and risk factors for pterygium: a systematic review and meta-analysis. BMJ Open 2013;3:e003787. [Crossref] [PubMed]
- Gupta N, Vashist P, Tandon R, et al. Prevalence of corneal diseases in the rural Indian population: the Corneal Opacity Rural Epidemiological (CORE) study. Br J Ophthalmol 2015;99:147-52. [Crossref] [PubMed]
- Graue-Hernandez EO, Córdoba A, Jimenez-Corona A, et al. Practice Patterns in the Management of Primary Pterygium: A Survey Study. Cornea 2019;38:1339-44. [Crossref] [PubMed]
- Dreyfus S. The five-stage model of adult skill acquisition. Bull Sci Technol Soc 2004;24:177-81.
- Gogate P, Biswas P, Natarajan S, et al. Residency evaluation and adherence design study: Young ophthalmologists' perception of their residency programs - Clinical and surgical skills. Indian J Ophthalmol 2017;65:452-60. [Crossref] [PubMed]
- Paley GL, Shute TS, Davis GK, et al. Assessing Progression of Resident Proficiency during Ophthalmology Residency Training: Utility of Serial Clinical Skill Evaluations. J Med Educ Train 2017.
- Baugh TP, Franzese CB. Extremes in Otolaryngology Resident Surgical Case Numbers: An Update. Otolaryngol Head Neck Surg 2017;156:1060-6. [Crossref] [PubMed]
- Objective Assessment of Surgical and Technical Skills (OSATS) - The Royal College Of Ophthalmologists. [online] The Royal College of Ophthalmologists. [Accessed 12 June 2022]. Available online: https://www.rcophth.ac.uk/curriculum/ost/assessments/workplace-based-assessments/objective-assessment-of-surgical-and-technical-skills-osats/
- Golnik C, Beaver H, Gauba V, et al. Development of a new valid, reliable, and internationally applicable assessment tool of residents' competence in ophthalmic surgery (an American Ophthalmological Society thesis). Trans Am Ophthalmol Soc 2013;111:24-33.
- Motley WW 3rd, Golnik KC, Anteby I, et al. Validity of ophthalmology surgical competency assessment rubric for strabismus surgery in resident training. J AAPOS 2016;20:184-5. [Crossref] [PubMed]
- Zarei-Ghanavati M, Ghassemi H, Salabati M, et al. A surgical skills assessment rubric for pterygium surgery. Ocul Surf 2020;18:494-8. [Crossref] [PubMed]
