A New Formative Assessment for Indonesian Medical Records Education: Validating DOPSAT-RSP
DOI:
https://doi.org/10.47701/cqnhga62Keywords:
Direct Observation of Procedural Skills, Educational Measurement, Workplace-based assessment, Health Records, Clinical CompetenceAbstract
The assessment of practical skills among Associate Degree or Diploma 3 students of Medical Records and Health Information (MRHI) during field practice (PKL) lacks a standardized workplace-based assessment (WPBA) instrument, resulting in variations across institutions and potential subjectivity in evaluation. While Direct Observation of Procedural Skills (DOPS) is effective in clinical assessment but not fully suited to the non-clinical context of MRHI practice. This study aimed to modify DOPS into a formative assessment tool for MRHI field practice and to examine its validity and reliability. A research and development design was applied. The modified instrument, DOPSAT-RSP, underwent content validity testing by ten experts using the Content Validity Index (CVI) and Content Validity Ratio (CVR). Reliability testing included inter-rater reliability using Weighted Cohen’s Kappa and internal consistency using Cronbach’s alpha, involving two raters and 39 students. All items met content validity criteria (I-CVI ?0.900; CVR ?0.800; S-CVI =0.950). The Weighted Kappa value (0.774) indicated strong inter-rater agreement, while Cronbach’s alpha (0.952) demonstrated excellent internal consistency. The DOPSAT-RSP instrument is valid and reliable for formative assessment of D-3 RMIK students’ practical skills during PKL and shows potential for wider implementation with user training.
Downloads
References
1. Nuryati, Apriliana TA. Perancangan Kurikulum Pendidikan Tinggi Program Studi Rekam Medis dan Informasi Kesehatan. In: Standar Akreditasi Rumah Sakit (SNARS) Edisi 1 Terkait Rekam Medis. 2020. p. 28–33.
2. Avupati VR. Direct Observation of Procedural Skills ( DOPS ): A Workplace-Based Assessment ( WPBA ) In Health Professions Education and Practice. Am J Biomed Sci Res. 2024;24(4):419–24.
3. Immonen K, Oikarainen A, Tomietto M, Kääriäinen M, Tuomikoski A-M, Kaučič BM, et al. Assessment of nursing students’ competence in clinical practice: A systematic review of reviews. Int J Nurs Stud. 2019;100:103414.
4. Yaqoob Mohammed Al Jabri F, Kvist T, Azimirad M, Turunen H. A systematic review of healthcare professionals’ core competency instruments. Nurs Health Sci. 2021;23(1):87–102.
5. Jalili M, Imanipour M, Nayeri ND. Evaluation of the Nursing Students’ Skills by DOPS. J Med Educ. 2015;14(1):13–9.
6. So HY, Ic F, Choi YF, Chan PT, Chan AKM, Ng GWY, et al. Workplace-based assessments : what , why , and how to implement ? Hong Kong Med J ©2024. 2024;30(3):250–4.
7. Anderson HL, Kurtz J, West DC. Implementation and use of workplace-based assessment in clinical learning environments: a scoping review. Acad Med. 2021;96(11S):S164–74.
8. Castanelli DJ, Weller JM, Molloy E, Bearman M. Trust, power and learning in workplace‐based assessment: The trainee perspective. Med Educ. 2022;56(3):280–91.
9. Zhang W ying, Peng Y, Zhuang H ren, Yu H ping, Liu Q, Gu Y, et al. Application of direct observation of operational skills in nursing skill evaluation of pressure injury: A randomized clinical trial. Int Wound J. 2024;21(3):1–10.
10. Aryal K, Hamed M, Currow C. The usefulness of work-based assessments in higher surgical training: A systematic review. Int J Surg [Internet]. 2021;94(September):106127. Available from: https://doi.org/10.1016/j.ijsu.2021.106127
11. Lagoo JY, Joshi SB. Introduction of direct observation of procedural skills (DOPS) as a formative assessment tool during postgraduate training in anaesthesiology: Exploration of perceptions. Indian J Anaesth. 2021;65(3):202–9.
12. Chen M. The Direct Observation of Procedural Skills ( DOPS ) Validity and Reliability Set for Occupational Therapy of Splint Fabrication. Chung Shan Med J. 2024;35(November):93–100.
13. Anderson DD, Long S, Thomas GW, Putnam MD, Bechtold JE, Karam MD. Objective structured assessments of technical skills (OSATS) does not assess the quality of the surgical result effectively. Clin Orthop Relat Res. 2016;474(4):874–81.
14. Andreou V, Peters S, Eggermont J, Schoenmakers B. A needs assessment for enhancing workplace-based assessment: a grounded theory study. BMC Med Educ. 2024;24(1):1–7.
15. Vanteemar S. Sreeraj S, Uvais2 NA, Mohanty3 S, Kumar3 S, Department. Workplace based assessment: A review of available tools and their relevance. Ind Psychiatry J. 2019;195–201.
16. Mathur P. Introduction of direct observation of procedural skills as workplace-based assessment tool in department of anesthesiology: Evaluation of students’ and teachers’ perceptions. Bali J Anesthesiol. 2021;5(4):239–45.
17. Ahmad R, Hena A, Uddin H, Hossin I, Ahmed AKMK, Hasan K, et al. Direct Observation of Procedural Skills ( DOPS ) in Enhancing Clinical Competencies of Postgraduate Medical and Dental Education : A Scoping Review. 2024;23(9):59–70.
18. Ebrahimpour-Sadagheyani H, Saadati SM, Tatari F. Investigating the Impact of the Direct Observation of Procedural Skills Assessment on the Learning and Satisfaction of the Operating Room Students of Neyshabur University of Medical Sciences, Iran, in 2022. Iran J Nurs Midwifery Res. 2025;30(1):110–5.
19. Nayyeri S, Bozorgvar A, Barzanouni S, Masoumain Hosseini T. Effect of Clinical Skills Evaluation by Direct Observation of Procedural Skills (DOPS) Method on Clinical Performance of Operating Room Students. J Med Educ. 2021;20(3).
20. Hasen Y, Mequanint A, Teshome T, Belete A, Bekele D. An action research to introduce direct observation of procedural skills (DOPS) as clinical skill performance assessment tool during medical internship in the department of surgery, Debre Tabor University, Debre Tabor, Ethiopia, 2023. African Res J Med Sci. 2024;1(1):42–50.
21. Shahid Z, Fatima K, Khan Y, Agha MF, Fatima U. Impact of Mini-CEX and DOPS on clinical competence and satisfaction among final year medical students: a comparative interventional study. Khyber Med Univ J. 2024;16(2):97–102.
22. Jafarpoor H, Hosseini M, Sohrabi M, Mehmannavazan M. The effect of direct observation of procedural skills/mini‑clinical evaluation exercise on the satisfaction and clinical skills of nursing students in dialysis. J Educ Health Promot. 2021;10(74):1–6.
23. Moore K, Vaughan B, Butterworth P. Adaptation of Direct Observation of Procedural Skills (DOPS) for assessments in podiatry. Focus Heal Prof Educ A Multi-Professional J. 2018;19(1):65.
24. Kemenkes RI. Keputusan Menteri Kesehatan Republik Indonesia No. 312 tahun 2020 tentang Standar Profesi Perekam Medis dan Informasi Kesehatan. Indonesia: Kementerian Kesehatan RI; 2020.
25. Kemenkes RI. Keputusan Menteri Kesehatan Republik Indonesia Nomor HK.01.07/Menkes/1424/2022 tentang Standar Kompetensi Kerja Bidang Rekam Medis dan Informasi Kesehatan. Indonesia; 2022.
26. Lynn MR. Determination and Quantification of Content Validity. Nurs Res. 1986;35(6):382–6.
27. Bujang MA, Baharum N. Guidelines of the minimum sample size requirements for Cohen’s Kappa. Epidemiol Biostat Public Heal. 2017;14(2):e12267-1-e12267-10.
28. Nuti L, Turkcan A, Lawley MA, Zhang L, Sands L, McComb S. The impact of interventions on appointment and clinical outcomes for individuals with diabetes: a systematic review. BMC Health Serv Res [Internet]. 2015 Sep 2;15:355. Available from: https://search.ebscohost.com/login.aspx?direct=true&db=mnh&AN=26330299&site=ehost-live
29. Lawshe CH. A quantitative approach to content validity. Pers Psychol. 1975;28(4).
30. Polit DF, Beck CT. The content validity index: are you sure you know what’s being reported? Critique and recommendations. Res Nurs Health. 2006;29(5):489–97.
31. Yuniarto D, Suryadi M, Firmansyah E, Herdiana D, Subiyakto A, Rahman ABA. Integrating the Readiness and Usability Models for Assessing the Information System Use. 2018 6th Int Conf Cyber IT Serv Manag CITSM 2018. 2019;(Citsm):5–10.
32. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977 Mar;33(1):159–74.
33. Amini A, Shirzad F, Mohseni MA, Sadeghpour A, Elmi A. Designing Direct Observation of Procedural Skills (DOPS) Test for Selective Skills of Orthopedic Residents and Evaluating Its Effects from Their Points of View. Res Dev Med Educ [Internet]. 2016;4(2):147–52. Available from: http://dx.doi.org/10.15171/rdme.2015.026
34. Shafiq Z, Ullah H, Sultana F. Challenges in implementing direct observation of procedural skills as assessment tool in postgraduate training of general surgery. Ann Pak Inst Med Sci [Internet]. 2019;15(2):66–70. Available from: http://www.pakmedinet.com/39035
35. Roebianto A, Savitri SI, Aulia I, Suciyana A, Mubarokah L. Content Validity: Definition and Procedure of Content Validation in Psychological Research. TPM - Testing, Psychom Methodol Appl Psychol. 2023;30(1):5–18.
36. Mehranfard S, Pelarak F, Mashalchi H, Kalani L, Masoudiyekta L. Efficacy of logbook as a clinical assessment: Using DOPS evaluation method. J Multidiscip Care [Internet]. 2022;11(4):184–9. Available from: https://doi.org/10.34172/jmdc.2022.62
37. Yuan Y, Wang C, Wen S, Li Y, Xu C, Yu F, et al. Pilot Study of a Modified DOPS Scale for Insulin Pump and CGM Installation Training in Chinese Medical Students During Endocrinology Rotations. Diabetes, Metab Syndr Obes . 2025;18(January):37–50.
38. Khanghahi ME, Azar FEF. Direct observation of procedural skills (DOPS) evaluation method: Systematic review of evidence. Med J Islam Repub Iran. 2018;32(1):254–61.
39. Sadeghi-Bazargani H, Farahbakhsh M, Tabrizi JS, Zare Z, Saadati M. Psychometric properties of primary health care trust questionnaire. BMC Health Serv Res. 2019;19(1):1–5.
40. McHugh ML. Interrater reliability: the kappa statistic. Biochem medica. 2012;22(3):276–82.
41. Chen M. The Direct Observation of Procedural Skills ( DOPS ) Validity and Reliability Set for Occupational Therapy of Splint Fabrication. Chung Shan Med J 2024; 2024;35:93–100.
42. Liu W, Ding J, Zhang X, Ma X, Yang J, Zhou P, Shen X. Workplace-Based Assessment in Emergency Surgical Education Improves the Clinical Core Competency of Residents. J Surg Educ [Internet]. 2025;82(10):103644. Available from: https://doi.org/10.1016/j.jsurg.2025.103644
43. Baboolal SO, Singaram V. The Use, Effectiveness, and Impact of Workplace-Based Assessments on Teaching, Supervision and Feedback Across Surgical Specialties. J Surg Educ [Internet]. 2023;80(8):1158–1171. Available from: https://doi.org/10.1016/j.jsurg.2023.05.012
44. Wang, X., & Cheng, Z. (2020). Cross-sectional studies: Strengths, weaknesses, and recommendations. Chest, 158(1, Suppl), S65–S71. https://doi.org/10.1016/j.chest.2020.03.012
45. Tay YX, Lau DS, Chow HC, Heng I, Yeo KCW, Yusof MKB, Chong SH, Lee W, Wei YM, Abu Bakar RB. Workplace-based assessment – Moving from participation to engagement using direct observation of procedural skills (DOPS). J Med Imaging Radiat Sci [Internet]. 2023;54(2):229–34. Available from: https://doi.org/10.1016/j.jmir.2023.02.001
46. Datta A, Nath A. Effect of directly observed procedural skills amongst interns in documentation and certification of trauma. Med J Armed Forces India [Internet]. 2025. Available from: https://doi.org/10.1016/j.mjafi.2025.03.004
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Untoro Dwi Raharjo, Rizky Yuspita Sari, Resia Perwirani, Jannatul Iza Ahmad Kamal
This work is licensed under a Creative Commons Attribution 4.0 International License.
