Department of Anatomy,Ibrahim Medical College,1/A Segunbagicha, Dhaka, Bangladesh
Abstract
Background and objective: Short answer question (SAQ) format has been introduced as a major component of summative professional examinations of MBBS (Bachelor of Medicine and Surgery) course in Bangladesh over a decade. No systematic evaluation has yet been conducted on implementation of SAQ as directed in curriculum to assess the medical students in the summative examination of MBBS course. The present study assessed the weightage given to the different components of cognitive domain in SAQs in anatomy in first Professional MBBS Examination under the University of Dhaka.
Materials and method: This cross-sectional study was conducted in the Department of Anatomy, Ibrahim Medical College. Anatomy SAQ papers, Paper I and Paper II, from January 2009 to July 2014 of University of Dhaka were selected. A total of 24 SAQ papers containing 572 questions were included in this study. Every question in a paper was categorized as recall, understanding application types. Then the total number of marks allocated for each of the type of questions were calculated and compared with the total marks (98) allocated for the questions in a paper. Then the resultant weightage of marks were compared with the curricular directive weightage of marks allotted for SAQ.
Result: On analysis it was found that during the period from 2009 to 2014 76.58% and 23.42% SAQ were recall and understanding types respectively. No question was found to assess the application component of the cognitive domain of the students.
Conclusion: The study revealed that SAQ introduced as an assessment tool in undergraduate medical curriculum was not properly implemented and its desired objectives were not fully achieved.
IMC J Med Sci 2018; 12(2): 69-72. EPub date: 19 August 2018. DOI: https://doi.org/10.3329/imcjms.v12i2.39663
Address for Correspondence: Dr. Jesmin Akhter, Associate Professor, Department of Anatomy, Ibrahim Medical College, 1/A, Ibrahim Sarani, Segunbagicha, Dhaka, Bangladesh. E mail: [email protected]
Introduction
Assessment is an educational tool to evaluate students and to understand how successfully the learning materials are delivered to the learners. It also serves to motivate and help students to structure their academic efforts [1]. Principle of assessment in medical education is to provide direction and motivation for future learning and protect the health of the public by upholding high professional standards [2]. Learning abilities must be assessed in multiple modes and contexts. In addition,goodassessmentcanhelpstudentsbecome more effective self-directed learners [3].
Although continuous formative assessment constitutes an integral part of MBBS curriculum of Bangladesh, the ‘pass’ and the ‘fail’ certificates are based to a great extent on students’ performance in the final summative professional examinations. The final summative examination consists of written papers, viva-voce sessions and practical exercises. The written examination is a useful evaluation format that not only tests students’ ability to recall facts, but also can assess higher-order cognitive functions, such as interpretation of data and problem solving skills [4]. Written examination is traditionally an integral part of the evaluation of the undergraduatemedicaleducation.Thecognitive ability is assessed by the written examination using essay question, modified essay question, short answer question (SAQ) and multiple choice questions (MCQ). The SAQ is an open ended, semi-structured question format. SAQs have a better content coverage as compared to long essay question [5]. Short answer open ended questions are more flexible in that, they can test issues that require, for example, creativity, spontaneity, and perhaps the most widely accepted type.Therefore, SAQ should be aimed at the aspects of competence that cannot be tested in any other way [6]. Medical education has been found to play a vital role in nation building. Hence, the review and formulation of curriculum component is an ongoing process in several medical schools all over the world [8]. In Bangladesh, medical education is centrally controlled by the Government and run a unique undergraduate curriculum throughout the country in both public and private sectors. The present official form of undergraduate medical curriculum has first evolved in 1988 followed by revision in 2002 and 2012 [7]. In the undergraduate medical curriculum of 2002 of Bangladesh, extensive modification of the assessment system was done. In this new curriculum, the written examination format was modified to SAQ and MCQ along with 10% mark added for formative assessment [5].Written examination consists of two papers and in each paper 70% marks were allocated for SAQs. To assess different cognitive domains of students in anatomy, while constructing questions for SAQ, the curriculum has recommended, 70% marks for recall, 20% for understanding and 10% for application types of questions [8]. This new format of assessment was implemented in 1st Professional MBBS Examinations in January, 2005. Keeping this fact in mind, the present study was conducted to determine the status of implementation of the curricular directive SAQ in assessing the different components of cognitive domains of the students in anatomy of MBBS course.
Material and Methods
This cross-sectional study was conducted in the Department of Anatomy, Ibrahim Medical College. SAQ question papers on anatomy of first professional MBBS course (Paper I and Paper II), from January 2009 to July 2014 held under the University of Dhaka were selected. In first professional MBBS examination, the summative examinations on anatomy are held twice a year. A total of 24 SAQ papers were included in this study. In each paper there were Group A (35 marks) and Group B (35 marks). In each group there were seven (07) questions with or without multiple segments to assess different components of cognitive domain. Each question carried 07 marks. So, in each group 49 marks were allocated for 07 questions and in a paper with two groups, the total marks allocated for 14 questions were 98. Therefore, a total of 572 questions were included in the study, 281 questions from Paper I and 291 questions from Paper II. Every segment of the questions in a paper was categorized as recall, understanding or application types as described elsewhere [9]. Then the total number of marks allocated for each of the type of questions in a paper were calculated and compared with the total marks (98) allocated for the questions in a paper. Then the resultant weightage of marks were compared with the curricular directive weightage of marks allocated for respective component of cognitive domains. The curriculum recommended mark was 70% for recall, 20% for understanding and 10% for application types of questions [8].
Results
A total of 24 SAQ papers were included in this study. Recall type of questions carried most of the weightage and none regarding application type of questions. Table-1 shows that the percentage of recall type of questions were always more than the curriculum directed percentage. It was lowest in the year 2010 (73%) and highest in the year 2014 (81%). The percentage of marks allocated for understanding type of questions also did not follow the curriculum directed weightage. It was highest in the year 2010 (27%) and lowest in the year 2014 (19%). No mark was allocated for application type of SAQ in anatomy in six years. Over the period of six years from 2009 to 2014, overall 76.58% and 23.42% of marks were allocated for recall and understanding types respectively. No application type of question was found.
Table-1: Distribution of marks allocated for different types of questions in SAQ in anatomy from 2009 to 2014
Discussion
The present study was conducted only on SAQ in anatomy of first professional examination under the University of Dhaka, and therefore, question papers from all universities in Bangladesh were not included. Analysis of the SAQs on anatomy from 2009 to 2014 of first professional MBBS course revealed that the SAQ set in the examination nearly fulfilled the curricular directive allocation of percentage of marks for recall and understanding types of question to assess the cognitive domain of the students. In the present study, the allocation of marks for recall type of questions was 76.58% which was almost within the range of 70% as recommended by curriculum. The percentage of marks allocated in curriculum for understanding type of questions is 20%; and in present study it was found to be 23.42%, which was not much deviated from the curricular directive. The only shortfall of the SAQ was that, no question was found to assess the application component of the cognitive domain of the students. The study revealed that the objective of introducing SAQ as an assessment tool in undergraduate medical curriculum in anatomy was achieved to a great extent. However, there is lack in preparing SAQ on anatomy to assess the application aspect of the students. Alam et al [2] in their study on structured oral examination found that the curriculum directed format was not followed properly. Without using test blue print in construction of questions, it is quite impossible to assess the candidate’s learning hierarchy.The actual scenario of assessingdifferent components of cognitive domain of students in other medical subjects using SAQ needs to be explored. Also, it is important to evaluate the situations of SAQ in anatomy summative examination of other universities in Bangladesh.
Conclusion
The study has revealed that the objective of SAQ in undergraduate medical curriculum is yet to be fully achieved. It is recommended that further study may be instituted to determine the situation in other subjects and to find out the causes of not achieving the objectives of SAQ in undergraduate medical evaluation system. The policy makers must take necessary action to arrange regular and intensive training program for faculty members to improve the quality of SAQ in undergraduate medical education as an assessment tool.
Author’s contribution
JA was involved in study design, collection and analysis of data, and writing of manuscript. SS was involved in collection and analysis of data
Competing interest
The authors declared no competing interests.
Funding
None
References
1. Adniyi OS, Ogli SA, Ojabo CO, Musa DI. The impact of various assessment parameters on medical students’ performance in first professional examination in physiology. Niger Med J. 2013; 54(5): 302-5.
2. Alam MS, Begum T. Evaluation of structured oral examination format used in the assessment of undergraduate medical course (MBBS) of the University of Dhaka. Ibrahim Med. Coll. J. 2015; 9(1): 1-10.
3. Wood DF. ABC of learning and teaching in medical problem based learning. BMJ. 2003; 326: 328-30.
4. Ghosh A, Mandal A, Das N, Tripathi SK, Biswas A, Bera T. Students’ performance in written and viva-voce components of final summative pharmacology examination in MBBS curriculum: A critical insight. Indian J Parmacol. 2012; 44(2): 274-75.
5. Lambert WTS, Cees PM van der Vleuten. ABC of learning and teaching in medicine written assessment.BMJ. 2003; 326: 643–45.
6. Tabish SA. Assessment methods in medical education. Int J Health Sci (Qassim). 2008; 2(2): 3–7.
7. Shittu LAJ, Zachariah MP, Izegbu MC, Adesanya OA, Ashiru OA. The differential impact of various assessment parameters on the medical students performance in the professional anatomy examination in a new medical school. Int J Morphol. 2006; 24(4): 659-64.
8. Haque M, Yousuf R, Abu Bakar SM, Salam A. Assessment in undergraduate medical education: Bangladesh perspectives. Bangladesh J of Med Sci. 2013; 12(4): 357–63.
9. Amin NF, Akter M, Kabir MH, Shamim KM. Medical undergraduates’ residual knowledge of structural, functional and clinical aspects of cell and histology at recall and understanding levels. Bangladesh J. Anat. 2013; 11(1): 34-43.
