EVALUATION OF ENTRANCE MCQ EXAMS AND THE CORRELATION BETWEEN QUESTION DIFFICULTY AND DISCRIMINATION

Abstract

Objective: The aim is to create multiple-choice questions (MCQs) that align with the curriculum and effectively evaluate the cognitive abilities of medical students. The questions should be complete and have a discrimination index (D) of 27% and suitable rpbis. The question has a moderate difficulty level, denoted as p, falling within the range of 0.6 to 0.7. However, for simple words with a probability greater than 0.8, rpbis demonstrates a favorable value, while D27% has a poorer value. This study aims to analyze the relationship between difficulty and discrimination (D27%, Dmax27%, rpbis) of the easy and very easy multiple-choice questions in the head anatomy graduate entrance exam for the master's training program at University of Medicine and Pharmacy at Ho Chi Minh City.

Methods: A cross-sectional research design was utilized to assess question 1 of the anatomy graduate entrance exam, which consisted of 120 questions and was administered in 2022. The study gathered 325 test results from a diverse set of applicants who were taking the postgraduate exam. The difficulty of the questions was determined using question analysis based on Classical Test Theory (CTT), along with the discrimination index D27% and the point-biserial correlation (rpbis) using the Basicstat software. The maximum discrimination (Dmax27%) was calculated in Excel, as well as the frequency of non-functional distractors (NPD) with less than 5% selection.

Results: The discrimination index D, which was 27%, exhibited a weak negative connection with difficulty (r = -0.34,

p = 0.00012 <0.01). The questions that were classified as easy/moderately difficult (P = 40% - 80%) showed the highest level of discrimination compatibility, as indicated by a D27% value of 0.6 – 0.7 and rpbis. However, within the category of easy questions (where the probability of success is greater than 70%), only 63% of the questions showed a significant distinction based on the D index, which requires a discrimination of at least 0.3. On the other hand, when utilizing the rpbis index, 94.5% of the questions achieved good discrimination. Significantly, out of the 19 questions that are considered very easy (with a success rate of 92% or more), the D index of 27% is below the threshold of 0.3, indicating that these questions should be revised or removed. Out of the 19 questions listed, 15 of them show a significant distinction with the rpbis index ranging from 0.313 to 0.531. This is because these questions have a concentrated amount of incorrect responses in the low score group, despite having nearly the same both high and low numbers of correct answers in both groups. The easiesty questions have 2 or more NPDs but have good discrimination between each distractor with their rpbis.

Conclusion: More than 80% of MCQ questions are very easy p >0.8 as well as there are more than 2 NPDs in the test, but they show good discrimination based on point-biserial correlations (rpbis). This underscores the intricate correlation between the level of difficulty and the discrimination power of the questions. When it comes to school or national question choice banks, it is advisable to utilize rpbis.

Keywords: difficulty; maximum discrimination; discrimination index; point-biserial correlations; non-functional distractors; classical test theory