Medical exam candidate logging in to exam feedback
Prompt, personalised exam feedback motivates students as well as giving them vital information to improve their learning and revision going forward. This is particularly important when exams are sat on a distributed basis.
We are delighted to announce that we have won an Epic Games MegaGrant to explore the use of Virtual Reality (VR) within our Maxexam software. Epic Games launched these awards to reward companies ‘doing amazing things’ utilising Unreal Engine – a suite of development tools for people/companies using real-time technology. Maxinity is a key award winner and we will be using the funds received to investigate the use of VR in a clinical exam setting.
Distributed exams, often also described as remote exams, are where the same exam is sat in a variety of different locations. For example, a candidate may sit an exam in their own home.  This is particularly relevant at present due to Covid-19. Our experience is that when you conduct an internet search on distributed or remote exams, you will only see the positives of running them, but we wanted to take a more balanced view looking at both the pros and cons of this method of assessment.
The Horst Partial Knowledge Index or PKI (Maxinity terminology) is a statistic which can be used to help understand and consequently improve item reliability. It is particularly useful at identifying problem items, as a negative Horst PKI indicates that more candidates answered a question with a specific wrong answer (most popular distractor) than the correct answer.
The PCC measures the strength of the relationship between two variables - in an exam this would be between candidates’ item/question/scenario marks and their exam marks. The Pearson's Correlation attempts to draw a line of best fit through the data points plotted on a graph, and the coefficient evaluates how far away from the line of best fit these data points are.
Are you considering or do you already use point biserial to analyse items in your exams? Our view is that it is useless for all but the simplest of tests - read more to find out why.
The discrimination index (DI) measures how discriminating items in an exam are – i.e. how well an item can differentiate between good candidates and less able ones. For each item it is a measure based on the comparison of performance between stronger and weaker candidates in the exam as a whole.
Item analysis uses different statistical measures in order to determine any problems with the ‘items’ that make up a question in an exam, and hence the question itself. Are your questions working as they should?
When developing a medical or dental examination, it is likely that you will want to include essential and ranking questions - but does your marking scheme discriminate between them?
Standard setting is a way to define levels of achievement or proficiency in an exam and the cut-off marks corresponding to those levels. We summarise the methods and advantages of each, as well as where they are most appropriate.