We discussed the basic assumptions underlying student evaluation in the previous chapter. With this basic knowledge, you are ready to go on to evaluation of specific learning outcomes. You will be in a better position to understand this chapter, if you quickly revise the chapter on Educational Objectives.

You may be remembering that learning outcomes are presented hierarchically so that an increasingly complex learning demand is made out of the student in the context of cognitive skills. The sequence is represented as Factual knowledge - Understanding - Application - Synthesis - Analysis - Evaluation. Thus, whichever evaluation tool we design, it has to be in line with the sequence designed above. As a general rule, early in the course, factual knowledge is presented to the student and as the course develops, higher domains of learning come into picture.

The commonest tool used for evaluation of knowledge is the traditional question. Let us discuss the various advantages used from of an essay question in most Indian Universities is what can be called an unstructured essay question. These questions are worded in such a way that the student has the freedom, within the subject context, to determine the nature and scope of the answer. look at the following examples :

• Write an essay on Protein Energy Malnutrition.

• Discuss the Universal Immunisation Programme.

You will appreciate that different students will write different answers to these questions - hence these questions are also known as free response questions. These questions are specially useful for evaluating the higher domains of learning and comprehension of the student but since a great deal of subjectivity is involved in evaluations these questions, the marks are often criticised as being unreliable. Did you note that we have not labelled essay questions as unreliable - rather, we have said that it is the marking which is unreliable. In other words, with deliberate effort, an examiner can build reliability into his marking of essay questions. We shall discuss this aspect a little later.

The second type of essay questions that are used can be called structured question (SEQ). They differ from unstructured questions in the freedom regarding scope and nature of the answer. They are framed in such a way that the student is provided considerable guidance regarding points to be included in the answer - hence they are also called restricted response questions. The restriction can be improved by the examiner either in terms of length of expected answer or in terms of points to be included in the answer. Take a look at the following examples :

• Write an essay on protein energy malnutrition covering the causes, classification, clinical diagnosis and initial management in the hospital.

• Discuss the Universal Immunisation Programme in the context of its objectives, operationalisation, targets and limitations.

Modified essay question (MEQ) is yet another type of essay question. MEQ is basically a problem solving type of question, wherein a short history is given to the student, based on which questions are asked. This requires the student to apply what he has learnt, in the context of a given situation. The following example will illustrate this point.

A 3 day old neonate presents with history of jaundice noted at 48 hours of age. Baby is full term and weighs 3 Kg. On examination, he looks very pale and has a palpable spleen. Rest of the examination is normal.

• What are the possible causes of this type of presentation ?

• Which investiations will you order to arrive at a diagnosis ?

• The billrubin has been reported as 20 mg%. What would be your next line of action ?

• If this baby is not properly managed, what can be the long term sequelae ?

You will note that these questions are mutually exclusive i.e. answer to a subsequent question does not depend on the previous question nor do the subsequent questions provide any clue regarding possible answers to the preceeding questions.

You must remember that proper construction of essay type questions is important to give clear unambiguous directions to the student so that the answer is marked with minimum of subjectivity. Here are some simple guidelines which will help you in writing better essay questions. This will also help in improving their reliability.

• Match the question to specific learning outcome. In other words, the objective of learning and objective of testing should be the same. If the learning objective was that the student should be able to enumerate 10 common toxic effects of a particular drug, then a matching essay question would be - Enumerate the common toxic effects of this drug. This indirectly means that the more precise and clearer we are with our learning objectives, the easier and better it will become to formulate good essay questions.

• The question should clearly specify to the student as to what is expected of him. Recall the question on protein energy malnutrition - a student can write about epidemiology, another about clinical features, yet another about domiciliary management and be technically correct. Further, different students may be marked on different abilities and still score equal marks.

• Phrase the question in a simple, clear and straight - forward language, avoiding what is called window - dressing, Remember that your aim is to assess the student's ability to interpret the scientific data and not his literary comprehension.

• Where necessary, provide reference points or supplemental information. (For example, management of hemolytic jaundice in a neonate and in an adult is totally different. In one of the examinations, students described kernicterus in response to a question on hepatic encephalopathy !)

• Indicate the weightage to different parts of the question. This helps the student to effectively plan his answer and avoid spending too much time on trivia.

Essay questions have the inherent drawback of a high degree of subjectivity involved in marking. Inspite of best of efforts, extraneous factors like hand writing, grammar, diagrams, underlinings and use of pens are all likely to influence the marks awarded. In addition, the examiner may be biased towards a particular point of view. Response to first question - well written or otherwise, is also likely to have effect on subsequent answers. It has also been seen that there is a significant inter - rater difference between marks. In other words, student's luck plays a major role in determining his marks - depending on whether who is evaluating the answer sheets is tough or lenient. Even a student can sometimes exploit these drawbacks and bluff the examiner. Many of these problems can be eliminated if we follow the following simple rules :

• Always keep the specific learning outcome in mind. While marking, you should use this outcome as the sole criteria and not be distracted by other factors. You have to filter out irrelevant information and ignore it.

• You should prepare a checklist and allot marks to different points to be included in the answer. This helps to reduce subjectivity to a great extent. Let us illustrate this by taking the question on protein energy malnutrition, assuming that the question carries 10 marks.

• You should mark essays question - by question rather than student - by student. Thus, you will be comparing the performance of each student on that particular question and not as a whole.