During your students days, you must have felt a lot of dissatisfaction over the marks awarded to you in essay type questions. Most of the times, it is that the marks do not reflect the true capability. Some studies conducted to assess the marking of essay type questions have shown that there may be a difference of as much as 35%  marks between two examiners - which in effect means that the same candidate may be failed by one examiner and awarded distinction by another. There are ways and means to check this kind of discrepancy - however, they are time consuming and given the constraints of time, very few type questions using the accepted methodology. Even if the essay type questions were to be evaluated by the suggested methodology, it would take a long time to rebuild the faith of the students as well as the society in such a system of examination.

Besides being liable to subjective marking, essay type questions also have another limition and that is the number of questions that  can be given within the alloted time. As you will learn in the chapter on Test and item analysis the  reliability of a test and standard error of measurement are directly proportional to the length of the given test. Thus, even with very careful and planned marking, essay questions are bound to have a low reliability.

Against this background, objective type questions offer a distinct advantage of being more reliable - not only because of a pre - determined correct answer but also because of the length of the test. They are easy to mark and can be used on repeated occasions. They provide a wider sampling of subject matter. There are many varieties evaluation. Broadly, they can be classified as selection type (i.e. where the student has to select a correct answer out of the list provided) and supply type (i.e. where the correct answer has to be supplied by the student). The selection type of questions can be further sub divided into the following types :

• Constant alternatives

• Multiple choice

• Multiple facet

• Matching

• Relation - assertion

• Rearrangement

In this chapter, we shall deal only with multiple choice questions (MCQ). Before proceeding further, let us make it clear that the only thing objective about objective type of questions is that they can be marked objectively - otherwise they are as prone to subjective errors as any other test. It is therefore important that due care and attention is given while formulating and writing these questions (they are conventionally called 'items' - in our subsequent discussions, we shall also use the same terminology).

Multiple Choice Questions (MCQs) :

We are all familiar with this format of question and they are being increasingly used in almost every kind of examination. A common notion exists that it is difficult to write good MCQs - however, this is not true and the technique of writing a good MCQ is easier to master than the technique of writing a good essay question.

Let us first look at the basic terminology used in relation to an MCQ. This has been illustrated in the following example :

The drug of choice for treatment of congestive cardiac failure is :

STEM

(a)  Propranolol

(b)  Aminophylline

(c)  Isoptin

(d)  Digitalis

The question part of the item is called 'stem' ; correct answer is called the 'key' while rest of the options are called the 'distractors'. It is worth mentioning here that the, quality of an MCQ depends on the stem and the distractors - that is, how far they are able to distract the student who doesn't know the correct answer. Thus, a thoughtful deliberate attempt has to be made to provide effective distractors.

You may be wondering, how to write a good MCQ. Let us take you though the following steps, essential for writing a good item. Happy reading !

• Select a specific learning objective which you want to test.

• Write a stem to include a question or a problem to be solved by the student.

• Write the correct answer to this stem. Be sure to cross - check the correctness of the answer. It should be unambiguous and unarguably the correct answer.

• Recall the common mistakes that the students make about this in your day to day teaching. If you have given a short answer question formative test, it provides you with a rich source of such mistakes.

• Select the most plausible alternatives and arrange them in the form of options.

• Read the item yourself for any possible corrections. One of the best ways of improving the quality of your items is to have them read by your colleagues. Experience has shown that on an average, half of the items are discarded as 'not good' by this method.

• Try the item on a group of students during class tests and from the number of students answering it correctly, calculate the Facility value and Discrimination index of each item. only those items which have these indices within the acceptable limits should be retained for further use. We shall be talking about these indices in a later chapter.

These are the standard steps to write an item. Let us now tell you some 'tricks of the trade' to further improve the quality of your items.

A)  Each item should test an important learning outcome. Avoid trivia and using statements from books as such for writing the item. Do you recall the term 'Table of Specifications' ? It gives you the relative weight that you should allot to different subject areas in setting a question paper.

B) Each item, as far as possible, should be complete and independent. If a subsequent item is dependent on the previous one, then the student may be penalised twice for one wrong answer. Look at the following items :

Scurvy is caused by deficiency of which vitamin ?

(a)   A
(b)  B
(c)  C
(d)  K

Key : c

The commonest source of this vitamin is :
(a)  Banana
(b)  Lemon
(c)  Carrots
(d)  Wheat

Key : b

The second item could have been framed differently to ask the source of a particular vitamin.

C) The items should be written in a standard format. It means writing the stem on top and beginning the options from the next line, one option in one line. The options can be numbered as 1, 2, 3, 4 or as a, b, c, d. Traditionally, the latter is preferred as using numerals may confuse the student with serial number of the item. Format on right is easy to understand.

(a)  Thyrotoxicosis

(b)  Basal ganglion lesion

(c)  Cerebellar lesions

(d)  Motor neurone disease is associated with intention tremors.

D) Each item should be based on a single central theme. e.g.

In case of sunstroke, the victim :
(a)  Should have head lowered
(b)  Has a weak pulse
(c)  Should be given a stimulant
(d)  Should have cold sponging

In this example, the options include symptoms, signs and treatment. It could have been better written as :

The commonest feature of sunstroke is :
(a)  Weak pulse
(b)   Pale face
(c)  High temperature
(d)  Excessive perspiration

E) On the same premise, an item should not become a series of true / false statements.

Which of the following is true about Measles :
(a)  Koplik's spots precede the rash
(b)  Tuberculin test becomes positive
(c)  Pancreatitis is a common complication
(d)  Prodrome lasts for 1-2 days. 

In a good MCQ, as soon as the student reads the stem, the correct answer should flash in his mind and all has to do is to compare the correct answer with the options provided. In the preceding example, he has to read each option and then decide if it is true. Since time constraint is always there in an MCQ test, we are being unfair to the student by adopting such a format.

F) The stem of the item should be a clearly formulated problem rather than a single word or a passive phrase. Do not write an item like this :

The mandibular nerve :
(a)  Is sensory
(b)  Supplies Buccinator
(c  Is a branch of vagus nerve
(d)  Enters through foramen magnum

G) The statement should be qualified where ever necessary.

The infant mortality rate is :
(a) 40
(b)  80
(c)  95
(d)  120 

Here, neither the denominator has been provided (per 1000 live births) nor it has been stated whether the IMR of urban or rural areas is required. The year under reference has also to be mentioned since figure of IMR is different in the stem. Look at the following two items :

H) The stem should be complete in itself and should not be department on the options. If there are certain common elements  in the options, they should be included in the stem. Look at the following two items :

Protein requirement of Indian reference man is :
(a)  1 gm per kg body weight
(b)  2 gm per kg body weight
(c)  5 gm per kg body weight
(d)  10 gm per kg body weight 

Key : a

Protein requirement (gm per Kg body weight) of Indian reference man is :
(a)  1
(b)  2
(c)  5
(d)  10

Key : a

I)  As far as possible, negative words should be avoided from the stem. If they are inevitable, then they highlighted by writing them in capitals, italics or by underlying them.

Clubbing of nails is NOT associated with :
(a)  Chronic lung disease
(b)  Cyanotic heart disease
(c)  Ulcerative colitis
(d)  Bronchial asthma

Key :d

Double negatives should be avoided at any cost. Following items illustrate this :

Which of the following are not loop diuretics except :
(a)  Frusemide
(b)  Chlorthalidone
(c)  Aldactone
(d)  Dytide 

Key : a

Which of the following are not loop diuretics :
(a)  Frusemide
(b)  Chlorthalidone
(c)  Aldactone
(d)  Dytide 

Key : a

J) Abbreviations should be avoided in the stem. This is specially true of non - standard terms.

K) Avoid 'window - dressing' of the statement. This means writing extraneous and unnecessary words which tend to confuse the student. This point has been illustrated in the following items.

With advancement of statistical methods, there has been concurrently paradoxical simplification of diagramatic representations. A suitable diagram for depicting continuous quantitative data like monthly hospital admission rate would be :
(a)  Histogram
(b)  Pie diagram
(c)  Simple bar diagram
(d)  Scatter diagram 

Key : c

A suitable diagram for depicting monthly hospital admission rate is :
(a)  Histogram
(b)  Pie diagram
(c)  Simple bar diagram
(d)  Scatter diagram 

Key : c

L)  Expressions like fairly high, considerable, majority, etc. should be avoided. Similarly, words like never, always, usually, sometimes should be avoided. These terms are more likely to confuse a brighter student rather than an average student.

M)  The options should be parallel in content and have same relation to the stem. They should  also be grammatically parallel.

Hemolytic anemia is suggested by :
(a)  High alkaline phosphatase
(b)  Reticulocyte count is high
(c)  Only jaundice
(d)  platelet count low.

Key : b

This item could have been better framed as follows :

Which of the following suggests a hemolytic picture :
(a)  Jaundice
(b)  Low platelet court
(c)  High alkaline phosphatase level
(d)  High  reticulocyte count

Key : d

N) The key should clearly be the best choice and subject experts should agree on the same. If there is a subjective element involved, the item may become confusing, specially for the brighter student. Look at some of the examples, where different students  can give different answers and yet be correct.

The treatment of bronchogenic carcinoma is :

(a)  Radiotherapy
(b)  Chemotherapy
(c)  Surgery
(d) Immunotherapy

What fraction of Choice active hepatitis patients will develop cirrhosis :

(a)  1%
(b)  2%
(c)  5%
(d) 10%

The commonest cause of abdominal distension is :
(a)  Flatulence
(b)  Electrolyte imbalance
(c)  Intestinal obstruction
(d) Paralytic ileus

O)   The distractors should be such that only lower ability students should be distracted by them. Avoid using 'bogey' just to create four alternatives. If you can not think of any effective distractor, leave the alternatives at 3 rather than using a distractor which will not distract anyone. The option 'd' in the following 2 items illustrates this :

Integrated child development program provides service till the age of :
(a)  1 year
(b)  3 year
(c)  5 year
(d) Life long

Key : c

The average weight of a normal adult kidney (in gms) is :

(a)  50
(b)  100
(c)  500
(d) 1000

Key : c

On the other hand, a good distractor will be attractive to the lower ability student. Look at distractors in the following item.

Double contour shadow on right lower heart border on x-ray chest is suggestive of enlargement of :

(a)  Right atrium
(b)  Left atrium
(c)  Right ventricle
(d) Right pulmonary vein

Key : b

P)  When writing options, try to avoid duplications or making options all inclusive.

Children of what age are covered under Universal immunisation programme ?

(a)  Under 5 years
(b)  0-1 years
(c)  0-3 years
(d) 0-5 years

Key : b

What percentage of Indian children are malnourished ?

(a)  Less than 20%
(b)  Less than 40%
(c)  More than 40%
(d) More than 60%

Similarly, do not make the options mutually exclusive, which in effect means that now the student is left with only 2 options rather than 4.

Which of the following change occur in bronchial asthma ?

(a)  VC is increased
(b)  VC is decreased
(c)  FEV is increased
(d) FEV is decreased

Q) The language used should be appropriate to the level and vocabulary of the students. Look at the following item and its modified version :

Cholera is spread by :

(a)  Tiny molecules in the air
(b)  Contamination of the food with infected excreta
(c)  Administration of blood from a patient
(d) Sex with an infected person

Cholera is spread by :

(a)  Droplet infection
(b)  Oro - fecal route
(c)  Blood transfusion
(d) Sexual contact

R) The options should be arranged in rank order so that the student does not have to hop through the options to spot the correct answer. Will you like the following sequence ?

The average duration of human pregnancy (in days) is :

(a)  258
(b)  280
(c)  266
(d) 290

Key : b

S) As far as possible, 'all of the above' should be avoided as an option, more so if it also happens to be the correct option. If a student can recognise 2 correct options, he can safely tick 'all of the above' without even reading the third.

Which of the following are diuretics ?
(a)  Frusemide
(b)  Digoxin
(c)  Aminophylline
(d) All of the above

Key : d

Similarly, 'none of the above' should not be used as a dummy options. Even if it is the correct alternative, it does nor ensure that the students knows the right answer or the  reason behind it :

The does of BCG vaccine is :

(a)  0.2 ml
(b)  0.3 ml
(c)  0.4 ml
(d) None of the above

Key : d

'None of the above' should definitely be avoided with negative stems.

Following contrast media are used for IVP except :
(a)  Conray
(b)  Urograffin
(c)  Dianosil
(d) None of the above

Key : d

in some of the situations, particularly those requiring mathematical calculations, 'none of the above' may be an acceptable option.

Replacement fluid for a 1 year old weighing 10 kg with moderate dehydration will be :
(a)  200 ml
(b)  300 ml
(c)  500 ml
(d) None of the above

Key   : d

T)We tend to provide clues by way of either singular / plural expressions or by use of a / an , as illustrated by following.

Imipramine is an :
(a)  Anti - depressant
(b)  Bronchodilator
(c)  Sedative
(d) Stimulant

Key : a

Length or precision of the key is another factor which may help the students to make a guess, without actually knowing the answer.

Water hammer pulse is characterised by :
(a)  High volume
(b)  Rapid upstroke and descent of pulse wave
(c)  Very rapid rate
(d) Low pulse pressure

Key : b

Protein content of 100 ml of buffalo milk is :
(a)  1 gm
(b)  2 gm
(c)  3 gm
(d) 4.3 gm

Key : d

In which of the following conditions does hemolysis play an important role in pathophysiology ?
(a)  Albers - Schonberg disease
(b)  Megaloblastic anemia
(c)  Henoch - Schonlien purpura
(d) Hemolytic - uremic syndrome

yes, you are right. The correct answer is d !

Another inadvertent clue may come from subconscious preference to a particular option or by adopting a well defined sequence to a particular option or by adopting a well defined sequence of options, as shown below :

U) Often a question is raised regarding number of distractors in a good item. Generally speaking, lesser the number of distractors, higher is the scope for guessing. However, at the same time, poor distractors should not be used just to maintain uniformity. Four options (i.e. 1 key and 3 distractors) are generally considered satisfactory.

Checklist for making MCQs :

• Does the item deal with one or more important aspects of the subject ? The minutiae of knowledge are to be avoided.

• Does the item call for information which any physician should know without consulting a reference source ? Drug dosage, limits of normal values, and other numerical data are to be included only if they deal with information that should be within the daily working knowledge of the physician.

• Is the item appropriate for the level of knowledge expected of the examinee ? The item should be neither too difficult nor too easy.

• Is the central problem stated clearly and accurately ? Wording that is ambiguous or fuzzy may mislead the examinee and destroy the validity of the item.

• Is the item written with as few words as possible to make it clear and complete ? Unnecessary words increase reading time ; the examination is intended to test medical knowledge, not reading speed.

• Is the item type the best one for the particular point or problem ? A topic difficult to test by one type of item may be tested without difficulty be another type.

• Are negatives avoided ? An item involving a double negative is usually improved by rewording to a positive form or by changing to a different type.

• Is the item written in conformity with the designated format ? For example, in open best response type of question, the proper number of choice (distractors) must be grammatically consistent with the main statement (the stem).

• Is each alternative (distractor) a plausible response ? Silly or irrelevant wrong answers fool nobody and have the effect of reducing the multiplicity of choice.

Although the discussion has been rather long, we have tried to show you some of the common pitfalls that can creep in while writing a good item. It is worth emphasising that the time and effort spent in writing a good item is more than repaid in the long run.

One of the common criticisms that is often levelled against MCQs is that they test only simple recall and do not require the student to reason out, analyse or synthesise the facts. This criticism is not valid, specially when we introduce you to certain other types of MCQs, which test the higher abilities of the student. These questions can test a wider range of abilities but require a lot more effort and ingenuity in farming.

Multiple approach :

These items are a variation of traditional MCQs and allow objective marking. However, they have the advantage that a student with partial knowledge does not get credit and guess work is reduced to a minimum.

These questions require a variation of the key and out of many formats possible, one is illustrated below :
(a)  If only I is correct
(b)   If only II is correct
(c)   If only II & IV are correct
(d)  If all are correct

Look at some of the questions below.

Vaccines given in the neonatal period include --
I    Tetanus
II   Polio
III  Pertussis
IV  hepatitis B

Key - c    

Which of the following foods contain proteins ?
I    Cereals
II   Pulses
III  Green leafy vegetables
IV  Eggs

Key - d

Which organisms cause meningitis beyond neonatal period ?
I     Streptococcus
II    Pneumococcus
III   E.coli
IV   meningococcus

A you would have noted, these items donot give any credit for partial knowledge and are therefore specially suited for higher levels of learning.

Analysis of relationship :

A. First has a beneficial effect on the second.
B. First has a detrimental effect on the second.
C.  First has no effect on the second.

A variation of this type of format can be used to assess the ability to analyse the evidence. The student is given certain facts and later certain conclusions. He has to indicate the facts which support those particular conclusions.

in breast fed babies --

Infections are less

f, e

Hypocalcemia is less

c, d, e,

Biovailability of Iron is better

e

We hope, you have understood the technique of writing good items. To confirm it, why don't you try writing few MCQs and discuss it with your peers. Even if some items are rejected, don't feel disheartened - it is only practice which will make you perfect.