MULTIPLE-CHOICE ITEMS FOR TESTING UNIVERSITY STUDENTS ACHIEVEMENT

BY: SOERADJI HERIANTO & HENRY ASMARA

1. Introduction

Multiple choice test items are the most highly regarded and widely used form of objective test type items. Almost any understanding or ability that can be tested by means of any other item form: short answer (completion), true-false, rearranging, etc. and even essay can also be tested by means of multiple choice test items. This paper tries to tell the readers “how” to write multiple choice items.

2. The Multiple Choice Item

For purposes of convenience and reference the multiple choice item is generally divided into : STEM, The Key or the ANSWER, and the DISTRACTERS. The key or answer and the distracters are both called the OPTIONS of the item.

1. The Stem.

This is the part of the item that is intended to elicit the response. An item may be in the form of a question or an incomplete statement involving the problem to be solved. If a stem is a question, it is also good to be followed by a short incomplete statement so that together with the answer it makes a complete sentence.

2. The Key

The key is the correct answer to the stem. This is the choice that a student is required to make from the options provided. There is only one key in the option.

3. The Distracters

The distracters are the other choice surrounding the key in the options

4. The Options

These parts of an items are those containing both the answer to the question or the solution of the problem and the distracters. Number of options varies from two to five depending on the degree of difficulty of the subject matter and on the grade of the students to be tested. Also it greatly depends on the aspects of learning to be tested, like: knowledge, comprehension, application, analysis, synthesis, or evaluation.

3. Points to Watch When Writing multiple choice Items

a. The item should be suited to the level of ability of the students to be tested, otherwise it may do harm to the testees.

b. Try as far as possible to test only one aspect in an item, not more.

c. The instructions for each item or group of items must be written as simply, concisely and precisely as possible without omitting the vital information. This often demands some care.

d. An item requiring the best answer is desirable.

e. An item using descriptive responses is also desirable.
Example :
Monogamy is:
a. refusal to marry.
b. marriage of one woman to more than one husband.
c. marriage of one husband to more than one woman
*d. marriage of one man to only one wife
(*answer key)

6. It is undesirable to ask examinee his option.
7. A good item does not use verbal trick.

Example : Choose the best answer to the question!
Democracy is the rule of the people.
What does “the people” here mean?
a. some people
b. the people not as a whole
c. most the people
d. the majority minus the minority

In this item it is obvious that “not” in the option b is tricky. Without the “not”, the choice b will be the key to the key to the answer. And now it is doubted if the “best” answer of the four options will meet what is really required by the item writer.

8. An incomplete stem is undesirable.

The teacher has demonstrated:
a. some aspects of the rating scale
b. the rating scale for evaluating students
c. the rating scale for evaluating clinical performance
d. the rating scale for evaluating medical students

To improve this item, we should include the problem chosen in the stem.

9. An item involving “window dressing” is undesirable.

Example: (undesirable)

Education is compulsory in Australia. To meet this the Australian Government also employs teachers from other countries. Some of them are from America.

Australian Children have to go to school up to the age of:
*a. fifteen
b. sixteen
c. seventeen
d. eighteen
except in Tasmania

10. Make sure that the item is with natural sequence of responses.

Example: (undesirable)

The electronic structure of chlorine is …………

*a. 1s²2s²2p⁶3s²3p⁵
b. 1s²2s²2p⁶3s²3p³4s³
c. 1s²2s²2p⁶3s²3p⁴4s¹
d. 1s²2s²2p⁶3s¹3p⁶

11. The stem of the item should be written as clearly and economically as possible so as to avoid any ambiguity or obscurity.
12. Avoid making negative stem if possible.
13. Include as many words as possible in the stem so that to reduce the wording of the options.
14. There is only one best or correct answer to the question.
15. Make sure that the distracters are completely correct and contains no clue.
16. Avoid putting the answer in the same place too often. It is good if the writer of the test do it in random.
17. Make sure that the distracters are completely wrong, yet they should be correct statements.
18. In spite of their being wrong, the distracters should be plausible to the poorer students.
19. Make sure that the distracters are not acceptable in any other varieties with which the candidates may be familiar.

Example: (undesirable)

The correct pronunciation of “quite right” is :

a. [ kwa? rai? ]
*b. [ kwait rait ]
c. [ kwaэ raiэ ]
d. [ kweit reit ]

20. The distracters should be wrong in themselves and do not become acceptable when the context is looked from another aspect.

Example: (undesirable)

When he realized what a stupid mistake he had made.
Tarka could have kicked ………………..

*a. him
b. oneself
*c. himself
d. he

(the key may be a or c depending on whom is “Tarka”)

21. The elimination of a distracters should not be more difficult than the choice of the answer..
22. The options should be grammatically appropriate to the stem.

Example: (desirable)

The object of sterilization in food material is …………………..
a. to grow micro-organisms
b. to make the colour attractive
*c. to prevent decomposition
d. to reduce the number of spores

23. The choices should be mutually exclusive, i.e. that they do not overlap or include each other. If two choices overlap, they can both be eliminated together or improved.

Example: (undesirable)

The study of relationship between plants, animals and their environment, both living and non-living, is called ……………..
a. Physiology
*b. Ecology
*c. Biology
d. Zoology

x. It is suggested that option should be arranged vertically rather than horizontally.

Example: (undesirable)

Whish of the following substances cause temporary hardness in water? a.Magnesium Clorida,
b.Calcium Sulphate,
c.Iron Chlorida,
*d.Calsium Bicarbonate.

y. The distracters should not be too similar.

Example: (undesirable)

The large number of insect species alive today is evidence of:
a. their relative freedom from attacks of predators.
*b. perfection of adaptation to their environment.
c. their very complex structure.
d. their great ability to produce many young.

Z. Items in a test should not interlock, i.e. that an item in a test may be used to provide the answer to the other item.

Generally multiple choice tends to be more direct and less misleading than some other item forms. Sometimes it is less ambiguous than true-false or completion items. It is alternatively easier to defend the correct answers to them. Guessing is also less in multiple choice.

Inspite of their virtues, multiple choice item cannot escape comments. Some of the criticisms reflect a general mistrust of all objective testing techniques. As a matter of fact, those criticisms are believed to be mostly affected by the absence of care of the instructors or writers in setting up multiple choice items.

Example: (undesirable)

The colour of the sky is : .........................
a. grey
b. blue
c. black
d. red

I believe you agree that none of the options is incorrect considering the weather and all that may affect the colour of the sky.

4. SCORING

W Score = R = ------- N - 1

The formula for scoring multiple choice is :

Where :

R : stands for “Right”, referring to the number of the right answers chosen by the testees.
W : stands for “wrong”, referring to the number of wrong answers chosen by the testees.
N : refers to the number of options (choices) in the item.
1 : (one) is a constant figure.

Basically, is meant to correct the will guessing possibly made by the testees in doing the test.
But since we cannot guarantee that a right answer is not the result from guessing, and for some reasons university students are regarded to have avoided guessing as far as possible and even reasoning may be omitted accordingly. So, the score of the multiple choice is the whole correct answers achieved by the candidates.

5. TRY OUT

It is always best if we could try out the test before use. Otherwise the writer should try ask his colleagues to view the tests he has just constructed or he should leave the tests for some days, say a week or more, to be improved later before he wants to use them. Improvement should be made continuously if the tests are intended to be used in a period of time. By doing so you will find out whether:

a. The “TIME ALLOWED” suits the requirement.
b. There is any weak items or options.
c. There is any clues in the options, etc.

6. WEAK ITEMS / OPTIONS AND STRONG ONES

An item is considered weak if all students or none can answer the question. A weak option is that only few students (testees), say, less than 5% of the whole, have taken it. A key is considered good if about half of the students have chosen it.

7. ANSWER SHEET AND ITS KEY

Make sure that the key to the answer sheet is exactly the same as the answer sheet itself, otherwise the correct or will be in trouble in the correcting. This point is unnecessarily considered when the correction is done by means of computer.

8. BANK OF ITEMS

It is good thing to have as many items as possible on a particular aspect of subject matter. If a teacher has as many items as possible on every aspect of the subject he is teaching, he will be able to pick up some from each to set a good test any time we wishes.

9. UPPER, MIDDLE AND LOWER GROUPS OF STUDENTS

Upper group of students is the top 27% of students in terms of total test score, and lower group is the bottom 27% of students in terms of total test score. The rest 46% (100-27-27+46) is middle group of students.

Example : N=100

The best 27, those nos. 1 up to 27 are the upper group.
The bottom 27, those nos. 74 up to 100 are the lower group.
Those numbered 28 up to 73 belong to the middle group, 46 candidates.

REFERENCES

Bloom, Taxonomy of Education Objectives.

Connor, D.V. and Wieneke, Chris, TERC Resource Document no.3 : The Assessment of Students

Dunstan, M, TERC Bulletin nos.5,11,12.

Ebel,R.L., Essentials of Educational Measurements

Murphy,M.J., Designing Multiple Choice