Item setting: A story by Professor Ho Man Koon

This story was written by Professor Ho Man Koon. The original version, in Chinese, can be found in a book titled “My Teachers”, which he published in 2009. For further information about the book, please contact Professor Ho Man Koon at the Faculty of Education, The Chinese University of Hong Kong. I am grateful to Professor Ho for allowing me to re-present his story in English.

I have also presented this story as the second episode of my CUHK podcast “Teachers’ Stories”. To listen, go to the following link, click on “Item Setting”, and then the ‘play’ icon.

http://podcast.cuhk.edu.hk/teacherstory/index.htm

=============

Item Setting

Ho Man Koon

This story happened in 1978, when I was studying for the Diploma in Education at the Chinese University of Hong Kong. One evening, after finishing dinner at the student canteen, I rushed to the teaching block, to help Professor Siu Ping Kee with setting questions for a research project. I was pleased about the opportunity to help him with his research on Chinese teaching.

The teaching block was quiet. The door to Professor Siu’s was open. There was light coming from his office. Inside, Professor Siu was fully engrossed in his work. I entered his office, and after listening to his instructions, I set out to draft the questions.

I said to myself, “Well, what could be easier than setting a few multiple choice questions. All you need to do is to think of a question, then four answer options, including one which is the correct answer. And that’s it!” So I got down to work, and in less than an hour, I had everything done. I thought, “Professor Siu will be quite pleased with my efficiency.”

What happened next took me by complete surprise. None of the items were accepted by Professor Siu, who explained to me: “Item setting has to satisfy certain requirements. Each item has to have a specific objective. Is it assessing analytic ability? Is it assessing inferencing? Is it assessing application? Even for the same grade level, if you have a different objective, you will need a different question.” Professor Siu continued to explain, “Of the four answer options, the correct answer is the easiest to think of. But coming up with the other three distracters is a real challenge. They cannot overlap with the correct answer. They cannot be so far-fetched that students can tell them apart easily. Otherwise the question would have no discriminatory power.”

Well, if I were to listen to these principles in a lecture, they probably wouldn’t ring a bell with me. But at that moment, Professor Siu’s reminder had a great impact on me. I realized the problem, and decided to start all over again. When I completed the first question which was finally approved by Professor Siu, it was already one hour before midnight.

When I left Professor Siu’s office, the teaching block was as quiet as ever. There was still light coming from Professor Siu’s office. I could see that he was still working, and then I realized that I had forgotten to ask him whether he had had dinner or not. Well, probably not, because once he was absorbed in his work, he would forget everything. Looking at the light coming from his office, I thought of his total dedication to his research, his patience when guiding me to set the multiple choice questions. At that moment, my mind was filled with respect and admiration for him. On the surface, I was there to help him with the item setting. In reality, he was helping me to learn. That evening, Professor Siu had opened the door of his office for me. It was also a door that led to my later journey in research. That night, he taught and inspired me, by example, as to what research was all about.

***

PS: Professor Siu Ping Kee was the Dean when I joined the Faculty of Education in 1991. I was interviewed by him and a few other faculty members when I applied for the job.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s