The dream for an automated composition grading engine

One of the ‘drudgeries’ of being a teacher of English, especially being one in Hong Kong, is the colossal amount of composition grading that comes with the job. In most schools in Hong Kong, teachers are expected to grade each and every composition written by students carefully. Grading here involves two tasks: scoring a composition according to a set of criteria; and responding to the composition, very often by correcting each spelling or grammatical error a student has made. With our still very large class sizes, especially in secondary schools, and the frequency at which teachers are expected to assign composition tasks, you can imagine how energy-draining composition grading is for teachers in Hong Kong. Despite calls by academics in L2 writing for more focussed grading so as to lighten teachers’ marking burden, this has not been widely followed in schools. One objection comes from school managers, who fear that focussed grading (as opposed to ‘thorough’, comprehensive, grading) may inadvertently gives students and parents the wrong impression that the teacher is ‘lazy’.

As a former teacher at 4 secondary schools, I understand the plight of teachers. Weekends are not for winding down, but for catching up with the composition grading that has been backburnered. I also know that what is most frustrating for the most conscientious teachers who are willing to sacrifice their weekends for marking compositions is that students keep producing the same errors. Hence, I am in full support of focussed response that prompts students to re-scrutinize their writing. This way, there is a greater chance that they will learn to write better. Granted there will be other errors not in question that will be left out, but learning to write is not a mechanistic process; as long as students keep on reading, and have a strong-enough interest in writing, their writing WILL improve. Teachers’ underlining every error in a composition and correcting it for students will only work under certain circumstances. There is also the question of how best teachers should use their time. For example, what about planning interesting and effective lessons?

As for the alleged objection by parents, this is where we should keep up our professional stance. Instead of succumbing to parents’ layperson  perception that the more red ink the teacher splashes on a student composition, the better (because at least the teacher is ‘hardworking’), schools should explain to parents what IS really best for their children, and this includes how children become good writers.

Anyway, to go back to my main topic, a major talking point in Computer Assisted Language Learning (CALL) is whether one day, composition grading can be entirely done by computers. My understanding had been that yes there were a couple of essay scoring machines, such as the e-rater Engine of ETS (Educational Testing Services), that had been developed, but they could only score mechanical, low-level, aspects of writing, such as spelling, punctuation, range of vocabulary, language form (e.g., ‘third-person’ singular in Simple Present tense), etc., not to mention the ability to RESPOND to a piece of writing. Six weeks ago, an acquaintance working in publishing assured me, during an informal chat, that the Intelligent Essay Assessor developed by Pearson was a highly sophisticated composition scoring engine that could be put to meaningful use.

I did not have much faith in that assurance, so I put the information aside. This morning, I thought, well, why not just check it out! The technology might have developed beyond my imagination! So I spent half an hour browsing IEA, half hoping for some miracle.

But there is no miracle. It might have made some advances over previous essay scoring programmes, but first, its application is still limited to certain learning situations, and cannot be freely used throughout the entire spectrum of school education; second, it still cannot RESPOND to what students have written, for example, by correcting grammatical errors that students have made or suggesting better lexical choices and sentence structures.

So, this is both good news and bad news for teachers. Bad news because they still have to burn the midnight oil toiling through student composition after student composition. Good news because their job security won’t be jeopardised at least in the forseeable future.

In fact, given the complexity of human language, I wonder whether a totally reliable and valid composition grading engine will ever be invented one day. One only has to look at Google Translator to get a sense of that complexity.

http://mywritinglab.com/assets/documents/IEA-FactSheet-20100401.pdf

http://www.ets.org/erater/how/
http://www.insidehighered.com/news/2011/02/21/debate_over_reliability_of_automated_essay_grading#ixzz1n8o7EhC4

Advertisements

3 thoughts on “The dream for an automated composition grading engine

  1. Dr Sze, Thanks for your recommendation and sharing! I have never thought about composition marking can become an automatic process by making use of those software. Though they may be useful in spotting lexical and syntactical mistakes, I wonder how rating could be given towards aspects like creativity, relevance of examples, feasibility of ideas or critical thinking towards a problem, etc. Perhaps use of technology is just a supplement to teaching rather than a substitution.

  2. Hello. Am retired tchr. now but so recall hours of reading/grading compositions. Yes would agree some computer-assisted help would be appreciated as I as teacher seemed to rake in the major benefit of error spotting/correction. In the 1980s our school-district shifted to peer edit partners and groups before turning in an assignment. This really did help to catch the pesky, minor errors (capitalisation, punctuation, etc.) which meant I got to focus more to content. Is there time for some peer edit go-rounds? Thanks for post.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s