Thursday, July 25, 2013

Professor for one year (week 15): Exams and why to prefer computer-based ones

Finally, last week was the last week of teaching.  In the seminars and lectures, we summarized the topics of the classes and tried to understand what the things learned could be used for in the future.  And for the Perl course, there was the final exam.

With Bologna, students get credits for every course they attend.  And to give credits, lecturers have to assess students' achievements.  To make this sound not as scary as "exams", Switzerland uses "Leistungsnachweis" (evidence of academic achievement) instead of "Prüfung" (exam), however, in Germany these things are still called exams.

To assess students' achievements, one can use a variety of tasks: the more traditional ones like oral exams, seminar papers, presentations, written exams; more modern ones like multiple-choice tests as a form of easy-to-assess written exams; or sophisticated ones like e-portfolios.  To choose an appropriate form of assessment, lecturers should consider the topic of the course -- there is no sense in writing a seminar paper when students should learn how to microscope or how to program.  The exam should be a task similar to the ones students had to solve during the course.  The lecturer should assess the knowledge and the skills student had to acquire during the semester. 

So for the programming course, students learned the basics of Perl and they had to program a lot.  As computational linguist, Perl is a perfect language for preprocessing data you would like to analyze or for extracting information from analyses.  So that's what we did during the course: opening and reading files, extracting words and sentences to count and order them, cleaning HTML files, printing to files, etc.  Students had to solve assignments during the term, making up 40% of the overall grade.  They could use whatever help they could find -- and it's pretty easy to detect copy-paste solutions from Google, just ask students to explain their submitted code in front of the others.

For over 10 years now, I conduct the exams for programming courses on the computer.  This way, the exam is very much like an assignment they had to solve the weeks before.  And students can test their code before submitting -- I don't assess code that is not running.  When I studied, I had to program on paper for exams, and when I came to the University of Zurich in 2001, this was still the case for the programming courses at the Department of Informatics.  However, in real live, you will never ever program on paper, so there is no sense in having an exam like this.

For the Perl course, students found two input files in a folder on the Learning Management System (LMS) and they had to submit their solution to this folder.  Using an LMS during the exam means: There is an Internet connection.  This means: You can google a lot!  But usually this doesn't help unless you have understood the task at hand to reformulate it yourself.  And in this case, you are almost certainly able to solve it without help -- except for looking up which bracket has to go where.

The results for the assignments during the semester haven't been too bad, although students reported that they had spent a lot of time for solving -- which was intended, you have to spend around 20 hours a week to really earn those 9 ECTS points.

For all programming exams, I have a more theoretical (or knowledge) part and a more practical part.  The theoretical part is easy to answer if students get aware that I start each lesson with a short recap of the lesson before -- the questions just come from these parts.  There seems to be a strong correlation between gender and success rate in the theoretical and practical parts: Women profit from having memorized for the theoretical part, men profit from having stamina and finding creative workarounds for the practical part.  Yes, programming is a lot about failure, and you have to practice a lot to feel comfortable with a new language.  To learn programming is a lot about to learn from your own errors and to learn not to give up, but try again and find a solution for the given task -- there is always more than one solution.  I'm not sure if this really correlates with gender or rather with being more a linguist or a programmer -- preferring humanities or engineering -- but as a computational linguist, you are both, so you have to build stamina to get this code running and providing you the information you need.

For the theoretical part, I prohibited the use of the computer -- that would have been too easy -- and students had to answer some questions on paper.  And there I understood why paper-based exams are really bad.  I had prepared sheets of paper with the questions (printed in 12-point font size) and enough space to write the answers.  Students wrote in even bigger font size, so the space was big enough, indeed.  12-point font is fairly readable. However, students almost wrote with their nose!  They bent to the table creating terrible hunchbacks -- it would have been a good demonstration for physiotherapist, I almost felt back pain myself.  For the practical part, however, everybody set straight, looked at the screen and sometimes at the keyboard and the paper with the task.  A much healthier impression. 

Isn't this a strong argument to abandon handwritten exams at all?  And for the final exams towards your BA or MA degree, you have to write 4 or more hours by hand.  Not to mention the poor examiner who has to read handwritten essays produced under stress by unaccustomed writers (when was the last time you wrote more than one page by hand?) -- it looks like a mess, the writing gets poorer towards the end and every page is sprinkled with revisions and annotations pointing to extra sheets, etc.

Oh, and I would love the integration of interactive checkers into an LMS.  Students get feedback from the programming environment on syntax errors and static semantic errors, but interactive checkers would also give feedback on semantic issues, thus facilitating the learning process for students as well as the assessing process for lecturers.  Combined with automatically assessible multiple-choice, short-answer, or sentence completion tests (yes, you can assess all levels of knowledge and skills with appropriate use of those tests, too), I would be freed from grading.

No comments: