Mathematics Take Away Open Book Assessment

With the current Covid-19 crisis many Higher Education Institutions (HEIs) are moving to take away exams which by their nature are open book exams. Many mathematicians do not have experience of take home open book assessments and this post is intended to help deal with this.

This has been put together quickly and is intended to be practical rather than theoretical. I hope you find it of use. I will update when I can so if you have any suitable links or ideas, then please leave a comment.


Principles

  • Throughout the next few months, students and staff will be sick, caring for the sick, living in isolation, perhaps mourning loved ones, and generally suffering from the stress and strain. Keep this in mind.
  • Mathematics lecturers and students are generally not familiar with take home or open book assessment and will need more instruction/guidance than usual. Furthermore, if they had known they were to be assessed in this way, then they would have approached their learning differently.
  • Assessment is likely to be online. Students sending scripts through the post is generally infeasible as HEI campuses are likely to be closed.
  • Plagiarism, collusion, and impersonation are difficult problems that will require thought (and likely some compromise).
  • Keep it simple.

Practicalities

  • Online assessment is not straightforward. Some HEIs have suggested assignments should be submitted in, say, Word. However, it is not unreasonable for students to submit handwritten work. See How to have students submit handwritten work in an online setting by Robert Talbert.
  • Gradescope are (at time of writing) offering free access to new courses. Gradescope is worth investigating if you have not yet seen it. It was invented by some tutors tired of marking mathematics. It can help with assessment of handwritten mathematics!
  • Involve external examiners. They will be dealing with the same problems and often have useful advice and experience.
  • Make it clear to students what assessment will be used and what a good answer looks like.
  • In an ideal world, students should be given a model paper and a mock paper to try. The model paper shows them what a good answer looks like. The mock paper should be administered like the real one. This gives students a chance to experience the log in, upload and so on so that they are familiar with the system.
  • Length of time for take away: Students may be in different time zones, not have access to equipment, work in noisy, shared spaces (siblings may not be at school), have special needs, and so on. Hence, a short timescale for uploading answers is problematic. A time of at least 24 hrs can mitigate many such problems.
  • Shorter exams can help reduce the opportunity for plagiarism.
  • Longer exams means asking for answers in Latex more reasonable. Such answers can be checked by plagiarism detectors such as TurnitIn.
  • Traditionally, student work which appeared not to be the work of the student could be checked by giving the student a viva. In the current situation this will be difficult and limited by resources. It may not be possible to interview all students on all modules. Also, conducting a viva by video conference is significantly harder than in person as mathematics may need to be written down and held up to the camera. Furthermore, the resulting picture may be low resolution due to bandwidth issues.
  • Consider whether you are assessing for pass/fail or giving a mark. Obviously a mark unseen by students can be useful to determine pass/fail. It is important to keep the marks in case of appeal.
  • Check with accreditation bodies what their policies are during this crisis. In particular, their advice may conflict with long time frames for assessments.
  • Many students have special needs to consider and may need more time to complete assignments. Hence, a system releasing an exam and expecting all students to complete at the same is likely best avoided.
  • Times stamps on uploads can be used but these can be easy to forge so care needs to be taken.
  • Mathematics has a higher proportion of neurodiverse students and this may need special consideration.

 

Setting take away open book exams

  • The simplest solution to the problem of setting a take away examination is to use a standard examination with reduced marks for bookwork (e.g., state the definition/theorem, give the proof of famous theorem). This clearly is less than desirable.
  • The next simplest solution is to remove the bookwork but this may mean not all the learning objectives of the course are assessed.
  • Do a web search for answers to questions you set. Assessors should do the same.
  • Plagiarism can sometimes by avoided by setting questions with different coefficients for different students.
  • Mixing the order of questions between students helps combat collusion but makes marking and collating the results harder. (One can’t just tell a marker to mark Q1 as there is no consistent Q1.)
  • Now for types of questions one can ask.
  • Ask “Why does the proof of a theorem go wrong for this particular example?”.
  • Students create an example that meets certain constraints and show it really is an example. This is hard to set if you have not set this type of question before. E.g., Give an example of a function with singular point x=3. The student answer f(x)=0 is valid and can show that the student really understands the definition. However, some students realise that using the zero function and other trivial examples often gives a suitable answer.
  • Similar questions can be written asking for non-examples, i.e., examples which don’t satisfy a certain set of constraints.
  • Ask for a book review. This has to be an online text to which everyone has free access to. This can be done for a textbook. Students demonstrate how much they have learned by identifying weak parts, good parts, or describing the contents of a chapter in their own words, etc.
  • Some questions can be for discussion. For example, “A metal bar is heated in the middle. Discuss.” This type of question needs good mock examples and very strict instructions on length and scope. Very easy to set but also very easy for students to misunderstand.
  • Ask students to explain the most important theorem (or example) in the course explaining why they think it is the most important.
  • Use incorrect answers. Either create your own or (quicker) use exam scripts from previous years and (suitably anonymised) present a student’s incorrect answer that the current students can critique. Number each line and not just equations.
For an example, see this example of questions from Philip Walker associated with the MATH1010 module at the University of Leeds.
 

Other links

Get the newsletter!

Fancy a newsletter keeping you up-to-date with maths news, articles, videos and events you might otherwise miss? Then sign up below. (No spam and I will never share your details with anyone else.)