Close
Designing open book exams
This Toolkit provides evidence-based recommendations for designing open book exams and preparing students to take them.
17 February 2021
This听guidance听is informed听by听UCL鈥檚听颁翱痴滨顿听尘颈迟颈驳补迟颈辞苍听for assessment, our听operating model听for听2020-2021 centrally managed听examinations听and听the introduction of the听new听digital assessment platform,听础蝉蝉别蝉蝉尘别苍迟鲍颁尝.听
Online open book exams provide a number of opportunities to promote high-quality student听assessment.听If they are well designed, they invite students to explore and demonstrate their insights in more complex ways, replicating access to sources of knowledge which they will have in the world of work.听
However,听as students听will have unfettered access to textbooks and web-based resources, there is an increased听risk, in the absence of invigilation,听of:
- copying and pasting information;
- collusion with peers or friends; or
- seeking help from contract cheating companies.听
The points below听will听help you to think through how you design your open book exam听and support your students:
When designing the exam, consider which module/programme learning outcomes you are testing and how your questions听reflect these.听Structure questions that test students鈥� ability to听apply听analyse, evaluate, create, synthesise, interpret etc听(table 1).听 听
Structure your open book exam questions around particular problem-based scenarios or real-world cases. Provide background information on a given topic or area of study.听This gives students an opportunity to听apply their skills and knowledge.听听
Link to relevant qualitative or quantitative data and ask students to interpret and apply this听in particular ways,听e.g.:听
What does the data show?
What relevance does this听data听or听the scenario have?
What other factors could potentially affect this data?
How would you test for these?听When devising questions,听Socratic questions and questions reflective of levels and stages of learning may be useful (see table 2).听
Devise coursework-type questions that probe student understanding, skills and knowledge rather than recall ().听听
Remove听鈥榖ookwork鈥櫶齪arts of questions: definitions; standard proofs etc,听as answers can easily be found in books or online.听
Adapt your assessment criteria听to reflect the changed emphasis from recalled knowledge to applied knowledge and skills, and听ensure that students are fully informed听of them.听Check assessment criteria with colleagues so they know how they have been adapted to the open book context.听听
If possible, before the exam,听give听students听the opportunity to apply the criteria to their work in small groups to promote discussion and deepen understanding.听Ensure they practise this new听assessment format.听
Create question papers with colleagues. Look upon this process as a supportive peer review.听听
Ensure you have听good communication听channels听with students so they can get answers to their queries before and after the exams. Act on their suggestions if you can. You could听set up听a Moodle Forum for shared communication听and create a set of FAQs based on students鈥� questions.听
Reduce听unnecessary stress through the judicious use of tone 鈥撎齛ddress students directly using 鈥榶ou鈥� rather than the more impersonal 鈥榮tudents鈥� or听鈥榯hey鈥�.听
Ensure that students have access to clear, detailed instructions on the open book exam(s) (see student guidance).听
State a word count听range and听provide听clear听recommendations听for how long a student should听take to complete an exam. There is a risk that some students will听spend 24 hours writing and听over-produce.听Explain in general terms what key qualities you are looking for in answers.听The QAA notes that students require clear instructions and communications about how long they are supposed to spend on this form of exam and how their work听will be marked (听5 February 2021).听
Encourage students to use the practice area in听础蝉蝉别蝉蝉尘别苍迟鲍颁尝. They will be able to access questions, do a mock test听and听practise听uploading their answers.听听
Table听1:听Example questions relating to recall and application听
| Type or level of question听听 | Students are asked to...听听 | Example questions and starters听听 |
|---|---|---|
| Knowing and remembering听听 | Recall knowledge of subject matter relevant to the discussion.听听 |
|
| Understanding | Demonstrate understanding by constructing meaning from information听听 |
|
| Applying | Apply knowledge and understanding of a particular task or problem.听听 |
|
| Analysing听听 | Examine different concepts and make distinctions between them. |
|
| Evaluating听听 | Make judgements about concepts or ideas.听听 |
|
| Creating听听 | Develop new ideas from what they know and understand.听听 |
|
Table 2: Examples of question types听
| Type of Socratic question | Example questions and starters听听 |
|---|---|
| Clarification questions听听 |
|
| Assumption questions |
|
| Reason and evidence questions听听 |
|
| Origin or source questions听听 |
|
| Implications and consequence questions听听 |
|
| Viewpoint questions听听 |
|
Reducing academic misconduct听
Students will need to agree to a declaration of integrity听before they submit their paper to AssessmentUCL.听听听
Discuss the use of Turnitin similarity reports with your students 鈥� many students will be new to this as well as the dangers of using contract cheating sites, even in timed exams. See our student article advising against contract cheating.听
Reduce听the chances of cheating by making it very clear what kind of behaviour听is unacceptable and detail what forms of material will be permitted in the context of the open book exam听(Table 3).听
Consider ways to reduce cheating (Table 3) and share UCL's guidance on academic integrity.听
Table听3:听Strategies to minimise chances of specific types of cheating听
| Cheating activity听听 | Strategies to minimise cheating听 |
|---|---|
| Collusion: students help other students via text message or phone during exams听 | Randomise the order of questions and answer options in multiple-choice question (MCQ) and short answer tests.听 |
| Impersonation: someone else sits the exam for you | Include assessment tasks which follow on from earlier work which the student has done as part of online discussions which preclude impersonation.听 |
| Advice/support from parents and family听 | This is difficult to provide in higher education as exams are likely to focus on relatively new research. Integration with previous work also helps here.听 |
| Sending questions to contract cheating firms for help during exams听 | Watermark exam questions to make it difficult to share them.听Alert students to the implications for careers.听 |
| Searching online for answers to questions听 | Ensure that MCQ/short answer exams are fast-paced and do not re-use questions which could be found on the web.听 |
| Copying and pasting in pre-prepared or earlier texts听 | Turnitin will detect texts which have been submitted before but cannot guard against copying and pasting of听newly-written听texts. Questions built on new cases or scenarios or asking for fresh perspectives on an issue/problem can make it more difficult to use pre-prepared texts.听 |
Marking,听moderating听and feedback听
- Work with your听faculty听Exam Liaison Officer (ELO) and decide how听markers/moderators and examiners听will听have听access to scripts and how the marking process will work.听础蝉蝉别蝉蝉尘别苍迟鲍颁尝听will听automatically capture听candidate numbers on submission听and in the filename itself. Submissions听are anonymised and听can be听distributed as PDFs.听听
- Marking within the new platform is optional but there are benefits including using听in-line comments, highlighting, stamps,听drawing听tools听and easily accessed Turnitin similarity reports.听The听drawback听of marking in the new platform is听the time spent becoming familiar with it.听
- Provide information听to students听on when results can be expected and how they can be accessed.听
What is an online exam?听
- Explain what a remote or online open book exam is (see for example our guidance on Supporting your students to complete an online open book exam or , 2021, p.3).听
How to complete an online exam听
- Highlight the importance of identifying and organising any materials you plan to refer to during the exam.听
- Advise students to revise thoroughly because time constraints limit the extent of the open book resources you can refer to during an exam.听
How to use the system听
- Refer students to Examinations at 皇家华人听2021 page听concerning听the technicalities of joining the exam, downloading question papers, starting and ending the exam.听
- Encourage students to practise uploading their answers. If they鈥檙e using handwritten responses (calculations, graphs, etc) they will have to scan or take a photograph of their responses, convert these to a pdf document听and upload these.听Check听the quality of these 鈥榤ock鈥� documents is acceptable and provide advice if needed.听
- Inform students fully of what to do if they encounter technical glitches during the exam; the Examinations at 皇家华人听2021听explains the purpose of the Exam Query Form and how to听report on any issues they have encountered.听
Time allowance
- Timeframes may vary for different exams听especially for听timed exams听within a 24-hour window. Your guidance should set out if there is an additional听upload window allowed听and students听should听be听made听aware of the听reason for this time.听
- Additional time allowances and rest times: make sure that students with disabilities know exactly how their听additional requirements听will be met during online exams.听
Time zones
- International times:听provide guidance to听students who are not based in the UK听on听when to start their exams and how long they have.听
Supporting students' wellbeing听
- Signpost to UCL's Student Support and Wellbeing听鈥� a team of expert wellbeing, disability and mental health staff providing听a safe, confidential and non-judgemental space, in which students can discuss any issues that may be affecting their ability to study.
- Provide tips on creating 鈥榶our own exam conditions鈥�: quiet space, comfortable chair, suitable refreshments (听advises safeguarding against spilling drinks), access to dependable time-keeping etc. See the 'Setting up for success - what you'll need on the day'听section of our page on听Supporting your students to complete an online open book exam.听
Aston, P., Grove, M., Houston, K., & Sangwin, C. (2020).听
Clay, J. (2020).
Elsalem, L., Al-Azzam, N., Jum'ah, A. A., Obeidat, N., Sindiani, A. M., & Kheirallah, K. A. (2020). , 60, 271-279.听
Emerge Education and JISC. (2020).
Gopalan, A., & Chatley, R. (2020).
Harrison, D. (2020).
Imperial College London. (2021).
Lancaster, T., & Cotarlan, C. (2021). Contract cheating by STEM students through a file-sharing website: a Covid-19 pandemic perspective.听International Journal for Educational Integrity, 17(1), 3. doi:10.1007/s40979-021-00070-0听
Oxford Brookes University. (2020).
Quality Assurance Agency. (2021).听听
Redden, E. (2021, 5 February).
Rees, A. (2020). 听Times Higher Education.
University of Newcastle Australia. (n.d.).
University of Oxford. (2021, 14 January 2021).
University of St Andrews. (2020). 听
This guide has been produced by UCL Arena Centre for Research-based Education.听Tables 1 and 2 are kindly听Reproduced from University of Newcastle Australia.听听You are welcome to use this guide if you are from another educational facility, but you must credit the project.鈥�