Designing Open-note Exams

Giving an exam online requires that you change the way you proctor, design and deliver the exam. One way to transition into giving online exams is to move from a more traditional closed-book exam to an open-note exam.

This is recommended for a number of reasons. It is difficult, if not impossible to ensure your students do not use outside materials while taking your exam. Designing an exam that responds to this situation rather than ignoring it places your students on a more even playing field.

There may be a concern that open-note exams will lead to higher grades from students or a lack of long term retention of content. However, a study conducted in a Psychology classroom indicated that while students may initially score higher when using notes, the long term retention was statistically the same between an open-note exam and a closed book exam (Gharib A., Phillips W. & Mathew N., 2012).  This can lend some relief to instructors that a well designed open-note exam can still be an effective assessment and testing tool for students. Other studies have found that student anxiety is reduced for open book exams.  Finally, a team of accounting instructors found that an “open-book exam approach is useful for enhancing student learning while effectively preparing our students for real-world operational decision making” (Green, S.G., Ferrante, C.J., Heppard, K.A, 2016).

General suggestions for Writing Open-Note Exams

Clearly communicate with your students. Identify what ‘open-note’ means in the context of your class. Have a conversation with students about what materials you want them to use and what ones you are asking them to avoid. Are you comfortable with them using their notes and textbooks but not communicating with each other? Don’t assume that they have the same definition of ‘open-note’ as you. Consider writing an honor policy (by yourself or in collaboration with your students) that students sign when they take the exam. For example language and details on the university’s policy please consult the Washington University Undergraduate Student Academic Integrity Policy.

Start with identifying the learning goals you want to assess. All assessments should be grounded in the learning goals for your course and open-note exams are no exception. Think critically about what learning goals you want to assess now. It is possible that these have changed slightly from your anticipated goals for the semester. Open-note exams are better suited to learning goals that ask students to apply knowledge or analyze situations rather than identify concepts or provide definitions.

Consider your grading capacity. Open-note exams do not need to involve short answer responses but they can. It is possible that you want to ask students to articulate reasoning behind a question now that you know they have access to definitions or a text. However, it is important to be realistic with yourself (and your Assistants in Instruction) about your capacity to grade student responses.

Be aware that these questions will be readily available to future semesters. By giving an exam online it is reasonable to assume that future students will have access. This is not particularly an issue for this semester but just to keep in mind for future semesters.

Consider randomizing question order. This can be done in Canvas where you upload a test bank of questions and students see them in a random order. This can mitigate some concerns about students working through the exam together.

Consider releasing your exam for a set amount of time. Canvas allows you to release an exam for a specific amount of time as well as set the amount of time students have to complete the exam once they have opened it. For example, you could open the exam for 24 hours and allow students 90 minutes to complete the exam once they begin. This allows students to determine the time of day that best suits them to focus on the exam.

Writing Multiple Choice or Fill-in the Blank Questions

Avoid questions that can be directly Googled. For example, use more scenario-based questions rather than pure definitions. Ask questions that encourage your students to apply or analyze knowledge rather than remember definitions. Don’t pull questions directly from testing banks that are readily available online. Grading fill-in the blank questions can be semi-automated through Canvas.

Write complex questions that require students to apply knowledge. Consider scenario-based questions that require students to apply knowledge to a novel scenario. This allows students to rely on the materials they have on hand while still assessing their understanding of the content. This can come from designing a question stem that relies on application or an understanding of multiple concepts covered in the course. It can also be done through providing answer choices that require a high level of understanding to discriminate between them (Brame, C., 2013).

Writing Short Answer Questions

Write questions that are clear and concise. The key with effective assessment questions is that you want students to spend their time answering the question and not trying to figure out what you want them to do. This is also relevant for writing multiple choice questions but particularly the case for short answer questions. Make it clear to students your expectations for length and detail of their response.

Write questions that probe higher order thinking. Short answer questions lend themselves well to an open-note situation because they often ask students to provide reasoning, articulate understanding or apply knowledge to a novel scenario. Lean on these kinds of questions to ensure that even students with access to notes will be required to demonstrate knowledge they have learned throughout your course.

Answer the questions yourself and (if possible) have a colleague answer them. It’s important for you to have a clear understanding of what you want students to provide as an answer. Having a colleague, Assistant in Instruction or Undergraduate TA provide answers will also give you insight into the complexity and clarity of the question.

Create a grading guideline. Think through how you will assign credit for these questions. Create some sort of grading scheme that you will use and share with any other graders. This could be a rubric or simply a list of expectations for correct answers. This will help you streamline grading and ensure consistency.

Consider how you want students to respond. Canvas allows students to either type in a response or upload a file. If you teach a course that relies heavily on math-based problem solving asking students to write out answers and upload a photo will be less cumbersome than trying to use the equation writing tool. This could also be helpful if you want students to be able to draw diagrams to indication relationships.