Reflections on my teaching: Assessments and Assignments
3 April 2022
How to set assessments and assignments that are useful for the students and not painful for the lecturer?
The problem(s)
One of the most challenging aspects of teaching is the setting of questions for exams and quizzes. Coming up with good assessments that test higher order thinking skills and not only students’ ability to memorize is often difficult and time-consuming. What makes matters worse is the need to come up with new questions every new semester or academic year. In this article, I intend to discuss these challenges and describe solutions that have worked well (and not) for me.
With so many important tasks on our plate as lecturers, the time required for the setting of questions and marking of assessments is a major consideration. To deal with this demand, lecturers tend to employ two different strategies. One strategy is to design memory-based multiple choice questions. This approach requires moderate time-investment for the setting of questions. The evaluation is rather simple and can often be automated or delegated. However, this approach comes with a number of major disadvantages. Firstly, it is not easy to come up with multiple choice questions that have plausible wrong answers, and to do so repeatedly each semester. Even more importantly, multiple choice questions have the major disadvantage that they commonly test the ability of students to memorize knowledge and are not very suitable to test the development of higher order thinking skills. And this is important, because how students learn is dependent on the assessment mode. If the assessment is based on memorizing, students feel little motivation to practice other skills.
Another common strategy is to design assessments or assignments based on open-ended questions. This approach frequently does test various higher order thinking skills and hence promotes student learning. Another advantage is that coming up with open-ended questions is fairly easy and requires little time. However, the major downside is that the marking is usually very difficult, time-consuming and subjective and leads lecturers to anticipate the marking period with agony.
Like many lecturers, I have experimented with different types of open-ended assignments and assessments over the years. For instance, in my large class undergraduate life science course, which used to have 300 students, I have in the past included a writing assignment. In the assignment, I asked the students to write a commentary on a research paper. The assignment was very easy to set. I only had to find a suitable research paper related to the course content and create some guidelines. The assignment likely also helped the students to develop independent learning skills. But it was extremely time-consuming to mark.
In an attempt to be more accountable and promote student learning, I tried to send short feedback emails on the assignment to each student. This turned out to be extraordinary difficult. The difficulty arose from the fact that although it is easy to realize that the writing is not comprehendible, finding out what makes it difficult to understand a sentence or paragraph is challenging. Thus, thorough marking of written assignments and provision of meaningful feedback not only require knowledge about paragraph and sentence structure, but also a lot of effort and concentration. As a result, this assignment was very exhausting for me and not sustainable in the long run.
When evaluating open-ended assessments or assignments, it is also very difficult to provide objective marks. What I consider a good assignment is probably different from what another lecturer considers excellent. And even my own standards tend to drift depending on my concentration level, my mood, my sympathy with the student, and whether the prior assignments that I marked were of high or low quality. If I have marked a series of very good assignments or assays, my standards tend to increase, and vice versa. And so if after finishing to mark all assignments in a class I go back to the first student papers, I often discover (in horror) that I no longer agree with the marks that I originally assigned. I am then left with the agonizing decision to either re-mark the papers or accept imperfection. For all these reasons, I have never been comfortable with grading of open-ended assessments and assignments.
Given the specific shortcomings associated with writing assignments, I have tried to implement different types of assignments, such as a diagram drawing assignment and a video assignment, for which the marking is less intellectually exhausting. Both types of assignments help students develop important skills. In the digram assignment, students are given the task to draw a cellular pathway. In order to complete this task well, the students need to be able to understand information independently and then process and synthesize the information to present it in a different format. The diagram assignment also promotes creativity and some practical skills. In the video assignment, students have to identify a scientific question and then discuss in an engaging video the background, why the question is important and explain the solution to the problem (or describe how they might find the solution). The video assignment has similar advantages to the diagram assignment. In addition, it promotes presentation and video editing skills.
The marking of these assignments is less demanding compared to an essay. Both the diagrams and videos are visual, enhanced by words or spoken language, and are hence easier to comprehend. They often are also very interesting. And a few simple steps can make the marking even easier as well as more objective. The first step is to come up with a suitable marking rubric. The second is to announce this rubric to the class and give clear and detailed guidelines of what a good assignment looks like. This avoids the common pitfall where students do something different from what we had intended. When student works do not meet the intended objective as a consequence of ambiguous instructions, we run into the problem of not knowing how to mark the assignments or assessments. Hence, it is really important that the students know what constitutes a high quality work. The best way to illustrate to the students what we want them to do us by providing examples. However, I tend to not use examples because they often stifle students’ creativity. Instead, I prefer to give very clear verbal instructions, but leaving enough space for flexibility and creativity. As a final step, I have converted the video assignment from an individual task to a group project. This step not only reduces the time required for marking, but also helps students to develop skills to work well in a team.
Despite all this, I still do not look forward to the marking process. The main reason for that is that marking open-ended assignments feels like a dead-end activity. Even though I provide some feedback to the students (which helps me to ensure that I put enough effort into the marking process), I know from experience that the students do not really use the feedback to improve (if they don’t have to) and only care about their marks or grades. And I can sympathize with this sentiment, given that the students likely have many other graded assignments, tests and other tasks on their plate. Those naturally take priority over revising something that is of no immediate practical consequence. The problem is of course that if student do not invest time to learn from their mistakes, it severely limits the usefulness of open-ended assessments and assignments.
In summary, the two commonly used approaches of assessment setting in higher education, multiple choice questions and open ended questions or assignments, both have major shortcomings. Multiple choice questions commonly do not test higher order thinking skills and open-ended questions are difficult to mark in terms of effort and objectivity. In the way in which both assessment types are commonly used, they do not effectively promote student learning.
The solution
There probably is not just one solution, but below I have described what has worked for me and what hasn’t.
When I faced the challenge of trying to come up with meaningful courses, one of the first questions I asked myself was this: What can I offer students that they cannot find in textbooks or online sources and that is at the same time useful for the students. The answer to this came quite easily when I considered what my unique strength is. I am primarily a researcher. I reasoned that there must be a reason, other than budget constraints, why research intense universities engage their research professors to teach students. Research professors, or lecturers with research background, are in the best position to prepare students for real life challenges they will face as a scientist. After this realization, I started experimenting with introducing research data and papers into my teaching. And thus, in my undergraduate year 2 life science course, I have gradually changed the emphasis from transmitting knowledge of cell biology to understanding concepts and applying these to real life research problems.
One immediate objection may be that this approach will not help students who do not want to become scientists. There are several things that can be said about this. Firstly, the reason why students do not want to become a scientist may be precisely because they have not been exposed to real research. The opposite may also be true. After being confronted with research data and problems, the students may make the conscious decision not to pursue becoming a scientist, which is also a much better outcome than if the students were to make uninformed decisions.
Secondly, it is worthwhile to consider the alternative, which is that we teach students textbook knowledge. The students usually master acquiring the content well as they are motivated to do well in the exam. However, they likely quickly forget what they have learned. And this is not a major problem, because they will be able to find all the information easily by themselves online, if they every needed to. This further demonstrates the futility of this approach. In contrast, introducing to students basic concepts and principles as well as research methodology and then letting them apply this knowledge to research data-based problems is a skill that they likely do not lose so quickly. And based on my experience, I must say that this approach has resonated with most students extremely well.
The approach of using research data and research papers does in fact help students to acquire a number of very useful skills. Most importantly, students learn how to interpret scientific data. This is a skill that is expected of a life science graduate and that is very useful for more advanced courses, for actual research work that the students might engage it as well as in many professions that life science graduates pursue. And even as a non-science professional, being able to access primary literature and evaluate information is useful, especially in today’s landscape of sensationalism and fake news. The approach also helps students to develop analytical and critical thinking skills, which are important transferrable skills.
Ultimately, what we want to achieve with our teaching is to equip students with the the tools to be able to learn on their own and to understand and solve complex problems. As lecturers, we often function as a translator of scientific research data into knowledge bites that students can easily digest. By focussing on understanding concepts in depth using research data, we provide students with tools to learn on their own from the primary source of scientific knowledge. We also build confidence in students and reduce their anxiety or apprehension to access primary research to learn independently.
An important outcome of the research data and paper based teaching approach is that students gain a better understanding of how research works, that it is an ongoing process and that there are sometimes contradictory findings in the literature. The students learn to appreciate that we cannot obtain definite answers from a single study and that there are still many unanswered questions, emphasizing the need for further biomedical research and funding for it. Finally, and perhaps most importantly, the use of research data and research papers proved in my experience to be the best approach to make student learning problem based as well as engaging and fun.
Teaching and assessing students based on research data has also resulted in improvements in my numerical and verbal student feedback. Students often express that they appreciate to be learning relevant skills. Students also appreciate that they do not need to memorize any information to answer research data based multiple choice questions. And in fact, all my assessments and exams are open book tests because the skills that students need cannot be found in the personal notes or in other materials.
One important consideration is the time and effort that the lecturer needs to spend to prepare the questions. In my experience, setting of multiple choice question based on research problems has made it much easier for me to come up with challenging questions with plausible wrong options semester after semester. There is a large amount of relevant research available and I often also invent results and scenarios. To reduce the effort on my side, I often re-use good questions from graded assessments in subsequent semesters for non-grades quizzes and in-class activities. Importantly, my lecture preparation consists to a large part of reading research papers to find practical examples. Being a scientist, this is something that I enjoy. What is more, it also helps my personal development tremendously. Needless to say, it does take time to set multiple choice questions based on research problems. But I personally enjoy coming up with interesting and challenging questions. And because the assessments are multiple choice questions, the marking process is usually automated and requires little effort on the side of the lecturer.
Below I have summarized some tips to make this approach of assessment successful:
1. It goes without saying that the teaching mode during classes and the exam mode should be well-aligned. In other words, the teaching activities during class time should prepare students for the final exam. Or put in another way, the assessments and exam should test what the students were taught. As such, most of the in-class time in my course is being spent on research data analysis and research data based problem solving. To free up time for this, it is tremendously helpful to use the flipped classroom approach and prepare some videos on basic content and concepts that the students watch before the lecture. Although the recording of these videos is initially time-consuming, it pays tremendously in subsequent semesters.
2. In my formal assessments and the final exam, I routinely let the students read one research paper that is related to the course content. The students are given the specific research paper weeks in advance to prepare for the assessment. In the assessment, all or a significant portion of my questions are based on the figures of the paper (see below for the type of questions asked). This approach has a number of advantages. Firstly, students know what will come up in the exam and they do not need to memorize knowledge. Understanding a research paper is an actual application of what the students have mastered to learn. Students often feel a sense of accomplishment when at the end of the course they are able to understand a research paper. Finally, using a new research paper allows me to come up with new questions every semester. In actual fact, I tend to ask similar questions every semester, but because the data are different and taken from a different context, it is not possible for students to benefit from knowing previous exam questions, unless they make the effort to read and understand the respective research paper.
Nonetheless, the approach does come with some downsides. It is not always easy to find a suitable research paper, although I have so far always managed to do so. It is also necessary to explain some of the methodology of the paper. Despite this, I personally consider this the best possible exam format, and and after using this approach for many years, I would not want to go back to a conventional way of asking questions and be in agony every new semester to come up with good new questions.
3. It is critically important to write out the answers and answer explanations before any assessments. This helps tremendously in avoiding any ambiguities. By writing the explanations, we are confirming whether our own train of thought is logical or has any flaws It also helps us to examine if any alternative interpretations are possible by trying to put ourselves into the shoes of the students and consider how they might understand (or misunderstand) the question. I also always make the answers and explanations available to the students immediately after the assessment or exam. It is a well-known fact that immediate feedback is essential for effective student learning. The ability to provide immediate feedback that students care about is in fact another advantage of the research problem-based approach compared to open-ended questions, where feedback can only be provided after a long delay when students likely have moved on and may not even remember the assessment well. Although revealing the answers and explanations means that I cannot re-use these questions for future formal assessments, I do re-use the good questions as practice questions in subsequent semesters, as discussed above.
4. To allow for more flexibility, I also use some question formats other than multiple choice questions in the assessments and final examination. Based on my experience, a very good question format is the drawing of diagrams to predict experimental results. For instance, I ask students to draw expected bands in a Western blot panel, cell cycle profiles, bar graphs for specific parameters etc. In order to answer these questions, the students need to have an understanding of the principle of the assay as well as the specific applied concepts. Diagram-based questions are not only very suitable to test higher learning skills, they are also easy to design. On top of that, marking of drawn diagrams is visual and hence requires minimal effort by the lecturer.
I often also include some short answer questions in formal assignments. Here, in order to make the marking process easy, it is important to ensure that the students answer the actual question and only the stated question. This can be achieved by specifying the number of potential or required answers (ideally only one or few) and setting of a word limit. This approach also helps students to focus on the most important points and to practice expressing themselves concisely.
Open-ended assignments
So far I have discussed research data-based multiple choice questions and diagram and short answer question formats. All of these assessment formats can test higher order thinking skills and are relatively easy to mark. These types of questions are not exactly easy to set, but the process of coming up with questions based on research problems (which consists of reading research papers, thinking about research problems and finding ways to challenge students) is something that I personally enjoy. This teaching approach has also benefited me greatly in my research and enabled me to better interact with colleagues scientifically. However, ideally we would also like to complement these question formats with open-ended assignments or assessments. In this section I want to discuss how I set and use open-ended problems and where I do not use them.
As discussed above, in open-ended assessments, the problem is not so much coming up with the questions. It is generally easy for lecturers to come up with good open-ended questions that test higher learning skills. The main problem with open-ended questions is the marking. Marking open-ended questions is often tedious, time-consuming and energy-draining, even more if one intends to provide feedback to the students.
Marking open-ended questions is often also not particularly objective and consistent. Every lecturer has different ideas of what a good assignment might look like, depending on their own background and experience. Every lecturer also has different standards. And even our own standards fluctuate and drift over time. Of course, we can and should come up with a very concrete marking rubric, which increases the objectivity to some degree. It is important to let the students know before the assignment what the marking criteria are and what a good assignment looks like, for instance by providing an example (although care must be taken to avoid that the students do imitate the example). But nontheless, marking open-ended question has usually not been a good experience for me, and I have never looked forward to marking traditional open-ended questions.
However, the main problem with the marking of open-ended questions is that there is no meaning in it, other than assigning a mark or grade to each student. This makes the marking purposeless. Even if I try to give students feedback, I know from experience that most students do not sit down and analyze the feedback. All they usually care about is their grade. Needless to say, this severely limits the usefulness of open-ended questions.
How then can the marking dilemma associated with open-ended questions be addressed?
One often-suggested approach is peer assessment. I believe that peer assessment has value in promoting learning on the side of the assessing student. Peer assessments can help students to develop skills in critical analysis, to learn from others, to self-reflect or to realize what a good assignment can look like. However, students are likely unable to assess other students very accurately. Major reasons for this are that their own understanding could be limited and that the standards of different students vary greatly. Although there are ways to mitigate this (employing multiple assessors for the evaluation of each student,limiting the grades that students can assign, letting students rank others as opposed to evaluating each student independently, etc.), it is likely that relying on peer assessment for the evaluation of high stage assignments will be viewed as unfair by students. Finally, peer assessment is also less likely to be able to provide consistent and reliable feedback.
Another option is to employ multiple instructors to mark open-ended assignments and assessments. This is in fact often done, and to reduce the impact of different standards, different instructors grade different parts of an assessment or assignment. However, this approach does not really address the fundamental problem that marking of open-ended question is generally tedious, subjective and only serves the purpose of assigning grades.
One approach that I have used is to make the marking of open ended assignments and assessments less tedious and more interesting. For instance, what helps me tremendously to increase my interest level is knowing the students by their names. If I know who the student whose assignment I am marking is, I am naturally more interested in finding out how the particular student did. It is like following a sports event where you know all the players as opposed knowing none. This is one reason why I try to learn all student names, even if the class size is large. Another approach I have used is to make the assignments and assessments more visual, such as letting the students prepare videos or draw diagrams, as discussed above. Finally, I feel somewhat more motivated if marking assignments or assessments is contributing towards my own development. For instance, continuously evaluating assays and presentations has really helped me to be able to identify what are the common mistakes that students make and how these can be addressed.
However, all these approaches only mitigate the difficulties associated with marking open-ended assignment and assessments. They do not address the fundamental problem that the marking mainly serves the purpose of obtaining marks and grades and that there is no or only little formative learning for students. Hence, I believe that the best approach is to move the main evaluation to before the final submission of the assignment or the final assessment. Evaluating the students early facilitates formative learning, while the required final submission still provides the necessary motivation for students to apply the feedback.
An example for how this has been implemented is my graduate course. Here our goal is that students learn presentation skills with the ultimate goal to give an engaging, informative and interesting 3 min-thesis presentation. After a number of interactive class sessions, the students prepare their talk and present it in front of the other students, myself and the two other co-lecturers. We then take extensive time to go through each presentation, discuss what was good and what could be improved, spending at 20 minutes on each presentation. For instance, we discuss how the presentation could be made more engaging or how the clarity, the delivery and the presentation tools could be improved further. The students then apply this feedback and give a final presentation, which is the only part that is graded.
During the feedback sessions, the presenting students benefit greatly. The non-presenting students benefit tremendously as well, by learning how to give feedback, learning what makes a presentation good and getting inspiration for their own presentations. For us lecturers the session is also very meaningful, because the students are highly motivated to apply the provided feedback. We as lecturers also benefit directly by practicing to give effective feedback and to evaluate presentations. Most importantly, this session is really fun for everyone involved. Finally, the graded part of the assignment consists of each student giving their 3 min-presentation and us lecturers assigning a mark. This is easy and seeing the level of improvement is usually very gratifying.
How to apply this to other types of assignments and assessments?
A similar approach could be applied to writing assignments. Students submit their initial version and the lecturer evaluates the assignments with the objective to provide feedback and not to assign marks. As marks are not awarded at this stage, it is not necessary to evaluate the whole assignment for each student in detail. It is sufficient to highlight the weak points for each students and provide some examples based on the assays from some, but not necessarily all students. This is because students can learn from other examples and apply it to their own writing. In fact, this is more desirable because it involves active participation on the side of the student. Providing feedback on the initial submission could be done in an interactive class setting. Because students know that a final submission of their assignments is required, they will likely be very motivated to actively participate in this session.
The students then revise their submission, which is when the most learning is likely to happen. It is not really necessary to ask the students to revise and submit the entire original assignment but focus only on a sub-section (that is announced AFTER the feedback session). Letting students only re-do part of the assignment is not problematic because student have already written and more importantly discussed the other sections during the feedback session. Hence, the learning has already occurred. Marking the shorter final writing assignments would be relatively easy, because the quality is likely to be rather high and it is sufficient to focus on whether the students have applied the feedback. As student learning has already happened, it is hence not necessary to provide feedback again. This is a much less pressurizing way of marking assignments. At this stage of the assignment there would also potentially be a good opportunity to implement peer grading, as the students are aware of the important marking criteria based on the provided feedback.
Having tried both presentation and writing assignments in a large classes, I have come to the realization that the described approaches to open ended assignments are probably not really feasible in large class settings, due to the time constraints of class time and marking by the lecturer. One often utilized approach is to give open-ended assignments with marks for participation only. Giving participation marks allows us to set creative open-ended assignments that test higher order thinking skills, without having to mark these assignments individually, but providing either model answers, let students discuss on their own and implement peer feedback. I do make use of participation marks, but they usually only amount to maximally 10% of the final grade. Moreover, they only work if we can motivate the students to do them properly. I use participation marks only for group-based and problem-based multiple choice questions where students know that these exercises prepare them for the graded assessments. To motivate students, it also helps to have group competitions. I do not use participation marks for open-ended assignments. The reason is that I feel that individual feedback is critical to improve student learning. Although students can effectively learn together and explain concepts to each other, they can not effectively give feedback. In fact, it took me many years to learn how to give meaningful feedback, and I am still learning.
One approach that I have tried to implement is to set regular open ended (research) problem-based group assignments. The students complete and submit their assignments. I read and mark the assignments and provide a document that includes feedback, which the students can apply for the next assignment. I feel that in principle these regular problem-based group assignments are better than conventional open-ended assignments. The main reason is that the assignments are formative, in other words, the students expect that understanding the feedback will help them to complete the next assignment or future assessments. This ensures that the students care about the feedback. As a result, as a lecturerI am much more motivated to mark these assignments, because I feel that my feedback will be meaningful in helping the students to learn. Knowing that the students are waiting for the feedback (and even express this in their submissions) motivates me to mark the assignments thoroughly. And seeing how the students answer and how they improve over the course of the semester is very gratifying. From the lecturer’s point of view, setting of this type of assignment is not different from setting other open-ended assignments. The marking is also not different, except that it requires to provide feedback. The group-based nature of the assignments also makes the marking easier. The group-based character is also an important factor that makes these assignments effective. In group-based problem solving, students are required to propose and defend ideas and explain concepts to each other. Thus, the group work helps both students and lecturer.
However, in the end the Make-or-Break question is whether it is possible for the lecturer to give timely feedback. I have found this to be rather challenging in terms of the time-commitment necessary during a busy ongoing semester. As such, everything taken into consideration, I feel that open-ended questions are for the most part not a practical approach in large classes to facilitate good formative learning. I am of the opinion that in large classes, time is best spent to design frequent challenging (research data-based) quizzes that test higher order thinking skills, where standardized formative feedback can be provided and where students ideally discuss answers in groups and explain concepts and misconceptions to each other.