My main focus thus far has been the second approach – using chatbots for problems students need to solve and evaluating chatbot responses. This is not only a skill that will be critical in a professional and private environment dominated by AI. The inclusion of activities to evaluate AI tools also helps me as the instructor to gain better insights on how these tools could be used effectively.

 

 

One example from my undergraduate course is using ChatGPT as a tool to test scientific hypotheses. Learning how we test a scientific hypothesis is one emphasis in my undergraduate teaching. It is a skill that is required to create new knowledge and that helps students to evaluate scientific information or claims they encounter in their work or personal lives.

 

In the past, I used to teach students the type of experiments that are critical in testing a hypothesis. For instance, if we hypothesise that drug X inhibits cancer cell growth by activating a specific cellular receptor Y, we need to first confirm that drug X indeed activates receptor Y. Secondly, we would determine if activation of the receptor by other means also inhibits cancer cell growth. Finally, we would determine if the effect of drug X on cancer cell growth is dependent on receptor Y, e.g. by inhibiting receptor Y. After teaching students this concept (or letting them figure out the concept by themselves through provided examples), the students then carried out practice exercises in which they had to propose approaches and experiments to test specific hypotheses, which sometimes they had to come up by themselves.

 

To incorporate chatbots, I first assessed how good ChatGPT is at proposing experiments to test a specific hypothesis and discovered that ChatGPT was rather good at performing this task. Nonetheless, upon greater scrutiny it became clear that not all  proposed experiments were ideal and some suggestions did not help to address the hypothesis at all. As such, the obvious activity became to let students evaluate experiments proposed by ChatGPT to test a hypothesis that was provided by me or that the students had to come up with by themselves as a group. This approach turned out to be very effective. As an alternative approach, we could also to let students first come up with their own approaches and then let them compare their suggestions with those of the chatbot.

 

Extending this approach, I have also assigned students to ask the chatbot to propose specific hypotheses to a research question proposed by me or by the students themselves. For example, in one of my class activities my undergraduate students had to work in groups to come up with their own research question to explore the mechanism underlying a well-known phenomenon (after initially discussing an example of a good research question). The students then had to prompt the chatbot to propose hypotheses for their research question and select one of the proposed hypotheses that they considered plausible. Finally, the student groups asked ChatGPT to propose three experiments to test this hypothesis and evaluated the suitability of the proposed experiments, based on the criteria that we had previously established in class.

 

Through this exercise the students were able to gain an understanding of the capabilities and fallacies of chatbots like ChatGPT when employing these tools to assist in answering a research question. The approach also created a very interactive and engaged class atmosphere. Importantly, it is relatively easy to incorporate this approach of letting students evaluate chatbot solutions into our assessments, either by letting students use chatbots or by providing students with chatbot responses and letting them assess their validity.

 

 

Apart from using chatbots as a tool to design and evaluate scientific hypotheses, we have explored the use of ChatGPT as a research tool for graduate students. For instance, in my graduate course we evaluated how useful and reliable ChatGPT is as a literature search tool. In this exercise, I asked the students to choose one specific historical discovery related to their own research area and then prompt ChatGPT to describe the specific discovery process. ChatGPT was able to retrieve and cite correct information. However, the information provided by ChatGPT was incomplete. This is problematic when we intend to use the provided information in academic writing.

 

We also evaluated ChatGPT’s ability to generate research ideas. Here we asked ChatGPT (and the students themselves) to come up with research questions based on a provided specific cellular pathway. These research questions could fall into three possible categories. At the simplest level, one may ask about more details within the described pathway. At a more complex level, one may ask a research question outside the constraints of the specific pathway, for instance by drawing connections from other phenomena or considering how this signaling pathway plays out when looking at a bigger picture. Finally, one may ask about implicit assumptions that are made within the pathway. This approach requires that ChatGPT is able to question and eliminate assumptions, which is a very important element in creative thinking.

 

Most of the research questions proposed by ChatGPT were in the first category. Although some questions were trying to look at the bigger picture, ChatGPT did not utilise any information that was not included in the provided pathway description. ChatGPT also did not come up with research questions that challenge implicit assumptions. As such, the level of creativity that ChatGPT exhibited in this exercise was not impressive. ChatGPT can help us to come up with ideas, but the ideas generated by the current version of the chatbot are not truly creative and groundbreaking.

 

Another potential exercise is to utilise chatbots such as ChatGPT in order to come up with experimental approaches to test a specific scientific hypothesis. Based on my own experience, ChatGPT gives good general suggestions. However, the detailed proposed experimental manipulations are sometimes problematic or infeasible. For instance, a drug proposed by ChatGPT to be used in an experiment involving mice was not membrane-permeable, precluding its use. When pointing out these concerns, ChatGPT usually readily acknowledged its error and suggested alternative approaches. For example:

 

 

This indicates that ChatGPT may not be helpful if we ourselves are unable to spot potential errors. On the other hand, by asking ChatGPT to double check its statements, it may correct its answer and potentially produce a more accurate answer.

 

 

In my graduate course we also explored the use of ChatGPT as a tool to help writing a scientific research paper.

When it comes to student writing assignments, AI tends to stir up great fears due the potential that students use chatbots to cheat. However, this threat can be easily be prevented by requiring that students use chatbots and evaluate the AI output, which is what we implemented in our course.

 

For the past few years, the main assignment in our graduate course has been the writing of a research paper, which the students do based on a provided set of fictional experimental data. In this assignment, we initially focussed on the results and summary section, by letting students come up with a storyline based on the randomly arranged data, asking them to write parts of the results section and abstract, providing in-class as well as individual feedback, and finally tasking the students to revise their write-ups.

 

Subsequently, we assigned students to utilise ChatGPT to aid in the writing of the remaining components of a research paper, including the title, the introduction and the discussion. In this assignment, the students had to use their written abstracts and results sections to get ChatGPT produce a research paper title, an outline of the introduction including references as well as an outline of the discussion. The students also had to solicit ChatGPT’s help to write the beginning of the introduction and discussion. In this process, the students were supposed to explore approaches to improve the ChatGPT output. They had to document this process by incorporating their prompts, the ChatGPT responses and their insights into a slide presentation.

 

During our first class session to discuss this assignment, the students were then assigned to groups of three students and each group was asked to exchange their experiences. Each group also had to come up with a joint presentation of their experiences, which they delivered during the subsequent class. The student evaluation was based on the students’ individual slide submission as well as their group presentations.

 

The students figured out a number of useful insights. What seems to matter most is to tell ChatGPT what the characteristics of a good title, of a good introduction outline or of good discussion points are. Thus, one of the students googled what a good title should look like and then giving this information to ChatGPT. Another student asked ChatGPT for the characteristics of a good research title and then let it come up with a title based on these characteristics.

 

One of the students demonstrated very nicely that with very specific sequential prompts one can go from a very unattractive to an excellent title. This process involves writing prompts where we ask ChatGPT to focus on specific title characteristics (for instance to make the title more concise or to add a point about the significance of the study) or where we highlight specific words or phrases that should be changed.

 

With regards to the actual writing, there is no question that ChatGPT is extremely good at putting together well-written sentences and paragraphs. However, ChatGPT did not prove useful for creative writing. This became particularly clear in the discussion outlines, where all of the ChatGPT suggestions were based on the input. ChatGPT did not provide insights or ideas that were truly creative or that integrated knowledge not provided in the prompt.

 

There was, however, one notable exception, where one student asked ChatGPT to “add one more possible future research area”. This resulted in ChatGPT giving a very good suggestion that was not based on the provided information. This example is consistent with my own experience that by including “possible” in my prompts, ChatGPT becomes more creative. This example highlights the importance of trying new prompts and not settling with the initial response.

 

Beyond this, there are obviously many other ways in which students can utilise and evaluate AI tools. For instance, we can assign students to use AI tools to design engaging presentation slides or to come up with suitable scripts for different audiences. We can even assign students to use AI tools to overcome any problem of their own choice. This may allow for more individual explorations and a productive exchange of ideas.

 

Finally, I would also like to let students leverage the power of AI to practice solving difficult problems. However, to design such exercises it is likely necessary to first learn how to use the AI tools myself and evaluate their potential. This is something that I am actively trying to do. Although it takes time and effort, it is important to constantly learn and improve ourselves to remain relevant and continue to make meaningful impacts.