 |
In my W101, in most class periods, students do a graded in-class writing (ICW) assignment. A few weeks ago, I did an experiment: I required students to use a large language model (LLM) like ChatGPT or Gemini to do the ICW.
I gave them the following instructions: “Use an LLM of your choice to write a response to the ICW prompt. However, you are responsible for the final text you submit, so revise the LLM output to make sure it reflects what you want to say.”
The learning goal was to help students understand that if they choose to use AI as a writing tool, they should not simply copy and paste unrevised AI-generated text. Any AI-generated output has to be assessed and refined by humans to be effective.
When students submitted their responses, they included a transcript of their conversation with the LLM. Students who prompted the LLM by first writing a draft on their own produced much more useful output than students who prompted the LLM to generate content that they then refined.
It was an eye-opening experience for me and the students. I learned more about how most students are engaging with AI to generate output, and, unsurprisingly, most don’t know how to prompt it well to generate useful output. And most don’t know what it means to refine AI output. Students struggle to revise their own writing; similarly, they struggle to revise AI output.
My students learned that unrefined AI output is pretty awful. For example, it tends to include a lot of unsupported claims. It makes generalizations about the text it is prompted to write about. Its evidence is superficial rather than detailed. Its evidence is often taken out of context and used incorrectly. I could go on. In the end, the activity was not just a lesson about AI output, but a lesson about the characteristics of good college-level writing.