Discussions about generative AI in higher education have been informed by studies of student-completed reports and self-reported survey data. While research shows that artificial intelligence tools can support learning, they also raise concerns such as the potential for student over-reliance, cheating, and decreased critical thinking and engagement.
Although these types of studies provide interesting snapshots of reported practices, there may be something important hidden in their methodology. That’s how writing actually unfolds when students are composing with the help of AI.
A pilot study I led with undergraduate writers at Kennesaw State University takes a different approach. Our study captures how students interact with generative AI tools during the writing process itself using a think-aloud protocol, a way for participants to verbalize their thoughts during performance. This method helps understand the decision-making process as it occurs.
Our preliminary findings suggest a more complex reality than the common narrative that students simply let AI write their assignments. Rather, many students appear to be negotiating when and how AI is included in their writing.
A look inside the writing process
In our study, 20 undergraduate students completed a 20-minute writing session by answering the following questions:
People spend a lot of time trying to achieve perfection in their personal and professional lives. People often expect perfection from others, creating expectations that may be difficult to meet. In contrast, some people believe that perfection is unattainable or undesirable.
The challenge was to draft a thesis and evidence-based text that argued for their position on the value of striving for perfection. Students were not expected to complete, but were told to move the writing process toward completion. Students were told that there was no right or wrong way to use AI, and were asked to use generative AI exactly as they normally would while writing.
Instead of direct observation, this study relied on post-session screen recordings and analysis of students explaining the process. By collecting this data (transcripts of computer activity and audio recordings), researchers were able to analyze it without interrupting the writing process. To reduce the chance that students would change their behavior if they felt they were being observed, the researchers set a timer and left the room during the writing session. The goal was to minimize the Hawthorne Effect, a phenomenon in which people change their behavior when they know they are being watched.
what we found
Across the transcripts, several qualitative patterns consistently emerged in how students collaborate with AI while writing.
First, many participants used AI at the beginning of the writing process to help generate ideas and draft papers.
What we see in this practice is students using the output generated by AI to inspire and shape their ideas. One student explained the strategy as follows: [generating a few ideas,] That’s all I usually use [output] as a prompt. ”
In moments like these, AI served not as the final answer, but as a brainstorming tool to help students get past the blank page.
However, students often continued drafting independently after generating their initial idea. Many transcripts included statements such as “I think my paper should be…” and “Let me write this part,” suggesting that some students were in control of their arguments.
Edit bot
Another strong pattern across transcripts is that students rarely accept AI texts without editing. Instead, actively modify the generated language. As one student described the process, the AI ”rewrites” the initial prompt, and then the student rewrites the AI’s output. This allows students to claim “authorship and ownership” of the final draft.
Another participant redirected the AI’s response that did not match the task. “The AI is not following the prompt…Please try again.”
These moments demonstrate that students are critically evaluating the output of AI and treating it almost as a sparring partner rather than simply copying it.
It was also found that some students completely rejected the AI proposal.
In some writing sessions, participants explicitly decided not to use AI responses. One student reflected on this decision while composing and said, “I don’t really use AI in my research.”
Other records have shown students returning to their own writing if they felt the AI’s response was too general or disconnected from their own argument. These moments show that students are not only collaborating with AI, but also drawing boundaries on where AI belongs in the writing process.
Finally, several transcripts showed students turning to AI in moments of anxiety or when they felt stuck.
One participant explained, “I used AI a lot because I was having a hard time.”
Even in these cases, students often used the AI to assist them in writing their essays, rather than directly copying and pasting the AI’s answers.
What this says about AI and writing
Our analysis suggests that generative AI is being incorporated into student writing as part of a negotiated collaboration, rather than as an outright replacement for human authorship. This result suggests that while students maintain control over argument selection, utterances, and final phrasing, AI most often enters the composition process during idea generation, revision, and moments of writer’s impasse.
Understanding not only what appears in the final essay, but also how the decision to use AI plays out during the writing process, could help educators design challenges and policies that keep human writers firmly in control.
Because our current findings were obtained from a pilot cohort of 20 undergraduate writers, the results should be interpreted with caution. To test whether these patterns hold true on a larger scale, the research team is expanding the study to 100 undergraduate participants. The expanded study will also investigate how neurodivergent writers interact with generative AI while composing, an area that remains largely unexplored in current research.
Kennesaw State University undergraduate student researchers Kylee Johnson, Vara Nath, Ruth Sikhamani, and Kaylee Ward contributed to the preliminary analysis described in this article.
