Writing process tracking is coming to Grammarly - and students and teachers need it

I tried Grammarly Authorship and largely like it though I hope for certain changes

Contents

• Accountability that's less stressful for students

• Is tracking the writing process invasive?

• Pros and cons of Grammarly Authorship

• Other writing process tracking options

• Encouraging student reflection on the writing process

Later this month Grammarly is launching Authorship for all users (see this Inc.com article and this BusinessWire article). I tried out the pre-launch version and wanted to share some thoughts. Basically, Grammarly Authorship allows anyone using the Grammarly extension for Chrome in Google Docs to generate a report on their writing process. The report will show how much time they spent on the document, show a sped-up replay of the document’s creation, and attempt to label each chunk of text as human-authored, AI-generated, “sourced,” or “unverified.” As I understand it, the system doesn’t use statistical AI text detection; it simply labels text “AI generated” if it was copied and pasted from ChatGPT. Any text that was typed is labeled “human authored.”

Now I am not getting any compensation from Grammarly, I have no ongoing relationship with them, and I don’t love everything they do. To be honest, I haven’t looked much at how they handle style, and I’m frustrated with the confusion that has come about since they introduced generative AI; people say “I used Grammarly” and that is understood to mean they got minor grammar and word choice suggestions. But even the free version of Grammarly gives 100 generative AI prompts a month. Users can just ask it to autogenerate whole passages. For my class, I ask students not to use Grammarly or any chatbot to write or rewrite their essays (though we do read about AI and they reflect on AI feedback on their writing). However, I still encourage my students to use the free version of Grammarly, review its grammar suggestions skeptically, and follow its links to explanations of possible errors. (This paragraph was updated after Carol Bailey pointed out that the free version of Grammarly offers generative AI, not just the Pro version.)

Like the free, relatively high quality grammar suggestions, I think Grammarly Authorship is going to be an overall boon to students and teachers. I hope Grammarly will continue to improve on various points, but this is an important step in the direction of transparency about what is AI text and what is not.

A screenshot from a Grammarly Authorship report on a document where I inserted AI-generated text from Grammarly Pro (labeled in purple as a “Known source” and not human authored.

Accountability that may be less stressful for students

To me, process tracking seems like a more fair and transparent approach to accountability than the statistical AI detection commonly used by teachers today such as Turnitin’s AI detection and GPTZero.

No, it is not foolproof at preventing students from using AI to skip valuable writing and thinking practice. It is possible to retype AI output; that retyped output will be labeled as human by the Grammarly Authorship report. Still, the amount of time spent in the document will be recorded. It is less tempting to retype AI output in about as much time as one would have spent writing one’s own words. 

Of course, this does nothing to address the root causes of AI misuse. The most important things we can do as teachers are still 1) to design writing assignments that are intrinsically motivating, 2) to cultivate relationships with students and between students that support student confidence in their own voice and ideas and 3) to teach and support the writing process itself to make it less stressful and more achievable step by step. It’s also helpful now to mix in low-stakes in-class writing and oral assessment. (See these slides on academic integrity for more on my approach.)

But as I and many other teachers have reported on social media and in professional listservs and Facebook groups over the last year, these strategies are not enough. We need other tools to support accountability and make last-minute recurrence to AI less tempting for students who are stressed, overwhelmed, and intimidated by academic writing. It’s probably because so many teachers believe there is a need for some accountability that Turnitin’s AI detection has remained popular. We’ve seen plenty of evidence that it produces false positives that lead to false accusations. Serious concerns have surfaced about bias against English language learners and studies have shown that it’s easy to evade detectors through automated paraphrasing software (Turnitin claims to have solved this but hasn’t offered evidence). Most teachers don’t want to police. But we also recognize that some accountability is needed in most human endeavors that involve sometimes disagreeable effort. 

Photo by Vlada Karpovich on Pexels.com

Is tracking the writing process invasive? 

I don’t love asking students to share their whole document history. I do think the practice is justified, but I think we should allow for alternate processes (We should, of course, also allow accommodations for disabilities). Here’s how I frame it to students: “Think of this as an online version of in-class writing that allows you more flexibility. It gives me a way to understand your writing process, both to encourage academic honesty and to encourage reflection on the writing process itself. If you have concerns or do not feel comfortable sharing your process in this way, please let me know and we can meet and work out another plan.” 

This informal poll I ran on X/Twitter in January 2023 suggested that opinion about process tracking is divided. 58% of 248 respondents thought it was fine and 42% thought it was intrusive or inappropriate.

Screenshot from an X poll

Let’s be clear what we’re comparing it to; if the alternative is in-class writing, that is also under close observation. I did another informal poll where I said “You are a student. Your instructor gives you a choice: Would you rather write your essay in class or write it out of class with full keystroke-by-keystroke transparency about your activity in the document?” There was a near-even split among 287 respondents on preference. Of course, these polls don’t represent a random sample but only the particular audience that comes across my X posts.

Pros and cons of Grammarly Authorship

I appreciate that this is available free to all Grammarly users. This means if we were already using Grammarly extension, we won’t be giving away data any more than we were already (See the Grammarly Privacy Policy). Like OpenAI, Grammarly defaults to using user data to train its models, but this option can be turned off. 

Screenshot of the default setting in Grammarly which allows it to use what we write to train its AI models

I like that in this model, the process report belongs to the author. The author can review it, choose to print it to PDF and share it with their comments. Students could do this to defend themselves against a charge of AI misuse. Grammarly Authorship doesn’t sync with learning management systems or provide ways to manage a whole class set of process reports. Still, teachers could require submission of PDF authorship reports along with essays. 

There are some ways that Grammarly Authorship as it currently works could lead to unjustified teacher suspicions. There are a few ways their reports will likely mislabel text. For now, voice typing shows up as “unverified” rather than “human authored” which might cast suspicion on students who routinely use voice recognition to write. The “AI generated” label suggests that the system can actually identify what is AI generated, but that might instill some false confidence. The label seems only to be applied when text is copied directly from ChatGPT. If copied from Claude or another AI app, it gets labeled “unverified.”  Also, many people use ChatGPT to suggest grammar or style revisions to text they have come up with themselves. Then they copy the resulting edited version back into their document. I tried that, and the Grammarly Authorship report identified my edited text as “AI generated.” Potentially a student could share a transcript of their chat session to defend themself in such a case, but they would still undergo the stress of being accused. I’d like to see text labeled more precisely as “Pasted from ChatGPT” or “Pasted from an AI app” rather than “AI generated.” 

I am hoping that the final version will distinguish between text edited by Grammarly and text wholly generated by Grammarly and acknowledge that the generated text comes from AI. So far in my experiments, anything edited or generated by Grammarly is labeled as a known source, but it doesn’t identify this source as Grammarly or label the generated text as “AI generated.” I hope they’ll change that! In my one meeting with Grammarly product marketing manager Cliff Archey, he seemed receptive to input.

Other writing process tracking options

Grammarly Authorship is one new option among many for writing process tracking. I have focused on it here because it seems likely to gain wide adoption since it is free and so many already use Grammarly. 

However, one I’d like to spend more time looking into is Integrito by PlagiarismCheck.org. I’m not clear on how the pricing works; there may be free options. Chrome extensions like Revision History and Draftback are definitely free, but these require students to share edit access to their Google Doc.

If the colleges where I teach could find funding, I would be glad to use

Antecedent Writing, PowerNotes Insight, or Rumi instead. They seem like the best designed to allow teachers to more easily track a whole class worth of process reporting. They are FERPA compliant and don’t include statistical AI detection. (Txtreplay might be another option.)

I’m going to participate in Turnitin’s beta testing of its own writing process tracking system this fall. Turnitin’s classroom management or learning management system integration would make it administratively easier to review student process, but I’m concerned they may not default to giving students first view of and control over their writing process reports.

Encouraging student reflection on the writing process 

In addition to the pedagogical approaches mentioned above, I think we will continue to need software that makes AI misuse less tempting and makes it easier for students to do their own writing outside of class. Process tracking seems to me like the best path forward for this, but process tracking also has other uses that haven’t been explored.  What I want to see ultimately is an app that invites students to reflect back on their process as they comment on it. The teacher could then respond to the comments in ways that support the student’s future writing strategies.

I’m going to try out using Grammarly’s Authorship report in this way by asking students to print it to PDF, upload it to Google Drive, annotate it there, and share it with commenting privileges. Here’s a sample where I’ve commented on my own Authorship report to highlight some of the pitfalls I mentioned above.

I’m looking forward to process tracking software that helps the student reflect on their process of authoring the document and also helps student and teacher distinguish between the student’s choices and any AI influence. That may be the ed tech we need to support writing for thinking and learning in an age of AI.

AI use statement

I reflected on Claude 3.5 feedback on drafts of this post but did not include any AI-generated text.

Your thoughts?

I know this is a tricky issue, and I hope to see much more discussion of process tracking as educators try to figure out whether it’s needed, how it compares to alternative approaches to academic integrity, and if it’s needed, how it should be done. Please feel free to comment here!

Update note September 4, 2024: I changed the references to Grammarly Pro because Carol Bailey pointed out on X that even the free versions of Grammarly include text generation and revision, though they limit users to 100 prompts per month. Thanks, Carol!