I'm incorporating Turnitin alongside process tracking, writing process assignments, social annotation, lots of student choice, peer review and tutoring, video assignments, and more.
Could someone “cheat” at Substack by using AI to write articles? Sure, but most readers would instantly recognize it as slop and they wouldn’t be successful. And if they were able to use the right prompts and techniques to produce articles that readers found valuable and interesting, then good for them.
Why isn’t a similar philosophy applied in education?
When we assign writing in a classroom, the goal isn't to produce an essay for readers. The main goal is the learning that's involved in writing the essay.
To clarify, I'm not arguing against AI but rather for transparency about what is AI and what is the student's so we can assess and design for learning.
I would not assume readers could recognize any AI-generated Substack as slop.
there's some great research on whether or not we can discern AI from human writing (less and less), whether we prefer if compared to human writing (sometimes yes), and whether we prefer it if we know in advance (we're biased to humanity)
You hit the nail on the head. The real issue is writing for grades. As long as that is happening, students will use AI to achieve better results. Substack's are "graded" by the number of subscribers and quality of the prose - if AI's are able to do this better than humans, we will see AI substacks. My go to comment on an assignment I think is AI generated is along the lines of "this is very well written, but seems detached from the rest of the paragraph" or something of the sort. If a sentence works and the narrative or argument is advanced, I assume the student is doing something right. But it's not always easy.
Glad to share it! I've been loving watching the video right before I read the paper. Then sometimes I make a video of feedback, so the whole thing feels more like a conference.
I wish I had time to meet with each student for each essay. But I do ask them to record a 1-3 minute video talking about their writing process.
Right now, I'm not running most student writing through detectors, partly because that would be too complicated on top of process tracking with Grammarly Authorship, but I have told students that if a question arises in my mind I might ask them to submit to Turnitin.
Especially in an asynchronous online class, it doesn't seem like enough to ask them to write something about what they learned. It's too easy to autogenerate that too...
I understand your concerns and appreciate your thoughtful engagement. As a student who struggles with traditional education, I can confidently say that distancing myself from AI feels impossible. Over time, I have discovered my voice through AI. AI cannot generate meaning on its own; it serves as a personal learning tool. It isn't effective for cheating :-)
This is a really thoughtful post. I appreciate it. But I have to ask (everyone) -- Why not just use linguistic fingerprinting? It serves the same purpose, but it doesn't involve accusing the student of using AI at all. All you are tracking is whether or not the writing matches their "voice" or their "fingerprint" from a Known Document...
I've been advocating for this for awhile, but it never seems to take. I can't figure out what is wrong with it, other than the fact that the reports are somewhat lengthy and difficult to understand if you don't have a linguistic background. But they are way more reliable -- have been used by law enforcement since the 1980's -- and don't involve trying to detect AI writing at all.
@Mike Kentz I'm definitely interested in this approach and am also puzzled that we haven't heard more about it! Thanks for the link. Linguistic fingerprinting would need to be tested for bias, as detection should be too... in combination they could be even more accurate. Linguistic fingerprinting would need to allow for the one one person can change style, genre, and voice, of course.
I empathize with your approach but I fall on the other side of the coin, not least of which is because AI detection tools will never be 100% reliable and the rupturing of a relationship with a student over a false AI accusation can be irreparable. I also did not get into teaching to spend precious time reviewing version histories of documents or try to parse whether a student typed their own work or someone else's. My initial interest in AI with respect to writing was sparked by an infographic which pre-dated the release of ChatGPT and this prescient line in particular: "Hybrid Human-AI Writing Will Become Normal - Hybrid Writing, co-created by human and artificial intelligence together is becoming prevalent. Soon it will become the norm. Trying to determine where the human ends and where the intelligence begins is pointless and futile." There are other tenets in the infographic that make clear humans are responsible for ensuring the accuracy and quality of their work, but the phrase "pointless and futile" really resonated and I fear all this angst will be for naught at the end of the day. I realize there are different concerns with younger writers but I'm still unconvinced that playing enforcer is the solution. I have a baseline blue book written assignment from students at the beginning of the year and it's fairly obvious if work is submitted that is AI generated. I teach HS and virtually no HS student writes grammatically flawless well-structured prose with relevant vocabulary on point with no strange personal tics or unique writing tells. It's actually not that hard to spot without detection software. But if students are incorporating AI suggestions, using rewritten and rephrased sentences of their own initial ideas, and folding these into paragraphs they wrote on their own, do you still count that as plagiarism? We will really need to confront how the role of AI editing assistants has transformed the writing process and ask ourselves which hill we are willing to die on. We have teachers who won't let students run their papers through Grammarly prior to submission which seems insane to me. It's ok to use spellcheck but not something else that will show you how to improve your writing?
@Steve Fitzpatrick I appreciate many of your points; thank you for responding! It's a tricky landscape with many pros and cons to various paths. There will be gray areas. I still don't want my students to miss out on significant experience with writing processes that don't involve AI in the idea formation or word assembly (I invite students to use AI for feedback through an app that encourages questioning of the AI outputs).
I think I recognize that quote--is it from Sarah Elaine Eaton? Oh yes, I see the link to her work at the end. I have used part of that quote in my presentations and pushed back a bit; I argued that in some contexts at least, we will want to know what the human did and what the AI did; we will want to preserve some space for human communication through text or audio or video recordings. Here are slides if you are curious about my argument on transparency: https://bit.ly/TowardsTransparencyGPT
I do teach students to use Grammarly grammar suggestions skeptically, but I wouldn't advise they use it to generate new versions of what they've written. Maybe beyond the first-year composition level I would teach more hybrid processes... Many have so little confidence in their own voices and I feel they need to build that confidence through human writing processes in my class.
Thanks for the exchange of ideas. I am in sympathy with the tone of that "which hill we want to die on comment..." :-)
I am grateful, Anna, for your open sharing and emphasis on the multiple layers involved in your assessment approaches — layers that are transparent, honest, and flexible. I respect your decision and applaud the choices you provide for students.
Thank you for such a kind and supportive comment, Michelle! That means so much to me that you see my good intentions and process though you might not make the same choices yourself.
I have an AI detector between my ears that’s been trained on a lot reading all kinds of texts for the last 50 years or so. It’s been pretty accurate detecting (even before AI) student writing that’s more sophisticated than the average first-year student is generally capable producing . When I have questions I ask the student to come in to my office for a chat. 10 minutes of conversation and a question or two usually settles the issue and leads to good discussions about intellectual integrity. It’s time consuming, yes, but there’s really no other way to do it.
Thanks for the good wishes! I think I should have clarified my position more in the post because I'm not against inviting students to use AI; I just want to facilitate transparency about what is AI and what is the student's so we can assess and support their learning....
Could someone “cheat” at Substack by using AI to write articles? Sure, but most readers would instantly recognize it as slop and they wouldn’t be successful. And if they were able to use the right prompts and techniques to produce articles that readers found valuable and interesting, then good for them.
Why isn’t a similar philosophy applied in education?
@JT Thanks for the provocative response!
When we assign writing in a classroom, the goal isn't to produce an essay for readers. The main goal is the learning that's involved in writing the essay.
To clarify, I'm not arguing against AI but rather for transparency about what is AI and what is the student's so we can assess and design for learning.
I would not assume readers could recognize any AI-generated Substack as slop.
there's some great research on whether or not we can discern AI from human writing (less and less), whether we prefer if compared to human writing (sometimes yes), and whether we prefer it if we know in advance (we're biased to humanity)
This is an evolving field for sure!
https://mitsloan.mit.edu/ideas-made-to-matter/study-gauges-how-people-perceive-ai-created-content
https://ar5iv.org/html/2412.18148
https://reutersinstitute.politics.ox.ac.uk/sites/default/files/2024-05/Fletcher_and_Nielsen_Generative_AI_and_News_Audiences.pdf
You hit the nail on the head. The real issue is writing for grades. As long as that is happening, students will use AI to achieve better results. Substack's are "graded" by the number of subscribers and quality of the prose - if AI's are able to do this better than humans, we will see AI substacks. My go to comment on an assignment I think is AI generated is along the lines of "this is very well written, but seems detached from the rest of the paragraph" or something of the sort. If a sentence works and the narrative or argument is advanced, I assume the student is doing something right. But it's not always easy.
Anna, I love this idea of asking students to reflect on their writing process through video. Thank you!
Glad to share it! I've been loving watching the video right before I read the paper. Then sometimes I make a video of feedback, so the whole thing feels more like a conference.
This seems very complicated. Why not just ask the students what they learned?
I wish I had time to meet with each student for each essay. But I do ask them to record a 1-3 minute video talking about their writing process.
Right now, I'm not running most student writing through detectors, partly because that would be too complicated on top of process tracking with Grammarly Authorship, but I have told students that if a question arises in my mind I might ask them to submit to Turnitin.
Especially in an asynchronous online class, it doesn't seem like enough to ask them to write something about what they learned. It's too easy to autogenerate that too...
I understand your concerns and appreciate your thoughtful engagement. As a student who struggles with traditional education, I can confidently say that distancing myself from AI feels impossible. Over time, I have discovered my voice through AI. AI cannot generate meaning on its own; it serves as a personal learning tool. It isn't effective for cheating :-)
This is a really thoughtful post. I appreciate it. But I have to ask (everyone) -- Why not just use linguistic fingerprinting? It serves the same purpose, but it doesn't involve accusing the student of using AI at all. All you are tracking is whether or not the writing matches their "voice" or their "fingerprint" from a Known Document...
I've been advocating for this for awhile, but it never seems to take. I can't figure out what is wrong with it, other than the fact that the reports are somewhat lengthy and difficult to understand if you don't have a linguistic background. But they are way more reliable -- have been used by law enforcement since the 1980's -- and don't involve trying to detect AI writing at all.
What am I missing?
https://www.edsurge.com/news/2024-05-07-can-linguistic-fingerprinting-guard-against-ai-cheating?utm_content=buffer5ee29&utm_medium=social&utm_source=twitter.com&utm_campaign=EdSurgeBuffer
@Mike Kentz I'm definitely interested in this approach and am also puzzled that we haven't heard more about it! Thanks for the link. Linguistic fingerprinting would need to be tested for bias, as detection should be too... in combination they could be even more accurate. Linguistic fingerprinting would need to allow for the one one person can change style, genre, and voice, of course.
I empathize with your approach but I fall on the other side of the coin, not least of which is because AI detection tools will never be 100% reliable and the rupturing of a relationship with a student over a false AI accusation can be irreparable. I also did not get into teaching to spend precious time reviewing version histories of documents or try to parse whether a student typed their own work or someone else's. My initial interest in AI with respect to writing was sparked by an infographic which pre-dated the release of ChatGPT and this prescient line in particular: "Hybrid Human-AI Writing Will Become Normal - Hybrid Writing, co-created by human and artificial intelligence together is becoming prevalent. Soon it will become the norm. Trying to determine where the human ends and where the intelligence begins is pointless and futile." There are other tenets in the infographic that make clear humans are responsible for ensuring the accuracy and quality of their work, but the phrase "pointless and futile" really resonated and I fear all this angst will be for naught at the end of the day. I realize there are different concerns with younger writers but I'm still unconvinced that playing enforcer is the solution. I have a baseline blue book written assignment from students at the beginning of the year and it's fairly obvious if work is submitted that is AI generated. I teach HS and virtually no HS student writes grammatically flawless well-structured prose with relevant vocabulary on point with no strange personal tics or unique writing tells. It's actually not that hard to spot without detection software. But if students are incorporating AI suggestions, using rewritten and rephrased sentences of their own initial ideas, and folding these into paragraphs they wrote on their own, do you still count that as plagiarism? We will really need to confront how the role of AI editing assistants has transformed the writing process and ask ourselves which hill we are willing to die on. We have teachers who won't let students run their papers through Grammarly prior to submission which seems insane to me. It's ok to use spellcheck but not something else that will show you how to improve your writing?
https://drsaraheaton.com/wp-content/uploads/2023/02/postplagiarism.jpg
@Steve Fitzpatrick I appreciate many of your points; thank you for responding! It's a tricky landscape with many pros and cons to various paths. There will be gray areas. I still don't want my students to miss out on significant experience with writing processes that don't involve AI in the idea formation or word assembly (I invite students to use AI for feedback through an app that encourages questioning of the AI outputs).
I think I recognize that quote--is it from Sarah Elaine Eaton? Oh yes, I see the link to her work at the end. I have used part of that quote in my presentations and pushed back a bit; I argued that in some contexts at least, we will want to know what the human did and what the AI did; we will want to preserve some space for human communication through text or audio or video recordings. Here are slides if you are curious about my argument on transparency: https://bit.ly/TowardsTransparencyGPT
I do teach students to use Grammarly grammar suggestions skeptically, but I wouldn't advise they use it to generate new versions of what they've written. Maybe beyond the first-year composition level I would teach more hybrid processes... Many have so little confidence in their own voices and I feel they need to build that confidence through human writing processes in my class.
Thanks for the exchange of ideas. I am in sympathy with the tone of that "which hill we want to die on comment..." :-)
I am grateful, Anna, for your open sharing and emphasis on the multiple layers involved in your assessment approaches — layers that are transparent, honest, and flexible. I respect your decision and applaud the choices you provide for students.
Thank you for such a kind and supportive comment, Michelle! That means so much to me that you see my good intentions and process though you might not make the same choices yourself.
I have an AI detector between my ears that’s been trained on a lot reading all kinds of texts for the last 50 years or so. It’s been pretty accurate detecting (even before AI) student writing that’s more sophisticated than the average first-year student is generally capable producing . When I have questions I ask the student to come in to my office for a chat. 10 minutes of conversation and a question or two usually settles the issue and leads to good discussions about intellectual integrity. It’s time consuming, yes, but there’s really no other way to do it.
Thanks for the good wishes! I think I should have clarified my position more in the post because I'm not against inviting students to use AI; I just want to facilitate transparency about what is AI and what is the student's so we can assess and support their learning....