The only reason you’ll ever need not to write with AI
Over the last year, our lab has been developing a policy on AI use. To do this, we did three main things:
We read a lot of academic publications and tech news.
We set up an #ai channel on our lab Slack to share news, experiences, and memes.
We had several long and grueling lab meetings talking through finer points of disagreement.
Eventually, I’ll share longer reflections from this process, including suggestions for how you could adapt it for your own workplace. In the meantime, I wanted to share the biggest thing I learned — something that feels incredibly urgent, and that I haven’t seen anyone say about AI use in academic or other professional settings.
Our lab has agreed on a blanket ban on the use of generative AI for writing.
Before I explain why, some common points for and against the use of GenAI as a writing assistant:
Many users, especially students, find that it helps articulate ideas that you’ve already formulated but can’t quite put into words. (Basically, something between a rubber duck and an antidote to writer’s block.)
GenAI tools are becoming a cheap alternative to narrowly-scoped spell and grammar check tools like Grammarly—and those tools are on their own embarrassing journey with GenAI now too.
Many GenAI tools have their own distinct and identifiable writing style, complete with over-use of certain words and stylings, detracting from your own unique voice as a writer.
GenAI repackages other writers’ ideas and words as your own, without giving them credit or citing appropriately—and when you ask it to cite its sources, it invariably hallucinates references. Plus, AI companies are buying and destroying old books, which sucks.
Weighing these considerations is a bit difficult. I don’t personally use GenAI to write, but I understand why other people do (especially non-native English speakers), and it’s hard to find a rule set that solves the tradeoffs here.
So how did we end up at a blanket ban?
Here’s the basic point I haven’t seen anyone make:
Using generative AI inherently exposes you to the risk of career-ending accusations of plagiarism.
Before, I said that “generative AI repackages other writers’ ideas and words as your own, without giving them credit or citing appropriately.” A lot of people would say that constitutes plagiarism—especially people who make their living as creative writers. I generally agree with them, but that’s not the point of this post.
What I’ve come to realize is that, regardless of any true similarity between your words and someone else’s, or your ideas and someone else’s, generative AI exposes you to risk that you, as a writer, cannot mitigate.
This is true for three basic reasons:
Plagiarism is subjective.
Plagiarism is a continuum.
Not all plagiarism accusations are made in good faith.
There’s always a steady stream of plagiarism stories in academic news, but rather than pick on any current examples, I think the most instructive case is the story of Harvard president Claudine Gay, who was forced to resign in 2024 following accusations of plagiarism in her dissertation and peer-reviewed publications. I don’t want to litigate whether or not Gay is guilty of plagiarism—that’s actually sort of the point of this post.
Here are the three things I think are important in this story.
First, edge cases count. Some of the specific passages of accused plagiarism are rather egregious, but others are not, and reflect some of the familiar constraints of academic writing. For example:
Screenshot from The Crimson.
Anyone who has experimented with Claude or ChatGPT as a writing assistant knows that, when it summarizes peer-reviewed research, it often reproduces original wording at a rate that is very similar to this example. You, as a person using a black-box model, cannot ever comprehensively check its training data to find those overlaps. Also, ideas matter as much as words: it’s pretty easy to accuse someone of stealing ideas, and language similarities are generally unavoidable as part of that—and the burden of proof that your ideas are your own is, generally, impossibly high. (Especially if you let the computer workshop your ideas with you.)
Second, some people need to be more careful. If you are a woman or a person of color in science, other people are more likely to refuse to believe your ideas are your own—and more importantly, you face a much greater risk from harassment of all forms. It’s well documented that one of Gay’s accusers was the person who is arguably the main architect of modern attacks on higher education, and the accusations were made on the heels of Gay’s testimony in a Congressional hearing about antisemitism on college campuses. In the end, none of that helped Claudine Gay. Plagiarism accusations are now a core part of the right-wing harassment toolkit, and knowing that those accusations were made in bad faith or for political reasons does not make them disappear.
And finally, most plagiarism accusations are a one-way ticket. If you’re a college student and you don’t cite your sources correctly in a paper, maybe you get away with a friendly but firm, one-time, come-to-Jesus talk from your professor. But once you reach the level of an independent scholar, it is nearly impossible to be accused of plagiarism and keep your career intact—especially if those accusations become public, where they can be debated ad infinitum. And anything you’ve written, at any point, can be relitigated a half-century later.
I look at all of this, and the only thing I can think is: why take that risk? I know that my ideas and my words are my own; I find plagiarism fundamentally repulsive as a practice, and the only reason I show up to work is that I enjoy my craft. But if I ever let GenAI near my writing, I’m exposing myself to risk from plagiarism accusations—whether or not the GenAI tool smuggles in something that does, actually, constitute plagiarism. If somebody comes to me in 20 years and says “Are these your original ideas and words?” about even a single sentence of my writing, I need to be able to answer yes—no reservations, no loopholes, no room for error.
Here’s the biggest problem.
Academic writing is a collaborative venture, and in my experience, very few researchers have a comprehensive conversation about GenAI use before they collaborate. Everyone tends to think that their practices are normal, well-reasoned, and probably pretty similar to where other people they respect have landed on AI—whether that’s abstaining from it altogether, just using it to code, or wholly embracing it across your workflow.
That means that all of us have the ability to end each other’s career. This has always been true—on the 100+ papers I’ve written in my career, I’ve trusted every single coauthor (and they’ve trusted me) not to introduce a single sentence that will detonate our lives somewhere between journal submission and retirement. We don’t usually think about this as a problem, because we trust the other people we work with. But GenAI adds back the possibility that other people will introduce writing into your paper that can be credibly accused of plagiarism.
I don’t know what we do about that. But I know where to start: when you work with the Carlson Lab, you have a guarantee that all of our words and ideas are our own. I’m trusting you to be able to make the same promise.
— c.c.