We Can Now Detect AI-Written Content Thanks To One College Student
A Princeton student has created GPTZero, an app that can detect AI-generated writing.
A Princeton student has created an app that can detect text generated by AI. Edward Tian created GPTZero during his winter vacation. He says the software can “quickly and efficiently” determine if a person or ChatGPT wrote an essay. The 22-year-old is currently studying computer science and journalism at the University.
Speaking to The Daily Beast, Tian said he didn’t expect GPTZero to get much traction. “I was expecting, at most, a few dozen people trying out the app,” he told the publication. But a few hours after being uploaded to the hosting platform Streamlit, over 2000 people had signed up for the beta version. The site eventually crashed as word about the program spread.
Tian created the app to combat an increase in plagiarism through artificial intelligence. Since the release of ChatGPT, there have been several instances of students using the language model to pass off AI-written work as their own. So he took it upon himself to find a way to detect AI content. When his app debuted, Tain said people deserved to know where written works came from.
“There’s so much ChatGPT hype going around,” he said in a tweet. “Is this and that written by AI? We as humans deserve to know!” GPTZero uses two metrics to detect AI. This includes “perplexity and burstiness.” Text copied into the app is assigned numbers for both units of measurement. If the number is low, a bot probably created the text.
Perplexity is a metric that gauges the degree of randomness in a sentence. When writing employs an unexpected word usage or structure, it tends to yield a higher perplexity score. Tian mentioned that he leveraged the capabilities of GPT-2, a free and open-source language model, to train his app for this particular metric that helps detect AI.
Burstiness refers to the level of randomness within a given text. When it comes to human writing, sentences exhibit a wide range of complexity. Some are simple, and others are highly intricate. But AI tends to generate sentences that are often relatively low in complexity, maintaining a consistent level throughout the entire text.
“There are beautiful qualities of human written prose that computers can and should never co-opt,” Tian told The Daily Beast, explaining that he was inspired to create the app after reading the works of four-time Pulitzer Prize finalist John McPhee. Although his app can detect AI, the 22-year-old isn’t against the technology.
Tian believes that there is a place for AI if it is ethically used with consent. “I’m not opposed to using AI for writing when it makes sense,” he said, admitting that he has used programs like CoPilot to support his coding. Although he plans to grow the app with explainers and other cool stuff, Tian has to focus on school for now.
Meanwhile, OpenAI has indicated they intend to introduce watermarks to detect AI-generated text in written works. But there is currently no established or official method of accomplishing this task. The absence of a reliable verification mechanism can create significant challenges for educational institutions and other creative sectors. It remains to be seen how much of a difference GPTZero will make.
