Can, Barely: Recognize Its Own Work
In late January 2023, OpenAI released a detection tool for AI-written text, to a chorus of joy from teachers. Hooray! Perhaps AI won’t cause a flood of cheating. But beware, the detector isn’t so robust. From my colleague Lauren Leffer’s story:
The application is intended to classify text samples based on how likely they are to have been generated by artificial intelligence vs. written by an actual person. Given a sample of text, it spits out one of five possible assessments: “Very unlikely to have been AI-generated,” “unlikely,” “unclear,” “possible,” or “likely.”
However, in OpenAI’s own tests, the tool only correctly identified generated text as “likely AI-written” about a quarter of the time. Moreover, about one in ten times, the classifier falsely lists human-made words as computer-generated, the company noted in a blog post.
According to OpenAI, even these meh results are an improvement on the company’s previous stab at AI-text detection. And the tech startup acknowledged that, thanks to its own invention, we need improvement.
