{"id":72553,"date":"2023-07-17T16:00:00","date_gmt":"2023-07-17T23:00:00","guid":{"rendered":"https:\/\/blog.mozilla.org\/?p=72553"},"modified":"2025-04-15T16:03:57","modified_gmt":"2025-04-15T23:03:57","slug":"how-to-tell-if-chatgpt-wrote-text","status":"publish","type":"post","link":"https:\/\/blog.mozilla.org\/en\/internet-culture\/how-to-tell-if-chatgpt-wrote-text\/","title":{"rendered":"Did ChatGPT write this? Here\u2019s how to tell"},"content":{"rendered":"\n
\"An
Credit: Mozilla Foundation<\/figcaption><\/figure>\n\n\n\n

The AI<\/a> wars are heating up. In late 2022, OpenAI\u2019s ChatGPT<\/a> made headlines for showing us what a new search engine could look like. ChatGPT (which stands for \u201cGenerative Pre-trained Transformer\u201d) is a chatbot \u2014 one that can process queries and spit out relevant information to answer questions about historical facts, recipes, car dimensions and lots more. As a bonus, ChatGPT lets you word questions in plain English, so you\u2019re not forced to write queries like \u201chow to stop dog pooping everywhere reddit.\u201d The result is, essentially, a search box that you can message back and forth with. It almost makes Google search look a little primitive. Microsoft, the maker of Bing and biggest investor in OpenAI, is okay with this<\/a>.<\/p>\n\n\n\n

ChatGPT, and the latest release GPT-4, provides thorough answers \u2014 it can even write your code<\/a>, write your cover letter<\/a> and pass your law exam<\/a>. It also provides thoroughly wrong answers sometimes. It\u2019s worrying how confidently ChatGPT presents inaccurate information<\/a>. That hasn\u2019t stopped newsrooms<\/a> from rethinking how many writers they hire nor professors from coming out against<\/a> the chatbot. (Though not all professors. Some embrace the change<\/a>.)<\/p>\n\n\n\n

The excitement around artificial intelligence is anything but artificial. At least for some. College professors or job recruiters are less than excited<\/a> to have to discern human words from chatbot chatter. Industry experts are less than enthused for a potential wave of misinformation, signing an open letter<\/a> that warns of AI\u2019s potential to \u201cflood our information channels with propaganda and untruth.\u201d Those who have signed say \u201csuch decisions must not be delegated to unelected tech leaders.\u201d Issues like this are exactly what Mozilla seeks to address with the Responsible Computing Challenge<\/a>, ensuring higher education programs emphasize tech\u2019s political and societal impact. And also with Mozilla.ai<\/a>, a startup with the mission of making it easy to create AI that\u2019s open source and ethical.<\/p>\n\n\n\n

As we enter this brave new world where even a friend\u2019s Snapchat message<\/a> could be AI-written, you might want to know a bit more about chatbots\u2019 capabilities and limitations. Can you spot a paragraph written by AI? Can you tell if your coworker is actually responding to you and not ChatGPT<\/a>? Do you know how to spot misinformation<\/a> within a chatbot\u2019s answers because ChatGPT-infused Bing definitely still gets facts wrong<\/a> at times? It’s not always possible to know if an AI wrote some copy, but sometimes, you can detect language written by ChatGPT and other bots by using a detector tool and watching for awkward language. Read on to learn how.<\/p>\n\n\n\n

How to detect ChatGPT text yourself?<\/h2>\n\n\n\n

You can detect ChatGPT-written text using online tools like OpenAI API Key<\/a>. The tool comes from OpenAI, itself, the company that made ChatGPT. It\u2019s worth noting that the app isn\u2019t perfect. OpenAI says the tool needs at least 1,000 words before it can sniff out AI-generated content, so something like an AI-generated text message may fly under its radar. Also, even if it gets the 1,000 words it needs, it isn\u2019t always 100% accurate at detecting AI vs human written language. AI-made text that has been edited by a human can also fool the tool<\/p>\n\n\n\n

(<\/em>Update<\/em><\/strong>: As of July 2023<\/a>, a new ChatGPT update no longer includes the use of the AI classifier which Open AI used to detect AI-generated text and, as of early 2024, the company has even taken their original tool offline. The company claims to be working on new, more effective ways of detecting AI-generated text as well as AI-generated audio and video.)<\/em><\/p>\n\n\n\n

OpenAI\u2019s tool may not be perfect but there are other offerings in the ChatGPT text detection world. The Medium blog Geek Culture<\/a> lists other options made by folks at Princeton<\/a> and Stanford<\/a>. If it\u2019s critical to know if text was written by a bot or a human, testing it on multiple tools might help. ChatGPT is changing quickly so your mileage may vary.<\/p>\n\n\n\n

Detecting ChatGPT text: The caveats<\/h2>\n\n\n\n

It\u2019s important to emphasize that no method of detecting AI-written text is foolproof \u2014 that includes options using tools available today. Jesse McCrosky is a data scientist with Mozilla Foundation who warns of AI text detection tools\u2019 limitations. \u201cDetector tools will always be imperfect, which makes them nearly useless for most applications,\u201d says McCrosky. \u201cOne can not accuse a student of using AI to write their essay based on the output of a detector tool that you know has a 10% chance of giving a false positive.\u201d<\/p>\n\n\n\n

According to McCrosky, it can be impossible to ever have a true AI-detector because it will always be possible for software to write \u201cundetectable\u201d texts or create text with the specific intent of evading these sorts of detectors. And then there\u2019s the fact that the AI tools available to us are always improving. \u201cThere can be some sense of an \u2018arms race\u2019 between ChatGPT text detectors and detector-evaders, but there will never be a situation in which detectors can be trusted,\u201d says McCrosky.<\/p>\n\n\n\n

How to spot misinformation within ChatGPT?<\/h2>\n\n\n\n

It\u2019s no secret that ChatGPT can (and has<\/a>) been spreading misinformation and disinformation. Microsoft may be using<\/a> tools like those from NewsGuard to limit the misleading responses its AI gives, but the issue is still cause for concern. The Poynter Institute<\/a> has our favorite tips to spot misinformation within ChatGPT: 1) check for patterns and inconsistencies, 2) look for signs of human error and check the context. If a ChatGPT answer repeats something multiple times, has weird errors that a person wouldn\u2019t make or says something that doesn\u2019t make sense in the context of what you\u2019re reading, you might be reading misleading content. Check the source links at the bottom of your ChatGPT responses and make sure you do your own research outside of ChatGPT too. Treat it as a starting point and not the final word.<\/p>\n\n\n\n

ChatGPT is fun, but watch out<\/h2>\n\n\n\n

ChatGPT offers an interesting glimpse into a chatbot answer-filled world, but it also acts as a warning of the downsides too. With great smarts comes great responsibility. As Bing and ChatGPT (and Sydney<\/a>?) learn how to be a better chatbot, we as the users will have to continue to use ChatGPT detection tools to verify that the words we\u2019re seeing are human-made and that the facts we\u2019re sharing are indeed factual.<\/p>\n\n\n\n

\n\n\n\n

Did Chat GPT Write This? Here\u2019s How To Tell<\/a><\/em><\/strong>

Written by: Xavier Harding<\/em>
Edited by: Ashley Boyd, Audrey Hingle, Carys Afoko, Innocent Nwani<\/em>
SEO Insight: Aslam Shaffraz<\/em>
<\/p>\n\n\n\n

For further reading on AI, check these out from the Mozilla Foundation:<\/strong><\/em><\/p>\n\n\n\n