Table of Contents
UPDATED 7th Feb: Added checks with OpenAI’s official detection tool, a link to Googles paper on how they could detect AI content & links to the source content to download.
Thinking about using AI content for your website and SEO? The results of this case study will change your mind.
I am going to show you precisely how easy it is to detect content generated by several different AI’s using public tools.
Don’t forget that:
So I wanted to see how difficult it was to detect AI generated content. The process was very simple:
- Generate AI content using 9 different tools
- Run all of that content through 6 AI content detectors and 2 plagiarism checkers
And just to spice things up a little bit, I also decided to include some of CNET’s Ai generated content (full case study) and some human spun content that I originally published in 2014.
The results weren’t what I was expecting!
The Case Study Results
In the graphic below you can see exactly what happened from testing each of the top AI content generators against several detection methods (more details on each below).
You can click on any symbol in the table below to see the screenshot of the results for each individual test.
(Download the source content used for the tests here)
Straight off the bat, you can see that-
- Rytr
- ArticleForge
- ChatGPT
- GPT2
Failed most of the AI detector tests by a long way.
ChatGPT (often hailed as the “most advanced AI content generator”) performed the worst out of all the AI tools:
- It passed none of the AI detectors.
- It had a big problem with plagiarism.
You can also see that the AI content published on CNET’s website was detected as 100% AI. Take a look at our full CNET Ai content detection case study to learn more.
But two AI content generators stood out from the rest…
They passed most of the AI content detection tools even though we used the quickest methods to generate the test content.
But here’s the huge problem…
Frase and Jasper were both cited for plagiarism by Quetext.
This essentially means that the AI content they produced isn’t unique.
That’s a big problem because it’s no secret how seriously Google takes plagiarised, duplicate content.
In fact, none of the AI writing tools could pass the plagiarism test from Quetext at all. So if Quetext detects that the content is not unique, is likely that Google can as well.
Interestingly the only 2 pieces of content that passed every test was my own content along with some human written content that was spun on the paragraph, sentence and word level by a human.
And that makes sense because they were both written by actual humans.
The AI Content Creation Tools Tested
I tested 8 of the most popular AI content creation tools available right now.
Below is a quick breakdown of each of the AI content creation tools we tested:
Jasper
Jasper is the market leader in the AI content creation market.
They have a much more advanced AI writing system than any other tool I tested.
The legendary Jasper “boss mode” is designed to produce long-form content such as blog posts and articles.
It easily produced the best quality content out of all the other AI content tools I tested in the case study.
Jasper also crawls more of the web than most other AI content tools.
That means it generally has more content to use as context, which significantly increases the quality of its output.
Frase
Frase is a powerful tool that started out by helping writers create website content and optimise their content perfectly to rank higher in Google.
It has evolved significantly over the last few years, developing its own AI writer to produce content faster. They recently signed a deal with the Copyrytr group, which has a collection of AI tools including:
This has helped them develop their own AI content generation models faster.
Check out my complete Frase review to learn more about it.
Rytr
Rytr is primarily an AI writing assistant that is best at coming up with different angles for copywriters, marketers and entrepreneurs.
Rytr is one of the most affordable AI writing tools with a cost-effective unlimited content writing plan. This has made it extremely popular over the last couple of years.
Kafkai
Kafkai is a decent AI content generator that specialises in producing SEO-optimised content for blogs and articles.
Kafkai uses natural language processing (NLP) and advanced AI models to produce long-form content that reads well.
ArticleForge
ArticleForge is also primarily designed for long-form content.
It can produce SEO optimised 1500 word articles in less than 60 seconds – with the click of a button.
ArticleForge is popular because the user interface makes it easy to use. You type in your target keyword and ArticleForge will produce the article.
All sounds good, right?
The problem is the quality. The AI test results show that the content…
- Isn’t unique.
- Is easily detected as AI-written content.
ChatGPT
There is a lot of hype about ChatGPT at the moment.
The premise is simple – Ask ChatGPT to write something (or ask a question) and get back a well-worded piece of content in a number of seconds.
ChatGPT is currently free to use which is why it has gone viral all over the internet but it failed all of our detection testing.
GPT2
GPT2 is another free AI content generator run by Watt AI.
It’s simple and fun to use but takes a lot of time if you want to produce anything decent.
GPT2 is best known for creating entertaining and amusing content from a story-driven perspective.
CNET AI Content
CNET launched a silent experiment by publishing finanical advice written by AI which has caused an awful lot of controversy.
However in this case study we were able to detect AI content in 87% of their AI created content:
While CNET haven’t released any specific details on what tools or processes they are using to create AI content, it seems they need a lot of improvement from a detection stand point.
Human Spun Content
In 2014 I published a tutorial that showed you how to fool duplicate content detection with human spun content.
The content in that tutorial was spun at the paragraph, sentence and word level by a human hand. Not only that but I actually shared the full article on my blog as an example, so I decided to take an output of that as a wild card test against AI content.
The AI Content Detection Tools We Used
So what were the AI content detection tools we used in this case study?
Here is a quick breakdown of what they do and what makes them unique.
Originality
Originality was by far the best AI contect detection tool that we tested both in this case study and our CNET AI content case study.
That means their AI detector is built to catch out AI-written content and ensure it is completely plagiarism free.
Originality is good at what it does and is continuously being developed to stay up to date with modern AI writers.
QueText
Not a single piece of AI generated content was able to pass the Quetext test.
It uses advanced DeepSearch technology, which understands the context of the written content. It then checks it against billions of sources in its content index.
And although its not an AI content detection tool per say, it is the most advanced plagarism detection tool on the market. AI learns from the content you feed it, so duplication is expected at some stage.
Open AI Text Classifier
The OpenAI team (the team that made ChatGPT) released a free tool on the 31st January 2022 to helo detect AI content:
This is the first official AI content detection tool as it is made by the OpenAI team in an effort to combat AI content being used where it shouldn’t be, like a thesis for example.
GPTZero
GPTZero is an advanced AI detector that was inspired to stop ChatGPT.
It uses advanced variables to track AI models and detects their digital fingerprints.
They are currently developing their technology for educators to stop students from taking advantage of AI for college and school assessments.
Writer
Writer is one of the most popular AI content detection tools around.
But Writer isn’t just an AI detector. It’s also an AI content writer.
The detector is designed to help people use AI-written content without being detected by search engines like Google.
OpenAI Detector
OpenAI detector is a free-to-use public AI content detector.
It was initially developed for GPT2 AI content.
The team behind it have been working hard on it and have developed a simple-to-use tool that does a decent job detecting at now detecting GPT3.
Content at Scale
Content at Scale is similar to Writer because the tool was designed to produce AI-written long-form content fast.
They have now added an effective AI detector to check your AI-written content before publishing it online. The detector is very good at detecting low-quality AI content.
Copyscape
Copyscape is about as simple as a plagiarism checker gets.
But just because it’s simple doesn’t mean it’s not effective. Copyscape has a full suite of plagiarism tools ready to detect copied content.
AI Content Detection Summary
Can Google detect content written by AI content creators?
Short answer – Yes because if we can do it with publically available tools, so can Google who have invested more in AI than any other tech company.
Not to mention the fact Google published a paper in 2020 showing how they can use text generation models to detect low quality content.
A simillar test by Jon Gillham, comprehensively showed that AI content detectors could easily detect AI-written content as did Surfer.
It’s also likely that email spam filters (the biggest is opearated by Google) and social media networks will start to integrate AI content checkers in the future. T
his could mean that your emails go directly to spam or posts on popular social media sites get limited reach.
So are AI based content generators valuable at all?
Yes, if you use them properly.
AI generators are great for:
- Generating ideas
- Generating outlines
- Coming up with new angles
…for human writers.
From my testing, it’s clear to me that the best AI tools can come up with some amazing ideas and angles for content creation.
But when it comes to actual content writing for a website – stay away!
If Google decides to penalise sites using AI content in the future, it could completely destroy an entire website.
Don’t get caught up in the hype.
Protect your site and avoid using AI writers for website content or anything to do with SEO. We won’t be using it to create social updates or emails either!
