In this article, we look carefully into the question of whether AI has taken over news. Specifically, what percent of recent news articles have been written by AI? The answer — to us anyway — was surprising. Only a small percentage of publications we sampled showed any signs of using AI at all, and the ones that did show signs of AI tended to be with low confidence.
Methodology: finding AI news
To perform this study, we used Google News. Recently, there have been rumors that Google is paying smaller news outfits to produce news using AI. Although the rumors have not been substantiated, they do seem consistent with what Google has been saying about AI in general.
In order to avoid bias, we only looked at smaller news outlets on Google News. The AP and Newsweek, given their resources, are unlikely to be using AI to write. We focused on small blogs and local news sources.
We also focused on specific categories, in order to avoid category bias. These include: entertainment news, business news, and science news. Other categories, like health, are less likely to be written by AI given the liability factors.
Our results: has AI news gone mainstream?
We ran an automated scan on 50 news articles that met our criteria, which is described above. The results of the scan are here:
In a major surprise to us, AI news is not as common as we thought. In fact, we found only a few articles that had high AI scores, and a handful of articles that had some probability of AI, but not a particularly high one.
It appears that even small blogs have not yet moved over to using AI to write. Their content is as human as ever, and it doesn’t appear that there is much movement toward AI news.
Future analysis
This study is something that we plan to continue. We expect to see the prevalence of AI increase over the next 18 months, and it will be interesting to plot the actual statistics against our assumptions. In further studies, we plan to look at other writing to see how it has changed. For example, blog articles and anonymous student-submitted writing.
Check back for these articles, as well as more studies about AI in writing.