Menu Home

AI Content Detector 简单评测

师姐和我吐槽,因为她related work里头有一段比较像chatgpt辅助完成的内容,师母把会议投稿撤销了。

我没读过师姐的论文,但感觉这个标准太过严苛了。chatgpt生成的内容,当然不能直接放到论文,但是在辅助写作方面,chatgpt确实可以起到很大的帮助。

出于好奇,尝试了几个免登录AI Content Detector。

把自己写的上一份工作的abstract和related work的一段放了上去。依稀记得,两部分都通过chatgpt辅助修改过,但是改的不满意,最后只采纳了很少一部分表达。

检测内容1:

Hyperlink prediction aims to predict interactions among multiple entries, constituting a practical yet challenging problem in the literature.
While a handful of solutions have been proposed, they generally operate on the entire hypergraph.
A practical subgraph-based solution not only enables better identification of localized characteristics of the central hyperedge but also alleviates scalability concerns.
In this study, we present SSF, an innovative hyperlink prediction methodology based on \underline{S}ubgraph \underline{S}tructural \underline{F}eatures.
The rationale behind SSF is that hyperedges and non-hyperedges exhibit distinct local patterns, which can be unveiled through the assimilation of subgraph structural features.
To this end, we utilize well-established structural heuristics such as walks and loops as the fundamental building blocks.
We commence by extracting a subgraph encompassing each focal hyperedge, subsequently integrating an edge weakening scheme to facilitate feature extraction from the initial subgraph and its variations.
The extracted feature vector is interpretable, and the designed edge weakening scheme empowers SSF with an adaptive capability to handle hypergraphs with varying densities.
Lastly, a multilayer perceptron classifier is trained for prediction. Experiment results on ten real-world hypergraph networks demonstrate the effectiveness of the proposed approach.

  • https://undetectable.ai/     100% HUMAN
  • https://huggingface.co/spaces/Hello-SimpleAI/chatgpt-detector-single  Human Prob 0.99956
  • https://huggingface.co/openai-community/roberta-base-openai-detector Real 1.0
  • https://openai-openai-detector.hf.space/ Real 99.96%
  • https://gptzero.me/  94% AI generated 6% human (interesting)

检测内容2:

An important line of traditional link prediction algorithms utilizes similarity scores of node pairs as the likelihood of corresponding edges.
Typical methods include Common Neighbors (CN) \cite{liben2007link}, Katz index \cite{katz1953new}, and Resource Allocation (RA) \cite{zhou2009predicting}.
CN and Katz have been generalized to hypergraphs and used as comparative methods in \cite{zhang2018beyond}, and they did not demonstrate satisfactory performance.
In the literature \cite{newman2018networks}, these two similarity indices have been interpreted from the perspective of random walks.
Analogically, a loop can also be defined by a walk that starts and ends at the same node.
Pan et al. \cite{pan2021predicting} proposed using the loop feature for hyperlink prediction.
This study considers the candidate hyperedge as a perturbation to the observed hypergraph and assesses the impact of this perturbation on the loop structure within the hypergraph.
Specifically, the loop feature is defined by the count of loops with increasing hops at both node-level and hyperedge-level, based on the permutation extraction of which a logistic regression is trained for prediction.
The loop feature has shown promising results in sparse metabolic networks.

  • https://undetectable.ai/ 100% HUMAN
  • https://huggingface.co/spaces/Hello-SimpleAI/chatgpt-detector-single Human Prob 0.99933
  • https://huggingface.co/openai-community/roberta-base-openai-detector Real 1.0
  • https://openai-openai-detector.hf.space/ Real 99.98%
  • https://gptzero.me/ 83% Human,15%mixed,2% AI

检测结果里,只有gptzero.me的结果比较奇怪,不知所以。

那么有个问题,Human-made 与 ai-generated 的边界在哪里?ai从人类产生的数据中生成内容,这些内容又为人所使用,产生新的“人类数据”,二者似乎已经密不可分。

简单推测 ai content detector 的构建思路:
首先,这是一个二分类问题,而构建分类器需要有足够的有标签数据。
从数据采集上看,chatgpt元年之前的所有已发表内容,大概率可以认为是Human-made,之后,通过chatgpt这些工具直接生成的内容可以认为是ai-generated。正负样本都有了,就emmm。

再做一个推测,随着时间推移,越来越多的 human+ai content 会出现,文本上的 ai content detector 的结果会越来越差,直到它们失去意义。

下一个阶段,是特殊数据例如图像和视频的ai检测!尽管图像生成已经可以做到比较强大了,经常还是一眼假或者两眼假。但是,近几年它一定会发展到人眼分辨不出来的程度。联想到英剧 真相捕捉,视频以假乱真也是如此。

update: 视频以假乱真好像已经做的很好了,e.g., https://anythinginanyscene.github.io/,期待更普通的场景和更长的连续视频生成能力!

Categories: Tech

Tagged as:

peiyan