3

With advances in AI, it is becoming clear that students are using paraphrasing software/sites (spinbot, quillbot, rephraser, etc.). I wonder if there are any free tools that can detect semantic similarity for all submitted assignments? I tried several of the available websites and they failed to detect the paraphrased text.

Improve this question
4
  • 8
    I think you are overly optimistic about the capabilities of AI at the moment - and probably for a long time to come.
    – Buffy
    Apr 30, 2021 at 23:47
  • 4
    @Buffy I’d say you’re pessimistic about what already exists. With GPT-3 you can generate whole essays from scratch, and judging from what I’ve heard on this site, they’re better than the average college essay.
    – knzhou
    May 3, 2021 at 21:50
  • 1
    @knzhou Generation seems easier than detection?
    – user2768
    May 5, 2021 at 7:47
  • 5
    @knzhou I don't know just how bad the average college essay is, but GPT3's abilities only stretch to producing text that reads well. The information it contains will have little-to-no basis in reality. It will happily include great detail about people who never existed and events which never happened. It's certainly quite impressive and might produce something that would fool somebody with no knowledge of the field the essay covers, but it's a long way from being able to "generate whole essays" that aren't >90% fictional rambling.
    – Chris H
    May 5, 2021 at 9:07

3 Answers 3

Reset to default
2
+100

You can try using Sentence-Bert, original paper here: https://arxiv.org/abs/1908.10084

Github link here: https://github.com/UKPLab/sentence-transformers

It is very easy to use since I had used that before and it took me just several minutes learning how to use it.

The usage is fairly simple, you generate embedding for each assignment, and compare the cosine similarity between each pair of assignments embeddings.

Then you can manually look at the ones that have very high cosine similarity.

The Github repository descriptions said: This framework provides an easy method to compute dense vector representations for sentences, paragraphs, and images. The models are based on transformer networks like BERT / RoBERTa / XLM-RoBERTa etc. and achieve state-of-the-art performance in various task. Text is embedding in vector space such that similar text is close and can efficiently be found using cosine similarity.

Improve this answer
1
  • 2
    If you use any automated tools for this, make sure that you have policies in place to handle the inevitable false positives. Just ignoring the problem is unethical. Excessive false negatives are also problematic, but all false positives need a fair method of adjudication.
    – Buffy
    May 8, 2021 at 16:35
2

Although there may be free tools to try to "detect" essays that were written with the help of AI, I would not fully trust them and waste my time trying to find something that helps. Because, eventually, there will be another software that will be able to "fool" your software. This means you will have to find yet another software that can outperform the software that "fooled" your previous software.

AI products can be much better at a specific task than humans. There are so many examples where AI beats people at a task the people are the best at. For example, Google's AI beat the best Go (one of the most complicated board game) player in the world (click here to watch the video).

Eventually most AI models are built in a way that they learn from humans. Initially they perform pretty poorly. They start to slowly get better and better until they reach a point where they can easily outperform humans.

The question is, how can we detect a robot that does (or will do soon) something as well as humans do (or potentially much better than humans)? Therefore, I think the problem should be solved in a different way (not involving another software that does the opposite). Because finding another software may only work temporarily, but not in the long run.

Improve this answer
0

Paper Rater has a robust Artificial Intelligent Plagiarism Detection tool. Please Note: The free tool is suboptimal.

More on Paper Raters Artificial Intelligent Plagiarism Detection here - you can see an example report.

Improve this answer
3
  • I copied some text from the internet and Paper Rater returned "This paper does not exhibit any signs of plagiarism. Well done!"
    – Thomas Lee
    May 9, 2021 at 15:47
  • Yes, that is the response using the Free Tool. Using the paid tool, it uses AI to compare text semantically after paraphrasing and detect plagiarism. Another tool of interest, (non-AI) could be text-compare.com.
    – Andrew
    May 10, 2021 at 0:57
  • The paid tool can compare a document with online documents, but I don't think with other assignments.
    – Thomas Lee
    May 10, 2021 at 1:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .