Detection of AI-generated Text: An Experimental Study

Artificial Intelligence is an important area in the computer science domain with a wide range of capabilities. With many people saying that they have encountered AI-generated content in the past years very few respondents feel confident in their ability to distinguish between real and fake content. This work aims to explore methods that effectively detect the content generated by AI. The investigation of the available online tools led to the determination of their limitations. Consequently, two separate algorithms were employed for detection and proper analysis: one is a classification-based approach that uses the Support Vector Machine (SVM) model for binary text classification, and the other is a statistical method that analyzes linguistic aspects to distinguish between writing generated by AI and humans. Both normal data, as well as rephrased data, were employed during the stress testing of these algorithms. The accuracy obtained on the raw dataset was very promising. The detection was 99.87% accurate. The accuracy rate of detecting rephrased texts reached 87.58% for the classification model and 84.91% for the statistical model. Additional analysis and observations were made regarding the ability of AI to generate text similar to humans.