Abstract

The increasing use of AI-generated text has introduced new challenges in detecting paraphrased content, particularly in lesser-resourced languages. This study investigates the effectiveness of different algorithms in identifying ChatGPT-paraphrased texts, focusing on the impact of word unigram and character multigram features, classification algorithm performance across English and Serbian corpora, and the comparative efficiency of commercial detectors like ZeroGPT against custom models. Additionally, it examines the role of syntax analysis and model temperature in influencing AI-generated text structures. A quantitative methodology involving classification algorithms, feature set evaluations, and cross-linguistic comparisons is employed. Results indicate that tailored algorithms outperform commercial detectors, especially when incorporating syntactic features. The study underscores the necessity for language-specific approaches to enhance detection accuracy and proposes directions for future research in AI text detection.