Methods for Detecting Suspicious Information From Individual Transactions of Pharmaceutical Products via Twitter (now X): Retrospective Observational Study

Background Individual transactions involving pharmaceutical products via social networking service (SNS) are considered an inappropriate distribution route and may serve as a guise for illicit business-to-consumer activities. In Japan, individual transactions of pharmaceutical products via the internet are recognized as inappropriate distribution routes, which not only lead to the inappropriate use of pharmaceutical products but also require more active monitoring and guidance from the viewpoint of pharmaceutical security and quality assurance. Objective This study aimed to develop a method to accurately detect SNS tweets suspected of involving individual transactions of pharmaceutical products, using text data from Twitter (subsequently rebranded as X), the primary platform for such activities in Japan. Methods We applied text mining to 1389 text tweets suspected of involving individual pharmaceutical transactions. Using the hashtag “#Okusuri mogumogu,” which was identified through manual searching and is commonly associated with trading psychotropic pharmaceuticals, we collected 7499 tweets posted in 2022 and 6461 tweets posted from January 1 to March 31, 2023, using our web crawler program. After manually categorizing whether each tweet was related to individual pharmaceutical transactions, we extracted words and summarized their occurrences and frequencies using the 2022 dataset. A decision tree model was then generated using the 2022 dataset and validated using the 2023 dataset to evaluate the reliability of detecting transaction-related tweets. Results Using web crawling, the number of tweets identified using the hashtag “#Okusuri mogumogu” was 7499 in 2022 and 6461 in the first 3 months of 2023. The crawling results also showed that the number of detectable tweets increased closer to the crawl date, suggesting that SNS tweets may frequently be deleted. From 3228 extracted words in the 2022 dataset, 452 were significantly associated with tweets suspected of involving individual transactions. Highly indicative terms included “kyuu” (request), “yuzuri” (transfer), “DM” (direct message), and transaction-related hashtags. The chi-square automatic interaction detection model demonstrated stable discriminative performance (area under the receiver operating characteristic curve values: training 0.83 and 0.84; Gini coefficient: training 0.65 and test 0.68). The overall accuracy using the 2023 validation dataset was 82.31%, indicating reasonable generalizability despite linguistic fragmentation and the presence of partial word forms characteristic of Japanese text. Conclusions Using transaction-related tags, text mining, and machine learning, we identified key terms linked to individual pharmaceutical transactions and developed a predictive model. This approach may aid in preventing inappropriate online transactions of pharmaceutical products.