AI-Model For Probabilistic Assessment and Classification of Harmful Digital Content

In today’s digital landscape, the sheer volume of harmful content necessitates smarter, more adaptable automated moderation techniques. This research introduces a novel, two- pronged AI approach designed for the probabilistic identification and categorization of harmful text, such as toxic comments, insults, threats, and obscenity. By thoughtfully integrating the speed and interpretability of logistic regression with the advanced contextual understanding of fine-tuned BERT models on the challenging Jigsaw dataset, we aim to overcome limitations inherent in traditional content moderation methods. Our method- ology uniquely incorporates a dynamic thresholding strategy, employing a grid search optimized for the F1 score, to achieve a more nuanced and balanced classification performance tailored to the specific characteristics of each harmful content category. Our experimental findings demonstrate a significant ability of this combined model to effectively differentiate between safe and harmful online communication, showcasing a promising and adaptive solution for real-time content moderation. Ultimately, this work highlights the substantial potential of intelligently blending machine learning classification techniques within digital platforms to cultivate more secure and inclusive online environ- ments for everyone.