Computer-Vision Based Automatic Rider Helmet Violation Detection and Vehicle Identification in Indian Smart City Scenarios Using NVIDIA TAO Toolkit and YOLOv8

Uttam U. Deshpande^*Vaidehi DeshpandeRamesh KotiRudragoud PatilRamchandra Alias Ameet ChateVarun R TandurSupreet S GoudarShreya IngaleVaishnavi Charantimath

• KLS Gogte Institute of Technology, Belagavi, Belagavi, Karnataka, India

Two-wheeler traffic offenses are a well-known fact about the Indian Road scenario. In addition to endangering the offenders, these offenses also endanger other commuters. Two-wheeler traffic violations can take many different forms, such as overloading, triple riding, and helmetless riding. Effective identification and enforcement strategies are necessary for these offenses since they pose a serious risk to public safety. Due to the inadequacy of traditional traffic monitoring and enforcement techniques, advanced technology-based solutions are now required. Deep learning-based systems have demonstrated significant promise in identifying and stopping such infractions in recent years. We propose a two-step deep learning approach that leverages the strengths of pre-trained object detection models to detect two-wheeler riders and specialised helmet classifiers to identify helmet wear status as well as detect number plates. In the first stage, we utilised a highly efficient, robust, and accurate object identification DetectNet (Model 1) framework developed by NVIDIA, and it uses the ResNet18 Convolutional Neural Network (CNN) architecture as part of the Transfer Learning Toolkit known as TAO (Train, Adapt, Optimize). The second stage demands accurate detection of a helmet on the identified rider and extracting numbers from the violator's license plates using the OCR module in real time. We employed YOLOv8 (Model 2), a deep learning-based architecture that has proven effective in several applications involving object detection in real time. It predicts bounding boxes and class probabilities for objects within an image using a single neural network, making it a perfect choice for real-time applications like rider helmet violations detections and number plate processing. Due to a lack of publicly available traffic datasets, we created a custom dataset containing motorcycle rider images captured under complex scenarios for training and validating our models. Experimental analysis shows that our proposed two-step model achieved a promising helmet detection accuracy of 98.56% and a 97.6% number plate detection accuracy of persons not wearing helmets. The major objective of our proposed study is to enforce stringent traffic laws in real-time to decrease rider helmet violations.