Deep learning architectures deployed for traffic sign recognition within advanced driver assistance systems show stark performance divergences across benchmarking protocols, utilizing the open-source German Traffic Sign Recognition Benchmark dataset to evaluate custom convolutional networks, standard architectures, and newer transformer-based models.
- Custom Convolutional Neural Networks and Reproduced CNN models achieved validation accuracies up to 97.82%, demonstrating high baseline efficiency for edge computing implementations in autonomous vehicles.
- Dataset partitioning strategies vary across model families, with traditional convolutional networks utilizing an 80/20 train-validation split and transformer implementations applying a 90/10 split on identical training subsets.
- Advanced architectures like VGG16 and transfer learning variants display significant validation loss fluctuations, dropping to approximately 60% accuracy during intermediate epochs before stabilizing.
The German Traffic Sign Recognition Benchmark dataset, sourced via Kaggle as detailed in Nature, comprises 43 distinct traffic sign categories designed for machine vision classification tasks. Each image is structured as a 32×32×3 array of pixels in the RGB color space, utilizing integer values ranging from zero to 255.
The dataset contains 39,209 images designated for training data and 12,630 images allocated specifically for independent testing. These samples feature varying ambient light conditions and complex, rich environmental backgrounds that test model robustness under real-world driving scenarios.
| Model Architecture | Dataset Split Strategy | Validation Accuracy |
|---|---|---|
| Custom CNN | 80/20 Train/Validation Split | 97.82% |
| VGG16 Transfer Learning | 80/20 Train/Validation Split | approximately 94% |
| Reproduced CNN | 80/20 Train/Validation Split | approximately 93–95% |
Different model families in the evaluation study adopted distinct split conventions before final evaluation against the 12,630 held-out test images. Benchmarking models including the Custom CNN and VGG16 applied a custom 80/20 stratified train-validation split to the official training set using a fixed random seed of 42.
Transformer-based architectures such as ViT-B/16 and DeiT-Tiny employed a 90/10 stratified train-validation split on the same training data. Meanwhile, YOLOv11 reorganized its training set into an 80/20 split after converting annotations into the required YOLO format.
The reproduced Custom CNN achieved a validation accuracy of 97.82%, a training F1-score of 97.25%, and a training precision of 98.37% with a validation loss of 0.0868. These metrics represent a notable improvement over the baseline validation accuracy range of 94% to 96% reported in prior comparative studies.
The validation accuracy curve for the Custom CNN exhibited noticeable fluctuations across epochs, dropping to approximately 77% at epoch 5 before recovering to 96% by epoch 6. This behavior indicates a sensitivity to batch ordering and residual overfitting to specific augmented batches during the training phase.
The VGG16 transfer learning model implemented via TensorFlow Keras applications reached a final validation accuracy of approximately 94%. However, its intermediate training metrics exhibited considerable instability, with validation accuracy dropping to approximately 60% at epoch 4 before recovering by epoch 8.