Journal of Transportation Research

Journal of Transportation Research

Vehicle Trajectory Extraction via Real-time Object Detection and Tracking in Video Analytics-Based Surveillance Systems

Document Type : Original Article

Author
M.Sc., Grad., Department of Civil Engineering, Iran University of Science and Technology, Tehran, Iran
Abstract
A portion of the country's road network is equipped with surveillance cameras. These cameras are continuously monitored by human operators. Given the large number of cameras, this approach to traffic monitoring is not only labor-intensive and costly but also highly error-prone. Therefore, an automated framework with real-time processing capabilities is needed to extract vehicle trajectories that transportation researchers and practitioners can use to investigate microscopic vehicle behavior in various contexts. This study aims to design an advanced and efficient framework for extracting accurate and continuous vehicle trajectories using real-time object detection and tracking in video-analytics-based surveillance systems. In this study, the YOLOv10 algorithm was employed for object detection, while ByteTrack was used for object tracking. Recognizing the importance of access to a diverse, high-quality, and representative dataset, a dataset named Surveillance Cameras was developed from a variety of traffic scenes with the aim of providing high-quality annotations. Transfer learning was employed to fine-tune the pre-trained YOLOv10m model on this dataset. The fine-tuned model achieved precision, recall, and mean average precision values of 96.3%, 95.2%, and 98.3%, respectively, with an inference latency of 5.08 ms. The results demonstrate that the proposed framework can accurately detect and correctly classify most vehicles while ensuring a high frame rate in real-time applications. Within the tracking-by-detection paradigm, this performance improves the data association process and, while maintaining track-ID continuity across consecutive frames, enables the extraction of accurate, stable, and continuous vehicle trajectories. The proposed framework provides precise information on the positional changes associated with each track ID over time and can be applied to motion-pattern analysis, position prediction, traffic parameter extraction, and the identification of potential traffic conflicts.
Keywords
Subjects

Abdel-Aty, M., Wang, Z., Zheng, O., & Abdelraouf, A. (2023). Advances and applications of computer vision techniques in vehicle trajectory generation and surrogate traffic safety indicators. Accident Analysis & Prevention, 191, 107191.
 doi.org/10.1016/J.AAP.2023.107191
-Ait Ouallane, A., Bakali, A., Bahnasse, A., Broumi, S., & Talea, M. (2022). Fusion of engineering insights and emerging trends: Intelligent urban traffic management system. Information Fusion, 88,218–248.
doi.org/10.1016/J.INFFUS.2022.07.020
-Azimjonov, J., & Özmen, A. (2021). A real-time vehicle detection and a novel vehicle tracking systems for estimating and monitoring traffic flow on highways. Advanced Engineering Informatics, 50, 101393.
doi.org/10.1016/J.AEI.2021.101393
-Azimjonov, J., & Özmen, A. (2022). Vision-based vehicle tracking on highway traffic using bounding-box features to extract statistical information. Computers & Electrical Engineering, 97, 107560. doi.org/10.1016/J.COMPELECENG.2021.107560
-Bakirci, M. (2024). Enhancing vehicle detection in intelligent transportation systems via autonomous UAV platform and YOLOv8 integration. Applied Soft Computing, 164, 112015. doi.org/10.1016/J.ASOC.2024.112015
-Bayat, M. (2026, July 9). Surveillance Cameras Dataset.
-Bunaiya Prayudha, R. (2021). AddInsight-Vision An Intelligent Video System for Object Detection in Traffic Scenes and Vehicle Re-Identification.
-Christián, U. (2023). Real Time Vehicle Detection for Intelligent Transportation Systems.
-Derek Adam Gloudemans, B., Adams, D., Bayen, A., Biswas, G., & Laine, F. (2023). Interstate 24 Motion: A Multi-Camera Vehicle Tracking Instrument For Large-Scale Trajectory Extraction.
-Ding, B. M. (2023). Radar and Camera Fusion in Intelligent Transportation System.
-Emiyah, C. (2021). Framework For Extracting Relevant Transportation Metrics From Drone Surveillance Videos Using Computer Vision & Machine Learning.
-García-Aguilar, I., García-González, J., Medina, D., Luque-Baena, R. M., Domínguez, E., & López-Rubio, E. (2024). Detection of dangerously approaching vehicles over onboard cameras by speed estimation from apparent size. Neurocomputing, 567, 127057. doi.org/10.1016/J.NEUCOM.2023.127057
-Gomaa, A., Abdelwahab, M. M., Abo-Zahhad, M., Minematsu, T., & Taniguchi, R. I. (2019). Robust Vehicle Detection and Counting Algorithm Employing a Convolution Neural Network and Optical Flow. Sensors 2019, Vol. 19, 19(20),
4588—4589.
doi.org/10.3390/S19204588
-Gomaa, A., Minematsu, T., Abdelwahab, M. M., Abo-Zahhad, M., & Taniguchi, R. ichiro. (2022). Faster CNN-based vehicle detection and counting strategy for fixed camera scenes. Multimedia Tools and Applications, 81(18), 25443–25471.
 doi.org/10.1007/S11042-022-12370-9/TABLES/9
-Hassaballah, M., Kenk, M. A., Muhammad, K., & Minaee, S. (2021). Vehicle Detection and Tracking in Adverse Weather Using a Deep Learning Framework. IEEE Transactions on Intelligent Transportation Systems, 22(7), 4230–4242.
doi.org/10.1109/TITS.2020.3014013
-Heidari, A., Navimipour, N. J., & Unal, M. (2022). Applications of ML/DL in the management of smart cities and societies based on new trends in information technologies: A systematic literature review. Sustainable Cities and Society, 85, 104089. doi.org/10.1016/J.SCS.2022.104089
-Hislop-Lynch, & Richard, S. (2023). Capture, processing and analysis of vehicle trajectories from multiple unmanned-aerial-vehicles with computer-vision and artificial-intelligence [The University of Queensland]. doi.org/10.14264/AB80CF8
-Hnewa, M. (2023). Object Detection for Autonomous Systems Operating Under Challenging Conditions.
-Katariya, V. A. (2023). Advancing highway safety: embedded-edge ai for real-time applications. doi.org/10.1109/TITS.2022.3172015
-Li, J., Xu, Z., Fu, L., Zhou, X., & Yu, H. (2021). Domain adaptation from daytime to nighttime:
A situation-sensitive vehicle detection and traffic flow parameter estimation framework. Transportation Research Part C: Emerging Technologies, 124, 102946. doi.org/10.1016/J.TRC.2020.102946
-Li, T., Bian, Z., Lei, H., Zuo, F., Yang, Y. T., Zhu, Q., Li, Z., & Ozbay, K. (2024). Multi-level traffic-responsive tilt camera surveillance through predictive correlated online learning. Transportation Research Part C: Emerging Technologies, 167, 104804. doi.org/10.1016/J.TRC.2024.104804
-Mattingly, A. (2023). Deep Learning Ensembles for Lightweight Object Detection.
 -Mehrizi, A., Gholampoor, I., & Movahedian, H. (2021). Automatic Traffic Incident Detection Based on Monitoring Camera Image Processing System.
-Messoussi, O., Nicolescu, G., & Bilodeau, G. A. (2021). Real-Time Vehicle Detection and Tracking From Surveillance Cameras in Urban Scenes.
-Muhib, R. B., Ahmad, I. S., & Boufama, B. (2023). Deep Learning-Based Vehicle Classification.
In Lecture Notes in Networks and Systems: 814 LNNS. doi.org/10.1007/978-3-031-47451-4_18
-Ngeni, F., Mwakalonge, J., & Siuhi, S. (2024). Solving traffic data occlusion problems in computer vision algorithms using DeepSORT and quantum computing. Journal of Traffic and Transportation Engineering (English Edition), 11(1), 1–15. doi.org/10.1016/J.JTTE.2023.05.006
-O. Boadu, B. (2024). Dual Vision: Enhancing Autonomous Navigation With AutoGuardian CNN and YOLO Object Detection Synergy.
-Ogunrinde, I. O. (2023). Multi-Sensor Fusion for Object Detection and Tracking under Foggy Weather Conditions.
-Patil, J. B. (2021). Camera Video Data Processing of Traffic and Development of Microsimulation Traffic SUMO Model for Roundabout.
-Psalta, A., Tsironis, V., & Karantzalos, K. (2024). Transformer-based assignment decision network for multiple object tracking. Computer Vision and Image Understanding, 241, 103957.
doi.org/10.1016/J.CVIU.2024.103957
-Rezaei, M., Azarmi, M., & Mir, F. M. P. (2023).
3D-Net: Monocular 3D object recognition for traffic monitoring. Expert Systems with Applications, 227, 120253. doi.org/10.1016/J.ESWA.2023.120253
-Sharma, P., Singh, A., Singh, K. K., & Dhull, A. (2022). Vehicle identification using modified region based convolution network for intelligent transportation system. Multimedia Tools and Applications, 81(24), 34893–34917. doi.org/10.1007/S11042-020-10366-X/METRICS
-Shepelev, V., Aliukov, S., Glushkov, A., & Shabiev, S. (2020). Identification of distinguishing characteristics of intersections based on statistical analysis and data from video cameras. Journal of Big Data, 7(1), 1–23.
doi.org/10.1186/S40537-020-00324-7/TABLES/9
-Siddique, A. (2023). Downstream Task Self-Supervised Learning for Object Recognition and Tracking.
-Song, H., Liang, H., Li, H., Dai, Z., & Yun, X. (2019). Vision-based vehicle detection and counting system using deep learning in highway scenes. European Transport Research Review, 11(1), 1–16. doi.org/10.1186/S12544-019-0390-4/TABLES/5
-Sousa, R. M. P. de. (2023). Detection and Tracking of Red Traffic Light Infringements Using DNNs.
-Stryk, F. (2023). Vehicle Location Detection and Distribution from Camera Images. doi.org/10.1109/AVSS.2017.8078520
-VAŇO, J. (2024). Traffic surveillance system. In theses.czJ VAŇOtheses.cz.
-Wang, A., Chen, H., Liu, L., Chen, K., Lin, Z., Han, J., & Ding, G. (2024). YOLOv10: Real-Time End-to-End Object Detection.
-Yang, H. (2023). Customizing Trustworthy Machine Learning and Advanced Computing Methods for Cooperative and Equitable Traffic Systems.
-Yang, H. (Frank), Cai, J., Liu, C., Ke, R., & Wang, Y. (2023). Cooperative multi-camera vehicle tracking and traffic surveillance with edge artificial intelligence and representation learning. Transportation Research Part C: Emerging Technologies, 148, 103982. doi.org/10.1016/J.TRC.2022.103982
-Zhang, S. (2021). Improving Pedestrian Safety Using Video Data, Surrogate Safety Measures and Deep Learning. In Electronic Theses and Dissertations, 2020-2023. https://stars.library.ucf.edu/etd2020/593
-Zhang, Y., Sun, P., Jiang, Y., Yu, D., Weng, F., Yuan, Z., Luo, P., Liu, W., & Wang, X. (2022). ByteTrack: Multi-object Tracking by Associating Every Detection Box. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 13682 LNCS, 1–21. doi.org/10.1007/978-3-031-20047-2_1
-Zhou, S., Ng, S. T., Yang, Y., & Xu, J. F. (2021). Integrating computer vision and traffic modeling for near-real-time signal timing optimization of multiple intersections. Sustainable Cities and Society, 68, 102775. doi.org/10.1016/J.SCS.2021.102775