Journal of Transportation Research

Journal of Transportation Research

Proposing an Optimal Control Method for Energy Consumption Optimization in Railway Signalling Systems Using Reinforcement Learning

Document Type : Original Article

Authors
Mohammad Ali Sandidzadeh 1
Majid Azinfar 2
Naser Mozayani 3
Farzaad Soleymaani 4
1 Associate Professor, Faculty of Railway Engineering, Iran University of Science and Technology, Tehran, Iran.
2 M.Sc., Grad., Faculty of Railway Engineering, Iran University of Science and Technology, Tehran, Iran.
3 Associate Professor, Faculty of Computer Engineering, Iran University of Science and Technology, Tehran, Iran.
4 Postdoctoral Researcher, Faculty of Railway Engineering, Iran University of Science and Technology, Tehran, Iran.
10.22034/tri.2024.406433.3174
Abstract
Nowadays, the optimization of energy consumption in public transportation systems is a serious issue. Since a large part of energy in transportation systems is consumed by subways, a new approach has been proposed for optimal control of a train to reduce energy consumption. The proposed model is based on the Reinforcement Learning algorithm. It is assumed that a train moves between two stations along a line with non-constant gradient, curve, and speed limits. Moreover, the train should complete its journey within a given time interval. The Reinforcement Learning of States, Actions, and Rewards are based on the selected Actions. In the proposed method, the train States are the velocity and position of the train, and the Action is acceleration or coasting motion. Unlike the former techniques, most stages of optimization in this method are offline and implemented only once for any route. Following the formation of the reward matrix, we could use this method in an online form and then the speed profile could be produced at a minimum time. The simulations of the proposed method are implemented in MATLAB and finally compared with those of the Genetic Algorithm.
Keywords
Train Speed Profile
Optimal Control
Energy Consumption Optimization
Reinforcement Learning Method
Railway Transportation System
-Blanco-Castillo, M., Fernández-Rodríguez, A., Fernández-Cardador, A., & Cucala, A. P. (2022). Eco-driving in railway lines considering uncertainty associated with climatological conditions. Sustainability, 14(14), 8645.
-Ding, Y., Liu, H., Bai, Y., & Zhou, F. (2011). A Two-level Optimization Model and Algorithm for Energy-Efficient Urban Train Operation. Journal of Transportation Systems Engineering and Information Technology, 11(1), 96-101.
-Esveld, C. (2001). Modern railway track. MRT-Productions.
-Gosavi, A. (2009). Reinforcement learning: a tutorial survey and recent advances. INFORMS Journal on Computing, 21(2), 178-192.
-Howlett, P. (1996). Optimal strategies for the control of a train. Automatica, 32(4), 519-532.
-Howlett, P. (2000). The optimal control of a train. Annals of Operations Research, 98(1), 65-87.
-Howlett, PG, & Leizarowitz, A. (2001). Optimal strategies for vehicle control problems with finite control sets. Dynamics of Continuous Discrete and Impulsive Systems Series B, 8, 41-70.
-Howlett, PG, & Cheng, J. (1997). Optimal driving strategies for a train on a track with continuously varying gradient. Journal of the Australian Mathematical Society-Series B, 38(3), 388-410.
 -Hu, H., Fu, Y., & Hu, C. (2010). PSO-based optimal operation strategy of energy saving control for train. Paper presented at the Industrial Engineering and Engineering Management (IE&EM), 2010 IEEE 17Th International Conference.
-Hwang, H.S. (1998). Control strategy for optimal compromise between trip time and energy consumption in a high-speed railway. Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on, 28(6), 791-802.
-Kang, M.H. (2011). A GA-based Algorithm for Creating an Energy-Optimum Train Speed Trajectory. The Journal of International Council on Electrical Engineering, 1(2), 123-128.
-Khmelnitsky, E. (2000). On an optimal control problem of train operation. Automatic Control, IEEE Transactions on, 45(7), 1257-1266.
-Liao, J., Zhang, F., Zhang, S., Yang, G., & Gong, C. (2021). Energy-saving optimization strategy of multi-train metro timetable based on dual decision variables: A case study of Shanghai Metro line one. Journal of Rail Transport Planning & Management, 17, 100234.
-Lu, Q., & Feng, X. (2011). Optimal control strategy for energy saving in trains under the fou-aspect fixed autoblock system. Journal of Modern Transportation, 19(2), 82-87.
 -Milroy, I.P. (1980). Aspects of automatic train control. Loughborough University of Technology.  
Monajem, M.S. (2007), Designing Metro and Railway Lines, 1st Edition, Angize Publications.
-Monajem, M.S. & Bababeik, M. (2009). A study of train behavior in a track using movement simulation algorithms, 11th Railway Transportation Conference, Nov 2009, Iranian Association of Rail Transport Engineering, Tehran.
-Nolte, R. (2003). Event Evaluation of Energy Efficiency Technologies for Rolling Stock and Train Operation of Railways-Final Report. Studie im Auftrag des Internationalen Eisenbahnverbands.
-Rochard, BP, & Schmid, F. (2000). A review of methods to measure and calculate train resistances. Proceedings of the Institution of Mechanical Engineers, Part F: Journal of Rail and Rapid Transit, 214(4), 185-199.
-Sandidzadeh, M. A., Askarian, M., & Soleymaani, F. (2020). The Effect of Using the Tabu Search Algorithm on the Speed of Achieving the Optimal Train Speed Profile (in order to Reduce Energy Consumption). Journal of Transportation Research, 17(4), 31-48.
-Sheu, J.W., & Lin, W.S. (2012). Energy-Saving Automatic Train Regulation Using Dual Heuristic Programming. Vehicular Technology, IEEE Transactions on, 61(4), 1503-1514.
-Sutton, R.S., & Barto, A.G. (1998). Reinforcement learning: An introduction. Vol. 1, Cambridge Univ. Press.
 -Xu, R., Meng, J., Li, D., & Chen, X. (2023). Energy-Efficient Optimization Method of Urban Rail Train Based on Following Consistency. Energies, 16(4).
-Yang, L., Li, K., Gao, Z., & Li, X. (2011). Optimizing trains movement on a railway network. Omega.
-Yeo, C., & Koseki, T. (2002). Optimization of Running Profile of Train by Dynamic Programing. Paper presented at the National Convention of IEEE.
-Zhu, Q., Su, S., Tang, T., Liu, W., Zhang, Z., & Tian, Q. (2022). An eco-driving algorithm for trains through distributing energy: A Q-Learning approach. ISA Transactions, 122, 24-37.