TY - JOUR AU - Kővári, Bálint AU - Angyal, Balint Gergo AU - Bécsi, Tamás TI - Deep Reinforcement Learning combined with RRT for trajectory tracking of autonomous vehicles. JF - TRANSPORTATION RESEARCH PROCEDIA J2 - TRANSP RES PROCEDIA VL - 78 PY - 2024 SP - 246 EP - 253 PG - 8 SN - 2352-1465 DO - 10.1016/j.trpro.2024.02.032 UR - https://m2.mtmt.hu/api/publication/34687657 ID - 34687657 AB - Sample inefficiency is a long-standing problem in Deep Reinforcement Learning based algorithms, which shadows the potential of these techniques. So far, the primary approach for tackling this issue is prioritizing the gathered experiences. However, the strategy behind collecting the experiences received less atention, but it is also a legitimate approach for prioritizing. In this paper, the Rapidly exploring Random Trees algorithm and Deep Reinforcement Learning are combined for the trajectory tracking of autonomous vehicles to mitigate the issues regarding sample efficiency. The core of the concept is to utilize the tremendous explorational power of RRT for covering the state space via experiences for the Agent to diversify its training data buffer. The results demonstrate that this approach outperforms the classic trial-and-error-based concept according to several performance indicators. © 2024 The Authors. Published by ELSEVIER B.V. LA - English DB - MTMT ER - TY - JOUR AU - Kolat, Máté AU - Bécsi, Tamás TI - Multi-Agent Reinforcement Learning for Highway Platooning JF - ELECTRONICS (SWITZ) VL - 12 PY - 2023 IS - 24 PG - 13 SN - 2079-9292 DO - 10.3390/electronics12244963 UR - https://m2.mtmt.hu/api/publication/34430090 ID - 34430090 N1 - Export Date: 5 January 2024 Correspondence Address: Bécsi, T.; Department of Control for Transportation and Vehicle Systems, Hungary; email: becsi.tamas@kjk.bme.hu Funding details: RRF-2.3.1-21-2022-00002, TKP2021-NVA-02 Funding details: European Commission, EC Funding details: Magyar Tudományos Akadémia, MTA Funding details: Nemzeti Kutatási, Fejlesztési és Innovaciós Alap, NKFIA, BO/00233/21/6 Funding text 1: This research was supported by the European Union within the framework of the National Laboratory for Autonomous Systems (RRF-2.3.1-21-2022-00002). Project no. TKP2021-NVA-02 has been implemented with the support provided by the Ministry of Culture and Innovation of Hungary from the National Research, Development and Innovation Fund, financed under the TKP2021-NVA funding scheme. T.B. was supported by BO/00233/21/6, János Bolyai Research Scholarship of the Hungarian Academy of Sciences. AB - The advent of autonomous vehicles has opened new horizons for transportation efficiency and safety. Platooning, a strategy where vehicles travel closely together in a synchronized manner, holds promise for reducing traffic congestion, lowering fuel consumption, and enhancing overall road safety. This article explores the application of Multi-Agent Reinforcement Learning (MARL) combined with Proximal Policy Optimization (PPO) to optimize autonomous vehicle platooning. We delve into the world of MARL, which empowers vehicles to communicate and collaborate, enabling real-time decision making in complex traffic scenarios. PPO, a cutting-edge reinforcement learning algorithm, ensures stable and efficient training for platooning agents. The synergy between MARL and PPO enables the development of intelligent platooning strategies that adapt dynamically to changing traffic conditions, minimize inter-vehicle gaps, and maximize road capacity. In addition to these insights, this article introduces a cooperative approach to Multi-Agent Reinforcement Learning (MARL), leveraging Proximal Policy Optimization (PPO) to further optimize autonomous vehicle platooning. This cooperative framework enhances the adaptability and efficiency of platooning strategies, marking a significant advancement in the pursuit of intelligent and responsive autonomous vehicle systems. LA - English DB - MTMT ER - TY - JOUR AU - Kővári, Bálint AU - Pelenczei, Bálint AU - Bécsi, Tamás TI - Enhanced Experience Prioritization: A Novel Upper Confidence Bound Approach JF - IEEE ACCESS J2 - IEEE ACCESS VL - 11 PY - 2023 SP - 138488 EP - 138501 PG - 13 SN - 2169-3536 DO - 10.1109/ACCESS.2023.3339248 UR - https://m2.mtmt.hu/api/publication/34418334 ID - 34418334 N1 - Budapest University of Technology and Economics, Faculty of Transportation Engineering and Vehicle Engineering, Department of Control for Transportation and Vehicle Systems, Budapest, 1111, Hungary Institute for Computer Science and Control, Systems and Control Laboratory, Budapest, 1111, Hungary Export Date: 2 January 2024 Correspondence Address: Becsi, T.; Budapest University of Technology and Economics, Hungary; email: becsi.tamas@kjk.bme.hu LA - English DB - MTMT ER - TY - JOUR AU - Kolat, Máté AU - Tettamanti, Tamás AU - Bécsi, Tamás AU - Esztergár-Kiss, Domokos TI - On the relationship between the activity at point of interests and road traffic JF - COMMUNICATIONS IN TRANSPORTATION RESEARCH J2 - COMM TRANSPORT RES VL - 3 PY - 2023 PG - 11 SN - 2772-4247 DO - 10.1016/j.commtr.2023.100102 UR - https://m2.mtmt.hu/api/publication/34224651 ID - 34224651 N1 - Department of Control for Transportation and Vehicle Systems, Faculty of Transportation Eng. and Vehicle Eng., Budapest University of Technology and Economics, Budapest, Pest, 1111, Hungary Systems and Control Laboratory, Institute for Computer Science and Control of the Eötvös Loránd Research Network, Budapest, Pest, 1053, Hungary Department of Transport Technology and Economics, Faculty of Transportation Eng. and Vehicle Eng., Budapest University of Technology and Economics, Budapest, Pest, 1111, Hungary Export Date: 9 November 2023 Correspondence Address: Kolat, M.; Department of Control for Transportation and Vehicle Systems, Budapest, Hungary; email: mate.kolat@edu.bme.hu LA - English DB - MTMT ER - TY - CHAP AU - Lindenmaier, László AU - Czibere, Balázs AU - Aradi, Szilárd AU - Bécsi, Tamás ED - Szakál, Anikó TI - A Runtime-Efficient Multi-Object Tracking Approach for Automotive Perception Systems T2 - IEEE 17th International Symposium on Applied Computational Intelligence and Informatics SACI 2023 : Proceedings PB - IEEE Hungary Section CY - Budapest SN - 9798350321104 PY - 2023 SP - 000785 EP - 000792 PG - 8 DO - 10.1109/SACI58269.2023.10158542 UR - https://m2.mtmt.hu/api/publication/34039486 ID - 34039486 N1 - Correspondence Address: Lindenmaier, L.; Budapest University of Technology and Economics, Hungary; email: lindenmaier.laszlo@kjk.bme.hu LA - English DB - MTMT ER - TY - JOUR AU - Lindenmaier, László AU - Aradi, Szilárd AU - Bécsi, Tamás AU - Törő, Olivér AU - Gáspár, Péter TI - Object-Level Data-Driven Sensor Simulation for Automotive Environment Perception JF - IEEE Transactions on Intelligent Vehicles J2 - IEEE Transactions on Intelligent Vehicles VL - 8 PY - 2023 IS - 10 SP - 4341 EP - 4356 PG - 15 SN - 2379-8858 DO - 10.1109/TIV.2023.3287278 UR - https://m2.mtmt.hu/api/publication/34027317 ID - 34027317 N1 - Budapest University of Technology and Economics, Department of Control for Transportation and Vehicle Systems, Budapest, 1111, Hungary Institute for Computer Science and Control, Systems and Control Laboratory, Budapest, 1111, Hungary Export Date: 12 December 2023 Correspondence Address: Bécsi, T.; Budapest University of Technology and Economics, Hungary; email: becsi.tamas@kjk.bme.hu LA - English DB - MTMT ER - TY - CHAP AU - Kolat, Máté AU - Esztergár-Kiss, Domokos AU - Tettamanti, Tamás AU - Bécsi, Tamás ED - Horváth, Balázs ED - Horváth, Gábor TI - A Point of Interest (POI) adatok és a közúti forgalom közötti összefüggés / Correlation between POI and road traffic data T2 - XIII. International Conference on Transport Sciences / XIII. Nemzetközi Közlekedéstudományi Konferencia, Győr PB - Közlekedéstudományi Egyesület (KTE) CY - Győr SN - 9786156443175 PY - 2023 SP - 293 UR - https://m2.mtmt.hu/api/publication/34012750 ID - 34012750 LA - Hungarian DB - MTMT ER - TY - JOUR AU - Szőke, László AU - Aradi, Szilárd AU - Bécsi, Tamás TI - Traffic Signal Control with Successor Feature-Based Deep Reinforcement Learning Agent JF - ELECTRONICS (SWITZ) VL - 12 PY - 2023 IS - 6 PG - 14 SN - 2079-9292 DO - 10.3390/electronics12061442 UR - https://m2.mtmt.hu/api/publication/33706849 ID - 33706849 AB - In this paper, we study the problem of traffic signal control in general intersections by applying a recent reinforcement learning technique. Nowadays, traffic congestion and road usage are increasing significantly as more and more vehicles enter the same infrastructures. New solutions are needed to minimize travel times or maximize the network capacity (throughput). Recent studies embrace machine learning approaches that have the power to aid and optimize the increasing demands. However, most reinforcement learning algorithms fail to be adaptive regarding goal functions. To this end, we provide a novel successor feature-based solution to control a single intersection to optimize the traffic flow, reduce the environmental impact, and promote sustainability. Our method allows for flexibility and adaptability to changing circumstances and goals. It supports changes in preferences during inference, so the behavior of the trained agent (traffic signal controller) can be changed rapidly during the inference time. By introducing the successor features to the domain, we define the basics of successor features, the base reward functions, and the goal preferences of the traffic signal control system. As our main direction, we tackle environmental impact reduction and support prioritized vehicles’ commutes. We include an evaluation of how our method achieves a more effective operation considering the environmental impact and how adaptive it is compared to a general Deep-Q-Network solution. Aside from this, standard rule-based and adaptive signal-controlling technologies are compared to our method to show its advances. Furthermore, we perform an ablation analysis on the adaptivity of the agent and demonstrate a consistent level of performance under similar circumstances. LA - English DB - MTMT ER - TY - JOUR AU - Kolat, Máté AU - Kővári, Bálint AU - Bécsi, Tamás AU - Aradi, Szilárd TI - Multi-Agent Reinforcement Learning for Traffic Signal Control: A Cooperative Approach JF - SUSTAINABILITY J2 - SUSTAINABILITY-BASEL VL - 15 PY - 2023 IS - 4 PG - 13 SN - 2071-1050 DO - 10.3390/su15043479 UR - https://m2.mtmt.hu/api/publication/33644779 ID - 33644779 AB - The rapid growth of urbanization and the constant demand for mobility have put a great strain on transportation systems in cities. One of the major challenges in these areas is traffic congestion, particularly at signalized intersections. This problem not only leads to longer travel times for commuters, but also results in a significant increase in local and global emissions. The fixed cycle of traffic lights at these intersections is one of the primary reasons for this issue. To address these challenges, applying reinforcement learning to coordinating traffic light controllers has become a highly researched topic in the field of transportation engineering. This paper focuses on the traffic signal control problem, proposing a solution using a multi-agent deep Q-learning algorithm. This study introduces a novel rewarding concept in the multi-agent environment, as the reward schemes have yet to evolve in the following years with the advancement of techniques. The goal of this study is to manage traffic networks in a more efficient manner, taking into account both sustainability and classic measures. The results of this study indicate that the proposed approach can bring about significant improvements in transportation systems. For instance, the proposed approach can reduce fuel consumption by 11% and average travel time by 13%. The results of this study demonstrate the potential of reinforcement learning in improving the coordination of traffic light controllers and reducing the negative impacts of traffic congestion in urban areas. The implementation of this proposed solution could contribute to a more sustainable and efficient transportation system in the future. LA - English DB - MTMT ER - TY - JOUR AU - Törő, Olivér AU - Bécsi, Tamás TI - Analytic solution of the exact Daum–Huang flow equation for particle filters JF - INFORMATION FUSION J2 - INFORM FUSION VL - 92 PY - 2023 SP - 247 EP - 255 PG - 9 SN - 1566-2535 DO - 10.1016/j.inffus.2022.11.027 UR - https://m2.mtmt.hu/api/publication/33313067 ID - 33313067 N1 - Funding Agency and Grant Number: European Union [RRF-2.3.1-21-2022-00002]; Ministry of Innovation and Technology of Hungary from the National Research, De-velopment and Innovation Fund Funding text: The research was supported by the European Union within the framework of the National Laboratory for Autonomous Systems. (RRF-2.3.1-21-2022-00002) The research reported in this paper is part of project no. BME-NVA-02, implemented with the support provided by the Ministry of Innovation and Technology of Hungary from the National Research, De-velopment and Innovation Fund, financed under the TKP2021 funding scheme. AB - State estimation for nonlinear systems, especially in high dimensions, is a generally intractable problem, despite the ever-increasing computing power. Efficient algorithms usually apply a finite-dimensional model for approximating the probability density of the state vector or treat the estimation problem numerically. In 2007 Daum and Huang introduced a novel particle filter approach that uses a homotopy-induced particle flow for the Bayesian update step. Multiple types of particle flows were derived since with different properties. The exact flow considered in this work is a first-order linear ordinary time-varying inhomogeneous differential equation for the particle motion. An analytic solution in the interval [0,1] is derived for the scalar measurement case, which enables significantly faster computation of the Bayesian update step for particle filters. LA - English DB - MTMT ER -