Time-Sensitive Network Simulation for In-Vehicle Ethernet Using SARSA Algorithm
1. Introduction
In recent years, there has been a rapid development of intelligent and connected vehicles, leading to a new trend in the automotive engineering field. An increasing number of electronic control systems, such as antilock braking systems (ABSs), advanced driver assistance systems (ADASs), and in-vehicle infotainment systems (IVIs) [1], have been implemented in automobiles. Although these electronic control systems enhance the handling, safety, and comfort of vehicles, they also result in the installation of a greater number of electronic control units (ECUs) in automobiles [2]. Therefore, smart connected vehicles require high bandwidth from the in-vehicle network, while also necessitating a network that is reliable enough to maintain low latency and jitter [3]. The traditional in-vehicle network has gradually become inadequate in meeting the diverse needs of products, thus emphasizing the need to expand and standardize the protocols for in-vehicle networks based on Ethernet. With these requirements in mind, it is necessary to conduct research and apply advances in in-vehicle Ethernet.
For in-vehicle Ethernet, the IEEE802.1 Working Group has been working on the development of Ethernet Audio Video Bridging (AVB) technology [4,5,6,7,8,9,10,11]. AVB technology supports various features such as clock synchronization, bandwidth reservation, low-latency traffic specification, and more. The AVB protocol cluster has been extended with the introduction of TSN technology, which has attracted the interest of many researchers. TSN builds upon the foundation of AVB technology and incorporates new mechanisms and enhancements in IEEE 802.1 bridges and end stations. This ensures a low-latency, high-reliability, bandwidth-isolated, and zero-blocking-loss network, with deterministic transmission time for Ethernet Local Area Networks (LANs). TSNs offer an enhanced functional implementation of traditional Ethernet in specific application scenarios such as automotive. Additionally, TSNs enable a single physical network for applications with different communication requirements, thus providing multiple network solutions [12]. Therefore, with its distinct benefits, TSN is increasingly utilized in various fields such as autonomous driving, intelligent cockpit, vehicle-to-circuit coordination, and more [13,14,15,16].
Regarding TSN technology, there have been numerous studies conducted both domestically and internationally. Scholars have focused on researching and discussing TSN’s practical application scenarios and trends [17]. Regarding delay analysis, Jahanzaib Imtiaz examined the delay of AVB in industrial real-time communication using the Priority Queuing (PQ) and Class Based Queuing (CBQ) methods [18]. Dorin Maxim studied issues related to upper bounds on the delay in AVB data streams and the transmission delay of different types of CDT time slots [19]. Sivakumar Thangamuthu conducted an analysis and comparison of the worst-case end-to-end delay and the impact on AVB traffic in TSN networks for three shapers: Burst Limiting Shaper (BLS), Time Aware Shaper (TAS), and Peristaltic Shaper (PS) [20]. Daniel Thiele thoroughly examined the worst-case delay in TSN networks using the TAS and PS mechanisms. This analysis encompasses the impact of various service types as well as the impact within the same services [21]. Luxi Zhao and Paul Pop proposed a network calculus-based approach to calculate the worst delay value for AVB data streams in TSN networks. They conducted an analysis and comparison of the upper bounds of delay in two different cases: with and without the frame preemption mechanism [22]. Ehsan Mohammadpour analyzed the worst end-to-end delay for reservation class traffic in TSN networks, focusing on the CBS and asynchronous traffic shaping mechanisms [23]. Yuefei Wang proposed a hierarchical scheduling mechanism and performed an analysis of the worst end-to-end delay for reservation class traffic. This analysis takes into account the influence of FIFO (First In First Out), CBS (Credit-Based Shaper), and TAS mechanisms [24]. Hao-Liang Xu put forward a frame preemption delay characterization model utilizing a hybrid probability model. Additionally, the impact of delay influencing factors on the component weights of the hybrid probability model is analyzed through simulation [25].
TSN guarantees real-time control of data traffic based on AVB, but it can impact the transmission delay of reservation class data within the network. Currently, there is ample literature on the time delay of reservation class traffic in TSNs, but much of it focuses on extreme cases, considering all possible factors that may disrupt reservation class traffic. In vehicular TSN networks, on the other hand, due to their unpredictability, it is difficult to accurately model or analyze the delay of reserved class data under extreme conditions. Therefore, it is essential to account for traffic scheduling randomness, develop a suitable set of delay analysis models, and conduct further research on the transmission delay of reservation class data.
This paper aims to analyze the data flow scheduling mechanism and algorithms within TSN networks. Due to non-deterministic scheduling challenges, this paper proposes an intelligent SARSA reinforcement learning algorithm for delay analysis of reservation class data in vehicular TSN networks, which will provide a more accurate simulation and analysis of TSN network.
4. Simulation and Result Analysis
The data flow at the gateway of an in-vehicle TSN network usually belongs to the periodic traffic that helps to calculate the delay value. The delay values of different types of data frames are calculated so as to verify the reasonableness of the model proposed in this paper.
Referring to previous studies [37], the total link bandwidth is set to 100 Mbit/s, each gating period is set to 500 μs, and the non-CDT transmission bandwidth is set to 90 Mbit/s. where the CDT time slot length and the non-CDT time slot length are set to 150 μs and 350 μs, respectively. the lengths of the data frames of Class A, B, and BE are all set to 400 bytes. The Class A data frame period and Class B data frame period are set to 125 μs and 250 μs, respectively, and the reserved bandwidth of the data stream is set according to the actual situation. The reserved bandwidths for the Class A and Class B data streams are 400 Mbit/s and 20 Mbit/s, respectively.
The corresponding program is run in MATLAB-R2020b and the results are shown in Figure 5, Figure 6 and Figure 7. Firstly, we compare and analyze the mean values of the delay of the data frame group of the SR class data frames under AVB and TSN networks, respectively, so that the parameters of the SR class data frames remain unchanged. Figure 5 shows the delay averages of Class A data frames in both TSN and AVB network environments when the BE load is 5 Mbit/s. It can be seen that although the protection window in TSN guarantees the transmission delay of CDT data frames, it has a great impact on the queue forwarding of SR class data frames, which is in line with the theoretical situation.
Secondly, the data delay of the two SR class data frames is compared and analyzed in the TSN network. Figure 6 shows the comparison of the average delay of traffic for Class A data frames and Class B data frames when the BE class load is 10 Mbit/s. The results are 110.93 μs and 181.35 μs, respectively, and also the discrete situation of Class A data frames and Class B data frames can be seen from the figure, i.e., the Class A mean square deviation is smaller than that of Class B. The results show that the transmission real time of Class A data frames is prioritized higher in TSN networks, i.e., Class A traffic is prioritized higher than Class B, which is in line with the previous data type analysis.
Next, the BE data load is analyzed to see if there is any significant effect on the delay of SR class data. As shown in Figure 7, Class A data frames are taken as the object of study, and the delay values of Class A data frames under different BE data frame loads are simulated and analyzed. It can be seen that the transmission delay of Class A data frames increases significantly with increasing load.
In summary, the TSN data forwarding delay analysis model developed in this paper matches the theoretical situation, and the fitting performance for the above scenarios is also better, thus verifying the reasonableness of the model.
5. Test Validation
The simulation testbed consists of RAD_Galaxy, gateway, surround view system and PC as shown in Figure 8. The video data are captured by the four cameras of the surround view system, and the data are transmitted to the gateway at the same time. RAD_Galaxy acts as an intermediary device connecting the gateway to the PC and the monitor, and also acts as a sending node for the audio data and control data. This allows the data messages at the gateway to be monitored at the PC, thus realizing a semi-physical simulation; the test bench is shown in Figure 9.
The experiment concludes by comparing the theoretical delay, obtained through simulation in MATLAB, with the actual measured delay values on the experimental bench. This comparison aims to verify the reasonableness of the TSN delay analysis model developed in this paper.
Due to the large variance of the individual time delay data of the measured samples, the average of the time delays was used as a comparison reference, with 40 data frames per subgroup. The results of the time delay comparison are shown in Figure 10 and Figure 11. The results show that the delay values obtained from both model predictions and experimental measurements increase with the increase in Class BE data load, and the deviation values of both have a small increase. The theoretical and actual measured delay values for Class A and Class B data frames are shown in Table 2 and Table 3. The deviation of both the theoretical and measured values of the reserved class data transmission delay is less than 5% for different BE data loads, with a maximum deviation of 4.64%.
Furthermore, in order to validate the feasibility of the method, Table 4 is included, which presents a comparison of delay results using the Q-reinforcement learning approach. Through this comparative test, it becomes apparent that the delay analysis model put forth in this paper successfully captures the delay characteristics of reservation classes in TSN networks.
