TNC Performance Evaluation of Multi-hop WirelessHART Network on a Real-life Testbed

Advances in WirelessHART standard in industrial control systems have led to performance evaluation and security analysis in both real-world testbeds as well as in controlled lab environments. We have conducted months-long experiments with WirelessHART network in a multi-hop setting in our laboratory. Latency, stability, and reliability have been used as metrics to measure performance of individual links and the overall network for five hops and seven hops. We have deliberately deviated from following the best practices in designing the topology to study network performance under strained conditions. In addition to using metrics as defined in WirelessHART literature, we have also studied network stability over multiple hops with single paths. Our findings show that having at least one low stability link can have an impact on multihop stability, while still maintaining a very high overall network reliability of 99.98% or higher. Details of the experiment along with results and lessons learned are presented in the paper.


Introduction
WirelessHART is a wireless sensor networking standard designed for industrial control systems. The protocol is standardized by the HART Communication Foundation [2], and provides an alternative solution for Bluetooth and ZigBee, in noisy environments [1,5]. WirelessHART was designed to create a wireless protocol solution for the already existing HART protocol [9], which is the most common protocol for field devices. The standard is based on IEEE 802.15.4 [10]. It utilizes a time synchronized, self-organizing, and self-healing mesh architecture, forming full mesh network topologies. This kind of networks contains a large number of nodes (henceforth referred as motes), which normally uses batteries as a power source a Real-life Testbed, Transactions on Networks and Communications, Volume 6 No. 1, February (2018) and organize themselves into a multi-hop wireless network. The network uses medium access control (MAC) technique [2] for successful operation. MAC protocol avoids collisions of data transmitted by assigning a time for each mote on the network.
WirelessHART is a global IEC-approved standard (IEC 62591) [3] that specifies an interoperable selforganizing mesh technology in which field devices form a wireless network that dynamically mitigate obstacles in the process environment. The WirelessHART field networks (WFN) communicate data back to host systems securely and reliably, and can be used for both control and monitoring applications. The similarities between traditional HART and WirelessHART allow end users to leverage training of existing process organizations when adopting WirelessHART. In addition, the reduced installed cost of WirelessHART extends the benefits of automation to end user applications that previously were out of reach since they could not justify the costs associated with typical wired capital projects. The opportunity for long-term benefit makes it compelling for users to expand process manufacturing project planning to evaluate the impact of WirelessHART on maintenance, safety, environment, and reliability. Additionally, by removing the physical constraints of wiring and power as well as reducing weight and space, wireless networks increase flexibility in project execution, providing solutions that can mitigate risk and improve project schedules. Its standard is a secure networking technology that operates in the 2.4GHZ ISM radio band and utilizes IEE 802.15.4 [4]. WirelessHART protocol communications are precisely scheduled using an approach referred to as Time Division Multiple Access (TDMA). This scheduling is performed by the network manager, a device that is the central component of the mesh network architecture. Most of the research conducted in Wireless HART networks focused on scheduling at the MAC layer, and designing frames for communications between motes and between motes and manager. Not much research was conducted to study performance of such networks over multiple hops in a real-world testbed. In [8] the authors performed a laboratory experiment on a wide scale deployment of wireless HART network over a period of 120 hours, and monitored packet loss, latency, and reliability. Their deployed network achieved a near 100% reliability, with very low packet loss and latency. However, as the experiment was conducted for only 120 hours, it fails to provide a long-term performance evaluation of the network as a whole and the stability of links in particular.
In this paper, we have presented results from a months-long study of wireless HART network deployed in our laboratory. The study was mainly focused on measuring performance of links over multiple hops and also the network as a whole with a network configuration that deviates significantly from the standard industry practices. Detailed experimental setup and results are discussed in the subsequent sections.

Method and Experimental Setup
The experiment was carried out in two phases. In the first phase, a five-hop topology was deployed in the Integrated Science and Engineering Laboratory Facility (ISELF) building on our campus, as shown in Fig 1 below. Nine motes were used in the experiment, these motes are from Linear Technology [6]. The topology was designed and deployed in a way that the farthest mote has a five-hop path to the network manager. We emphasize that we have deliberately deviated from following the best topological practices followed in industry to study the network performance under strained conditions, and implemented only single paths in the configuration. The topology spans across two floors in the ISELF building, with approximately 450 square meters per floor. The experiment was conducted for two-and-half months. Following five-hop routes were formed: In the second phase, a seven-hop topology was set up in the same building, as shown in Fig 2 below. Eight motes were used, including a mote designed by the Electrical Engineering students. This mote is a SmartMote system which integrates WirelessHART and embedded components to collect measurements from atmospheric sensors [7]. The system can join or exit an existing system without additional programming device. The SmartMote system also has solar powered batteries and is designed for easy configuration by users. It measures luminosity, temperature and humidity. The topology was designed and deployed in such a way that the farthest mote has a seven-hop path to the network manager. Once again, we deviated from topological best practices on purpose. The experiment was conducted over a period of one month.

Results and Analysis
In the following sections, results from our five-hop and seven-hop experiments were discussed in details.

Five-hop experiment
Nine motes and a network manager were used for this experiment. Three parameters, as defined in the WirelessHART literature, were used to measure the network performance: latency (in milliseconds), stability (in percentage) and reliability (in percentage). In addition, we studied the multihop stability of the five-hop path based on individual link stability values. Due to the distance and obstacles among the motes and the network manager, latency, stability and reliability varied in each link. Average values were recorded for all three measurements throughout the network. Fig 3 shows a screenshot from the Application Programming Interface, with all motes being operational, the first device shown is the network manager.

Latency
Latency is the average time required for a data packet to travel from the originating mote to the manager. Latency also varies in each link across the network, and the value represents average network latency. The network manager calculates latency for each packet by subtracting the time when the packet was received from the packet timestamp in the network layer header, which indicates when the packet was generated or accepted by the mote.
The formula to calculate latency is:

Figure. 4. Latency values
Latency increases with distance between motes and the network manager. Latency can be decreased by decreasing the number of hops or by decreasing the distance between the network manager and the motes. Table 1 below summarizes average latencies observed for each hop for both five-hop paths.

Stability
Stability is the average percentage of packets that were acknowledged by the MAC layer recipient. Stability is calculated in different places in the network. It is calculated between motes (link to link), between mote and network manager, and the average stability for the whole network. The formula to calculate stability between motes (link to link) or between a mote and the network manager is: Where, Sij is stability between mote i and mote j (link to link)

Tspk is Total packets sent
Tackp is Total acknowledged packets Formula to calculate average stability is: Where, Savg is average stability for the whole network Sij is stabilities of each hops (link) in the network n is the number of hops (links) If a mote sends a packet to another mote, but fails to receive an acknowledgment from the destination, the sender mote assumes that the packet is lost. Fig 5 is

Figure. 5. Stability values
The first two rows show the stability between the network manager and two motes that are directly connected to the manager which is labeled as 1; the stability values are respectively 73.56% and 50.98%. The rest of the lines show stability between motes. Stability varies from link to link, the variation is attributed to the locations of the motes and obstacles between them. Looking at the last line in Fig 5  above, it can be seen that the stability value between motes 13 and 17 is only 38.06%, which is attributed to the fact that the two motes are placed in different floors with mote 2 between them. However, motes 13 and 17 were occasionally able to connect with each other and transfer packets. Looking at the figure again, stability values between motes 2 and 13 and between 2 and 17 are 74.57% and 92.4% respectively, which shows that stability is inversely proportional to the distance between motes. Tables 2 and 3 below summarize stability values for each hop-path for both routes. Table 4 summarizes the average stability values for both routes over each-hop path.    T r a n s a c t i o n s o n N e t w o r k s a n d C o m m u n i c a t i o n s ; V o l u m e 6 , N o . 1 , F e b r u a r y 2 0 1 8   C o p y r i g h t © S o c i e t y f o r S c i e n c e a n d E d u c a t i o  Where, R is reliability of the network

Tpkg is Total packets generated
Upkr is Unique Packets received, offsetting duplicate packets received Figure 6 below shows reliability value for one snapshot of time. As can be seen from Fig 6, reliability is 99.98% which means 99.98% of the packets generated have reached their destination irrespective of number of retransmissions. Here, the network manager computes reliability only for its one-hop motes.

Multihop Stability
In our experiments we have computed multihop stability based on the probability of link failure, which in turn is based on individual link stability, as below: Where, = Multihop stability between motes s and d = Stability value between motes i and j We computed multihop stability values at different times of days during the entire period of the experiment. Table 5 below shows these values for our five-hop experiment for both routes 1 and 2.

Seven-hop experiment
Eight motes and a network manager were used for this experiment. As in the previous five-hop experiment, three parameters from the literature were used to measure the network performance: latency, stability, and reliability. We also studied the multihop stability of the path based on individual link stability values. Topological design was a deviation from industry best practices to study performance under strained condition. Due to the distance and obstacles among the motes and the network manager, latency, stability and reliability varied in each link. Average values were recorded for all three measurements throughout the network.

Latency
The average latency calculated by equations (1) and (2) is shown in Table 6.  These findings show that communication over more than five-hops in WirelessHART standard is feasible and practical depending on the design of the topology.

Stability
The same equations (3) and (4) are used to compute stability values.
A snapshot of stability values for the experiment is shown in Fig 8 below. The first two rows show stability of the network manager with mote 2 and 5, which are directly connected to the network manager with stability values of 39.01% and 81.56% respectively. Stability for the link between mote 2 and network manager is less because of the mote's location outside the room. Some motes in this network have more than two neighbors, but only have good stability with one of them. Because WireleesHART supports graph routing, the motes choose the best route for the data to be transferred.   Again, comparing hop-by-hop stability values for both experiments, it was observed that seven-hop network has better average stability than its five-hop counterpart, which confirms the previous findings about the feasibility and practicality of more than five-hop communications in Wireless HART.

Reliability:
Same formula as in equation (5) is used to compute reliability.

Figure. 9: Reliability Values
Reliability for this network is 99.99%. Even though latency has increased for this network compared to the five-hop network, it achieved better stability values. It takes longer for the data being transmitted to reach its destination but data loss was less.

Multihop Stability
We computed multihop stability from individual link stability values at discrete time slots over the entire lifespan of the experiment using (6).

Conclusion
We have conducted extensive experiments with multi-hop communications using WirelessHART in laboratory settings. Up to eight sensor motes were used for the experiments, which ran for months. Topologies were designed to provide five-hop and seven-hop communication paths designed to be run in two phases respectively. We have deliberately deviated from following the industry best practices in designing the topology to study network performance under strained conditions, which consist of single paths between motes. Network performance was measured using latency, stability, and reliability as defined in the WirelessHART literature. In addition, we have also studied network stability over multiple hops. The seven-hop network proved to have better results in terms of latency and stability, partly because of less number of motes used and the topology design. This shows the feasibility and practicality of multi-hop communications of Wireless HART beyond five-hop paths, given an efficient topology design.
We have found the reliability values of the five-hop and seven-hop networks in our chosen network configurations are still very close to 100%, 99.98% and 99.99%, respectively. These reliability values are based on the formula in the literature as shown in equation (5), which computes reliability without taking into account the effect of retransmissions. If packets need to be retransmitted, and eventually all retransmitted packets reach the network manager, reliability is computed as 100%. We have also studied multihop stability of the five-hop and seven-hop paths based on individual link stability values. As our topology design deviated from industry best practices, we have found that having at least one low stability link can have an impact on multihop stability, while still maintaining a very high overall network reliability of 99.98% or higher.