We ran some tests with a few freeware WAN emulators and compared them to a professional appliance. Show your boss this guide to justify your need for a pro appliance. The results were insane!
EXECUTIVE SUMMARY
Apposite Technologies challenged a junior engineer to run a series of performance tests using three open-source WAN emulation tools; NetEm, WANem, and DummyNet, with the purpose of comparing the user experience and test results to those of a professional WAN emulation appliance.
Testing with freeware WAN emulators cannot offer the granularity, control, and hardware compatibility needed to produce determinable and repeatable results. The average PC wasn’t engineered to provide sub-millisecond accuracy at the application level. When running a testing exercise in an industry where milliseconds make a world of difference, timing is the entire point. Any results you can acquire from a freeware WAN emulator aren’t going to be reliable enough to deliver conclusive results.
Open source WAN emulation tools are not enterprise solutions. They’re incapable of testing on server hardware with up to 10Gbps speeds and are further limited by the hardware on which they’re running. Network interface controllers offer a wide breadth when it comes to quality. Lower-end models may not offer the degree of buffer control required to handle proper packet control, while higher-end models attempt to circumvent potential performance issues by forwarding packets to the CPU in groups, which further skews timing.
Even if they work alright under certain circumstances, they are inaccurate under other circumstances and there’s no telling whether the results are valid. The results simply cannot be trusted.
KEY FINDINGS WITH WANEM
WANem is based on Linux v2.6 from 2011, so the set up was difficult. It took two weeks to get tests running using WANem, versus minutes with the Apposite WAN emulator. Some rules in the command line interface didn’t make sense. Overall, the WANem test results were unreliable.
An iptables network address translation (NAT) method was used to test the features and capabilities of WANem. The first test was run to add 20ms of latency. This feature only provided partial functionality and applied an additional 25% of latency and double that amount on certain test cases. Additionally, the application can’t apply background traffic and high bandwidths of up to 1Gbps are impossible to handle.
The options for reporting and statistical views only communicate the current settings, bytes sent, packets sent, and packets dropped for each interface on the systems. These values are presented as totals since the last time the refresh button was pressed and doesn’t provide reporting on a more specific measurement of time. Packet loss is not reported correctly and packet corruption or duplication can’t be tracked at all.
KEY FINDINGS WITH NETEM
NetEm is a command line interface (CLI) based utility. There is no GUI and the CLI commands are dozens of characters in length. The packet reordering setting was only partially functional and works by assigning a fixed delay to all packets except a specified percentage, which are let through immediately with no added latency. However, via ping, no reordering appears to occur unless the latency is set at a minimum of 400ms.
Like WANem, NetEm can’t handle high bandwidth speeds. The results also yield different numbers when run on multiple machines conducting the same methods of data transfer, leading to the conclusion that testing cannot be accurately replicated.
KEY FINDINGS WITH DUMMYNET
DummyNet is not compatible with modern Linux distributions and hasn’t been updated since 2015. It is only capable of emulating basic network impairments such as bandwidth throttling and packet loss. Attempting to emulate latency resulted in massive amounts of jitter which fluctuated unpredictably between tests. When setting latency of 20ms, ping results indicated results anywhere between 17-21ms, which is far too wide a margin of error to be considered reliable or useful.
Because it had trouble with performance at low levels, i.e; a basic ping test, higher throughput wasn’t tested. It has no support for packet corruption or duplication.
LINKTROPY 8510 WAN EMULATOR
The Linktropy 8510 is a purpose-built WAN emulation appliance offering enterprise level testing precision. With 8 GigE RJ45 emulation interfaces, the performance features include 8 Gbps maximum throughput capabilities and a maximum packet forwarding rate of 12 million PPS. The Linktropy 8510 can emulate common network impairments such as bandwidth throttling, latency, packet loss, and bit errors with ease. The distribution of latency can also be specified. It’s rack mountable and can be installed as a bridge or router, making it a perfect fit for both the lab and data center.
Setting up the device is as easy as it gets. Simply place the device between two LAN segments, connect the testing PC to the management port, and configure the parameters of your test through a friendly web-based GUI.
Once the device is configured with the appropriate settings, emulation begins by toggling the power button from off to on. Traffic running between the two LAN segments then passes through the device which applies the impairments to the frames before forwarding them to the adjacent port.
It precisely emulates bandwidth throttling, latency, packet loss, and packet corruption. All features function in the manner intended and produce results equal to the specified parameters.
It’s easy to use and requires no training or development expertise to operate. Real-world testing can start in minutes and the Linktropy 8510’s real-time operating system can handle anything you throw its way.
CONCLUSION
The performance of freeware WAN emulation tools is not only limited and dependent on hardware and software, but also the overall composition of the systems on which they reside. There is no standardized set up as each system is different. As such, obtaining repeatable and deterministic results is impossible.
In conclusion, a purpose-built WAN emulation appliance with a real-time operating system is the only way to go. The hidden costs associated with the time and labor required to properly configure and deploy the freeware tools make them practically useless. Freeware isn’t free. Save your time and money and get the job done right the first time.
