In an attempt to find out if the local telecoms provider (Verizon) will ever start carrying the local sports teams (Lakers, USC, UCLA) on TV, I stumbled upon on a Verizon discussion site. There was surprisingly little dialogue about why a mile from the UCLA campus I can watch every Purdue football game on FIOS but not UCLA, but instead the site was filled with chatter about broadband speeds.
I pawed through the copious vitriol, mostly along the lines of “Verizon is a rabid pack of lying, cheating, thieves.” (I’m paraphrasing a bit, but I think that accurately summarizes the sentiment.) The main thrust of the argument was that Verizon provides a bandwidth testing application (http://my.verizon.com/micro/speedtest/broadband/) that apparently gives higher results than third party bandwidth test tools such as Speedtest.net, evidence that Verizon is not only shorting people of bandwidth, but doctoring the test results in a crude attempt to hide it.
This is clearly a topic near and dear to our hearts here at Apposite (or more importantly, to our wallets), so I decided to run my own tests to see what had this excitable crowd in such a lather over bandwidth testing.
I ran Speedtest.net on my 25/25 FIOS line, and it claimed that I was getting 30.7 Mbps inbound and 25.3 Mbps outboard. Certainly nothing to complain about there. Then I ran the Verizon test and it reported that I was getting 30.6 Mbps inbound but only 12.9 Mbps outbound. Curious.
Most people assume that bandwidth testing is easy – just check the line and see how much bandwidth it has. But it’s not that simple. Consumer “bandwidth testers” (most are OEM-badged versions of a single system from Ookla) run a file transfer between the user’s PC and a server somewhere on the Internet, and assume that the speed of the transfer is limited by the bandwidth of the user’s connection to the carrier.
While this can provide a reasonable estimate of available bandwidth, it’s also subject to a number of variables that can render the results meaningless. For example, if your son is playing online video games, your daughter is watching Netflix, your wife is chatting on Skype, and your PC is downloading Java security patch 37, the tool will measure not how much bandwidth you have but how much bandwidth you have left. Second, the client and server need to move data quickly, and anything else running on either machine, like the installation of Java security patch 37 or even an Excel autosave, can cause a hiccup that distorts the results. Third, the transfer is not just testing the access link between the home router and the carrier, but the entire path between the user’s PC and the application server, and congestion anywhere along that path will lower the transfer speed. Lastly, and most difficult for people to understand, the file transfer is running over TCP, and TCP throughput is sensitive to latency caused by the distance between the two end points. The test will therefore report different results for “bandwidth” depending on how far away the server is located.
I re-ran the Verizon and Speedtest applications three times each, but the results were essentially identical each time. Using the traffic monitor feature of Apposite’s Linktropy Mini network simulator, I verified that the actual throughput matched the reported values.
So what accounts for this discrepancy in reported bandwidth between the two test tools? Since the results were consistent across multiple tests, it is unlikely a problem with congestion on the network or availability of the client or server which would cause erratic or inconsistent results. Nor are the numbers being doctored in a bandwidth conspiracy since the results match the actual data transfer rates.
To figure out what was really going on, I fired up Wireshark and did a packet trace on both connections. The first thing that became obvious was that the applications were completely different, with Speedtest using Flash and Verizon using Java. But the transfer details were also curious. For the outbound connection, Speedtest actually ran two transfers simultaneously, each with an 80 KB window. Since the application can’t control the TCP stack on the client PC the way it can on the server, this hack helps overcome some of the effects of latency and packet loss. The Verizon application ran a single transfer with a window size set at 16 KB. Consequently, throughput was limited not by the bandwidth of the link, but by the TCP settings of the application.
So is Verizon a pack of lying cheats? From this evidence, it appears that they’re merely incompetent at writing network applications. It looks like they wrote an application that worked fine for DSL users at 3 Mbps and under, and didn’t bother to verify that it worked at FIOS speeds. But though I can’t vouch for other users, I certainly have no grounds for compliance about how much bandwidth I’m getting from Verizon FIOS. Now, if they would only start carrying those UCLA games…
