Round 1 - Wireless power - Intro

So the one aspect of The Hubs that seems to net the most complaints in the wireless. And wireless is where most of my networking expertise lies, so that's what I'll test first and foremost.

In round 1, I'll look at the wireless output ("signal") of the comparison group. In this test, I only measure the wireless signal strength and distribution of the equipment in question, and at this stage make absolutely no representation as to the quality, reliability, or performance of that signal. In addition, only output power is measured, as there is no reliable method to measure input signal levels consistently across all devices.

Since I don't have an integrating sphere, and haven't been able to locate a sufficiently cheap nearby wireless test lab to use, it'll have to be done on the cheap with what I have to hand - i.e. a hundred or so £400 Cisco wireless access points and a few £20k controllers.

Aims:

To measure the wireless output power of the devices in question, as well as beam pattern (mainly because one of the devices has its internal antennae partly occluded by the main PCB and I want to know if this makes any difference).

Method:

Each device will be placed, in turn, on the same marked spot in a building with a very high density of Cisco managed wireless access points. Each device will be placed on said spot in the same orientation, and then powered up.

Our managed APs constantly scan the radio spectrum looking for other nearby wireless networks, both to detect and avoid interference as well as to detect rogues. The wireless output (primarily beacons) will be measured simultaneously by the hundred or so existing wireless APs surrounding the test point on all sides, vertically and horizontally. The device will then be rotated through 360 degrees horizontally and vertically and signal levels received by each measuring AP at every angle recorded. This'll form the basis of our signal level comparisons and beam pattern calculations.

Sure there's a lot of variables, but we eliminate most of them by averaging readings over numerous receivers and measuring the output at every angle, from every angle.

All devices will be set to channel 6 on the 2.4Ghz band (pretty much in the middle of their operating range) and 44 on the 5Ghz band. While the 5Ghz band is wide, containing 24 overlapping channels in the UK, the Superhub can only use 4 of them, which means the total wireless capacity available at 5Ghz is no different to at 2.4Ghz for Superhub users.

Because of the nature of the measurements, these numbers can't really be used as an absolute reference point, but can be used to accurately compare the relative performance of each test subject.

And again, only the signal strength is measured at this stage, so this is largely a synthetic benchmark to get an idea of what we're dealing with. No attempt is made to connect to or use the wireless networks in question. How each device performs in actual use will become clearer in later - i.e. "real-world" - tests.

Battle of the Hubs - The prequel

After my last attempt at inspiring level headed debate (half-hearted as it was) resulted in yet another abject slanging match of "Superhub sucks" "Superhub is fine" "No it sucks" "No you suck" "You suck more" "Your momma" "STFU NOOB" etc., I decided it might be better to post results... elsewhere. That elsewhere is, as you guessed, here.

Battle of the Hubs is as the title might suggest, a comparison of the two biggest UK ISPs' consumer home gateway equipment (commonly known as wireless routers/modems, but referred to by the ISPs as hubs). These are supplied by default to all new customers, including on their top end "Fibre optic" packages*. These will be compared to a number of low, mid and high-end third-party aftermarket home gateways. It should be noted that while both ISPs supply their respective CPE equipment as the only officially supported equipment on their service, only one of the two actively prevents users from using any alternative CPE (although a "modem mode" is offered bypassing many of the "router" functions).

* Not really fibre optic. Actual services are HFC FTTN or VDSL2 FTTC.

Both these CPEs share a similar feature set:

• Both are custom-built in both hardware and software, made to order for each ISP
• Both have built-in modems for their respective connection technologies (Cable and DSL)
• Both are provided on all of each ISPs packages without variation, including "fibre optic" packages, all the way up to 100mb.
  
• Both support "Up to 300mbps" (2x2 MIMO + HT40) wireless N, and come with two built-in, non user-changeable wireless antenna
• Both ISPs make extravagant claims about the wireless capabilities ("The UK's most reliable wireless", "Unbeatable wireless" "Best wireless performance available" "best ever wireless signal" etc.)
• Both come with 4 LAN ports, at least one of which is gigabit.
• Both are claimed to be highly energy efficient
• Both have identical external controls (power switch, WPS, pinhole reset)
• Both stand upright rather than the traditional flat-book style, presumably for better wireless dissipation

And the most important thing of all, both have received numerous complaints about their performance and reliability, some of which have been admitted.

So where do I come in? I'm a customer of both providers top packages, I'm in possession of both CPEs, a fair bit of information technology and networking experience, a large amount of testing equipment, multidisciplinary scientific training, and a taser. For the really misbehaving routers.

So, for those who haven't guessed yet, the two "hubs" I'm talking about are the BT Home Hub 3 and the Virgin Media Superhub. And I'll be pitting them head to head with each other and a dozen other wireless routers to see exactly just how they perform. To the best of my ability, I'll be testing the wired, wireless, routing & NAT performance, power consumption, and reliability of the above devices. Though without approval from either ISP, I'll be limited in some ways.

The comparison devices to be used will be a selection of what I have to hand, which for now are a WRT54G, WPN824, DIR-615, DIR-825, WNDR3700, WR2543ND, and possibly some HP 530wls. Plus a thousand or so Cisco enterprise WAPs we have at work. Client devices for "real-world" performance tests include laptops equipped with Intel 5100, 6205, and 6300 cards, Atheros AR92xx and AR93xx cards, Realtek 8188CE and 8187B, and mystery Broadcom card. Additionally, mobile handsets including a HTC Desire, Samsung GS2 and Nokia Eseries will be used. Course, I won't be testing every possible combination of everything, I'd probably grow old trying.

Oh, and in addition to a "Standard" stock Superhub, I'll be comparing a modified Superhub equipped with two external RP-SMA connecters for upgradeable antennae. I'll only be using cheap, basic omni rubber-duck antennae rather than anything big or special.

So without further ado, let the battle begin.

Wireless - preliminary tests

Wireless - preliminary tests. Ignore these, they're very rudimentary.

location 1 - park @ 280m
bthh 8.53 2.42
n111 26.82 7.44
n11 0.19 0.59
sh1 2.90 0.64
sh2 0.22 0.15


location 2 - kitchen
bthh 6.08 3.46 15.05 5.63
sh1 3.91 0.70 17.00 5.48
sh2 16.52 5.15 27.17 7.67
n11 26.32 8.07 21.80 7.98
n55 37.37 7.91 37.62 7.84
n111 3.56 0.33 11.41 3.65


location 3 - bathroom
bthh 32.37 8.19 37.18 7.88
sh1 20.74 7.84 34.44 7.99
sh2 30.48 8.05 24.66 8.15
n111 17.31 6.40 21.49 7.72
n55 37.48 2.03 37.49 5.98
n11 26.25 8.21 37.76 8.05

location 4 - stairs
bthh 2.27 0.24 2.77 0.41
sh1 - - 0.97 0.10
sh2 1.79 0.11 1.34 0.15
n11 -- -- -- --