Solving the LTE Puzzle: Comparing LTE Performance

After Verizon’s early adoption of LTE gave the carrier a commanding 2011 lead in the battle for data speed supremacy, things are heating up considerably in 2012. AT&T is aggressively rolling out its own LTE service, Sprint has promised LTE later this year, and T-Mobile isn’t far behind. At the moment, though, AT&T and Verizon command the data speed battlefield.

We’re testing carrier coverage across the country to give consumers a real-world look at mobile data performance. As part of this ongoing process, we extensively measured performances across multiple LTE markets during the first quarter of 2012 and have put together a head-to-head comparison of AT&T and Verizon based on these results. Using off-the-shelf phones, we tested both indoors and outdoors during driving, at day and at night, ultimately generating more than 50,000 data tests.

As our tests show, just because a carrier advertises a market as LTE-enabled doesn’t mean that you will always be on its LTE service. Moreover, the consumer experience of a carrier’s network is impacted by data failure rates. If you’re in the middle of uploading a file or downloading a movie and lose your data connection, those LTE speeds are meaningless. Looking solely at LTE misses the forest for the trees. It doesn’t give you a true picture of a network’s real-world performance.

Many pieces offer the whole LTE picture

Given these additional dimensions to the real-world data experience, an article that looks only at LTE would be a disservice to the public, giving you only one piece of a much more involved puzzle. Instead, we’re offering a comprehensive report from the front-lines of the growing LTE battlefield, taking you inside how a consumer actually experiences AT&T and Verizon in markets where both offer LTE build-outs. Our battle report covers five interlocking skirmish points:

How fast each carrier’s LTE service was within and across 15 markets (see charts below for the actual markets).
How often we were able to access each carrier’s LTE service. After all, it doesn’t matter how fast LTE is if you can’t reliably expect to access it.
How fast each carrier’s average speed was. This is real-life speed, a combination of LTE whenever we could access it and speeds from legacy networks included in the measurement when LTE was not available.
How often our test speeds surpassed 5 Mbps. Actual, real-world performance is what matters, not the LTE label.
How often we experience a data failure on each network. The fact that LTE is available might not mean much for your real-world usage if you’re experiencing frequent data failures.

Together, these measures of speed and accessibility help show not only where the LTE battle stands today, but also gesture toward what might become increasingly important points of differentiation as LTE service continues to mature. In short, when carriers and other reports focus solely on LTE, they are selling you the LTE sizzle; we’re giving you the sizzle plus the steak.

LTE speeds

Our head-to-head comparison of these two networks measures performance over the first quarter of 2012 and across multiple markets, throwing Verizon’s more mature LTE network together with AT&T’s nascent one to see what performance each offers consumers. We’ve stripped out all non-LTE test results to give you unvarnished, no-additives-included LTE speed.

If you’re simply comparing average download speeds on LTE, AT&T had the clear advantage. AT&T proved faster than Verizon for LTE average download speed in 11 markets, Verizon recorded faster LTE average speeds in 2 markets, and the carriers statistically tied in 2 markets. Combining our test results from all 15 markets, AT&T’s LTE service averaged a download speed of 17.4 Mbps, while Verizon averaged 15.2 Mbps.

Upload speed, though, offered less differentiation between the carriers. Instead, we found a neck-and-neck race, with AT&T offering the fastest average LTE upload speed in seven markets, Verizon doing so in four markets, and the carriers offering statistically comparable results in four markets. Their across-market LTE upload averages, moreover, were nearly mirror images, with AT&T delivering 7.98 Mbps and Verizon turning in 7.97 Mbps.

Reliable access to LTE

While AT&T appears to have the edge in average LTE download speeds, Verizon owned the battlefield when we compared how often we could actually access each carrier’s LTE networks. Verizon offers the more mature LTE network and it shows in the ability to access LTE. We found LTE service much more often with Verizon than we did with AT&T. Taking into consideration all 15 markets, we found LTE during 92 percent of our download tests with Verizon and only 66 percent of the time with AT&T.

For downloading, we were able to access Verizon’s LTE more often than AT&T’s in 13 markets, while the carriers offered statistically comparable rates in 2 markets. AT&T never recorded a greater percentage of LTE connections than Verizon. The numbers were nearly the same during our upload tests. Presumably, this difference could shrink over time as AT&T continues to grow its LTE footprint. At the time of our testing, though, Verizon offered significantly better rates for accessing LTE.

Average speed

LTE is great. But just because you have an LTE-enabled phone and are in an LTE market doesn’t mean that you will experience the blazing-fast speeds that the carriers advertise. When LTE isn’t available, you’ll be on slower, legacy technology. Looking at the broader, overall averages from each market gives you a truer sense of how your carrier’s LTE promises play out in the real-world.

While AT&T outperformed Verizon in pure LTE download speeds, Verizon had the clear upper hand in the more meaningful real-world experience of average download and upload speeds. AT&T’s LTE might be faster, but our overall experience with AT&T was still slower than what we found with Verizon.

Surpassing the 5 Mbps threshold

When the rubber meets the road, what truly matters isn’t an LTE label but how often you’re actually experiencing fast data speeds. To further delve into what the consumer could reasonably expect to find from each carrier, we compared how often AT&T and Verizon offered speeds greater than 5 Mbps for downloading and uploading. The standard for what constitutes “4G” speed is muddy, but looking at the consistency with which a carrier offers speed greater than 5 Mbps gives a good indication of a “4G” experience.

Here too, Verizon easily pulled away from AT&T. Verizon exceeded 5 Mbps during 82 percent of our download tests and 66 percent of our upload tests; AT&T surpassed this number in only 68 percent of our download tests and 40 percent of our upload tests.

The discrepancy between AT&T leading in LTE-only measures while Verizon leads in the other, more comprehensive measures is particularly interesting given that the legacy technologies offered by AT&T and Verizon are so different. AT&T touts its HSPA+ network as a viable “4G” alternative for when LTE isn’t available. It’s a valid, though perhaps limited, point: if you aren’t able to access LTE, there’s no doubt that AT&T’s HSPA+ is faster than Verizon’s EV-DO network.

Our tests, however, show that Verizon customers can expect to be on LTE the vast majority of the time they use their data network. In part because the “slips” out of LTE are few and far between on Verizon’s network, their overall average speeds still outpace those of AT&T. Our tests show that, as it now stands, Verizon’s ability to offer greater access to LTE appears to trump its reliance on EV-DO as a backup.

Focusing only on LTE speeds misses this type of nuance and favors the optimum experience over the actual everyday consumer experience. Of course, as AT&T’s network begins to offer increased percentages of access to LTE, its overall averages and percentage of tests above 5 Mbps could shift in their favor. When LTE access rates become comparable, the ability to fall back on HSPA+ rather than EV-DO could become a more compelling point of differentiation.

Data failures

In general, both carriers offered reliable data service. For those, though, who are looking for the most reliable service, Verizon edged ahead of AT&T. Verizon consistently outperformed AT&T in all data failure measures, with the exception of upload data drop failures, where the carriers offered statistically comparable performances.

Final considerations and what it all means

LTE is only one piece of a much more complicated puzzle of how consumers actually experience their data networks. It’s the hot topic, but it shouldn’t be the only topic. A well-rounded approach that puts LTE performance in the context of real-world data speeds, LTE accessibility, and data failure rates shows each carrier in its best, and worst, light and reflects true consumer experience.

If LTE download speed is your primary concern, AT&T has proven faster so far.
If consistency of LTE access is the most important factor for you, Verizon beat AT&T for both downloading and uploading.
While Verizon offered a slightly better average download speed, they outpaced AT&T by even greater margin when looking at average upload speed.
If consistency of speeds above 5 Mbps is the reason you’re interested in 4G technologies, our tests show that Verizon surpassed this threshold more often than AT&T, especially during upload testing.
If data success rates are your driving interest, both carriers performed well, but Verizon was slightly better.

We’re firm believers in giving consumers the broadest possible picture of how the carriers’ networks are performing. Consider this the first of many upcoming LTE battle reports, especially with Sprint’s 2012 entry to LTE looming.

We’ll continue to monitor the data speed battlefield to see if AT&T can maintain its impressive speeds as traffic grows and to see if Verizon’s lead in percentage of tests accessing LTE shrinks as AT&T’s LTE rollout grows. In the meantime, we encourage you to use our CoverageMap app to see what speeds you find in your own neighborhood.

Bill Moore is president of RootMetrics.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.