Gyroscopic Investing

Academic researchers have improved wireless bandwidth by an order of magnitude—not by adding base stations, tapping more spectrum, or cranking up transmitter wattage, but by using algebra to eliminate the network-clogging task of resending dropped packets of data.

...

The practical benefits of the technology, known as coded TCP, were seen on a recent test run on a New York-to-Boston Acela train, notorious for poor connectivity. By increasing their available bandwidth—the amount of data that can be relayed in a given period of time—Medard and students were able to watch blip-free YouTube videos while some other passengers struggled to get online. "They were asking us 'How did you do that?' and we said 'We're engineers!' " she jokes.

More rigorous lab studies have shown large benefits. Testing the system on Wi-Fi networks at MIT, where 2 percent of packets are typically lost, Medard's group found that a normal bandwidth of one megabit per second was boosted to 16 megabits per second. In a circumstance where losses were 5 percent—common on a fast-moving train—the method boosted bandwidth from 0.5 megabits per second to 13.5 megabits per second. In a situation with zero losses, there was little if any benefit, but loss-free wireless scenarios are rare.

http://www.technologyreview.com/news/42 ... akthrough/

Basically forward error correction, much like that used to create a RAID parity drive.

I'm amazed nobody thought of it before now.  It's a great idea, and it sounds like real world tests prove it out.

I can't wait.  I do wifi router firmware and I've been researching this for a couple of days now. 

AgAuMoney wrote: Basically forward error correction, much like that used to create a RAID parity drive.

I'm amazed nobody thought of it before now.  It's a great idea, and it sounds like real world tests prove it out.

I can't wait.  I do wifi router firmware and I've been researching this for a couple of days now.  :)

Amazing and very cool.  Ironically, pirates have been doing this for years now by using technologies like RAR for compression that generate XOR parity files, simply because files often get corrupted during transfer and instead of having to ask "hey can you please resend me the 1 file out of 100 that got corrupted" you would have enough parity files that you could rebuild the missing bits in case a few got lost.

This is also awesome because in the real world, TCP retransmits have a paralyzing effect on network traffic.  You can think of a wireless network like a crowded bar.  As 5:00pm rolls around and happy hour fills the bar, the more people having conversations at the bar make the background noise level get louder and louder.  Pretty soon people are shouting over the din.  Every time someone has to shout "What?  I couldn't hear you?"  it causes even more noise as someone has to repeat what they were saying all over again.  So, ironically, network performance and actual throughput degrade even worse due to the very algorithms that ensure our Internet actually function somewhat reliably (retransmitting lost packets).

Now, on to the implementation - it seems that in order to implement this effectively, not only would your wireless AP have to support it, but your client computer would as well.  The implementation the researchers did seemed to pre-encode the traffic (most likely in UDP to avoid TCP's automatic retransmission mechanisms) at an Amazon server, then send it to the client.  So, in any case, updates need to be made to both the client and the server for this to work.  If you update the access point and wifi driver, it can work for all traffic, or you can update each client/server app individually, but other apps that were not written for this and still use TCP will suffer and cause retransmits.

The best solution would be for wireless chipset vendors to incorporate this algorithm into both the head-end for cellular/mobile/wifi networks as well as the client chipsets.

That's why you have to study algebra, kids.