BitTorrent, traffic shaping and trusting users

It has been a while since we have posted. We had the best intentions of writing a series of wrap up posts (JO has some overdue videos I made for him!) but after living in the convention centre for three weeks we were all really exhausted. The fact that tech•ed 2010 planning is already well underway means we sort of need to wrap up the 2009 blog loose end (that and I recently posted to the ausdotnet mailing list that you have to be careful allocate enough time to a blog so it doesn’t become abandonware – only to have two people mail me and say “What? Like TechEd Backstage??” 🙂 )

Anyway, I wanted to write a bit about a particular position I advocate each year and how disappointing to was to see that position abused by a very small number of network users to the detriment of other users at peak usage times.

At the outset of network planning, there is almost always someone who advocates locking down the network environment so that user access is restricted by protocol or all user traffic is forced through an HTTP proxy. I have always dug my heals in arguing that unfettered network access is the best possible delegate experience, and that there will always be delegates with oddball use cases for network access where HTTP/HTTPS only access is not sufficient. In recent tech•ed years the network has been structured like this as private IP address space with NAT – and we’ve never had complaints from users. Good old RRAS does the job!

This year, however, the scenario was a lot different. We ended up with some peak times of network congestion – admittedly only a few times – and the cause was users with their new NetBooks running BitTorrent in a way that showed a complete disregard for other users of the network.

We saw a fair few people huddled around furtively running BitTorrent. It was really disappointing to have to approach users to ask them to be more considerate to others, only to get “But I’m not running BitTorrent” as a reply. Come on people: The network ops people at tech•ed didn’t come down in the last shower.

There was even a case where one guy took his Netbook to the helpdesk to get his network access restored (we had some automated scripts blocking torrent users – but that is a post in itself). He too, allegedly, wasn’t running BitTorrent apparently (there was a folder called “TV Shows” on the desktop of his brand new Netbook).

Running BitTorrent is one thing, but there were even people who removed all of their natural limits from their client. There was one guy who would have 2000-3000 peers running at once.

All in all it was a bit depressing as I really felt like we were having our good nature taken advantage of.

BitTorrent Protocol

The first thing that people suggested (and kept suggesting to the point that Jorke and I were going to start throwing chairs at people :))  is that we simply just ‘block them’ or ‘block BitTorrent’ or even funnier “block the BitTorrent port” (You know who you are..) . Unfortunately, this is not so easy.

Broadly, there are two parts to BitTorrent. The first is the tracker protocol, and the second is the peer protocol. For the purposes of this article, I’ll use BitTorrent to refer to a number of peer to peer protocols that behave similarly. I’m also going to gloss over a lot of detail to keep this post under control otherwise it will turn out like one of my wireless ones and take all day.

Tracker Protocol

The BitTorrent tracker protocol is a relatively simple HTTP/HTTPS protocol that is used on a centralised server to:

  • Advertise torrents via an announce URL
  • Centralise information about the peers participating in the torrent
  • Collect statistics about the health of the torrent

Typically you will access a tracker over HTTP to search for torrents via a web page (this is not the Tracker Protocol obviously). Once you find a torrent you download a .torrent file which stores the torrent meta data, including the announce URL. You then launch a BitTorrent client that uses this metadata to talk to the tracker.

This is very light weight and does not really have much of a discernible network impact.

Peer Protocol

Once the client has located the tracker, it asks for a list of peers for the relevant announcement. The tracker comes back with a bunch of data, but importantly this includes the IP addresses and remote ports of peers participating.

With a list of peers in hand, the client will then attempt to communicate with those peers in order to start exchanging chunks of normally around 16 kilobytes in size. The protocol is fairly nuanced and allows clients to sniff each other out a bit like two dogs (whether they are interested in the chunks each other has, or whether one of the clients thinks the other is unfairly leeching and wants intends to throttle that client back). I wonder if that dog metaphore will make it past StalinJorke.

Filtering BitTorrent

The problem with the conceptual overview above is that you’ll see the protocol is very flexible in terms of end-point negotiation.

  • There is no fixed BitTorrent port (true, use TCP ports 6881-6889 by default but anyone who is going to specify infinity for a number of concurrent peers is probably  not going to use the default port range).
  • There is no fixed relationship between the tracker and the members of the Torrent. Once the client has a list of peers from the tracker it can continue to exchange chunks of data for a long period of time and blocking the tracker will not necessarily be effective.
  • Deep packet inspection does not help when you have Azureus, Bitcomet, and uTorrent all cooperating to implement transport level encryption.

User Behaviour

As I said above, I’ve always advocated the point of least interference in the network in the interests of a better customer experience. This has always worked really well and so it is interesting to note that tech•ed 2009 represented such a massive departure from previous tech•ed events. I am not sure whether or not to attribute this to:

  1. the fact we had really high speed Internet connectivity; or
  2. the fact we gave out Netbooks and so there were a number of users at the event with PCs who normally would not have PCs in that environment.

Regardless, the really disappointing thing to see what a lack of consideration for other users and then lying about it when cornered.

[youtube IGnMCQogijA]
1 Comment