Thursday, September 27, 2007

BitTorrent Content for Apple TV, Pt. 2

OK, as promised in BitTorrent Content for Apple TV, Pt. 1, here's a discussion of one of the things you need to do to maximize your success with a BitTorrent client like Azureus or Transmission.

BitTorrent is a file-sharing protocol that makes it easy to download huge files (or folders full of them). Video files such as movies ripped from DVD qualify as huge. If you want to download movie files to play (usually after a format conversion) on Apple TV, you need to use BitTorrent.

So let's say you've installed Azureus and used a BitTorrent search engine such as TorrentScan to locate and download the small torrent files associated with the online movies you covet. You've opened these downloaded torrent files in Azureus, and downloading of the actual target files has begun ... but at a snail's pace. Is there anything you can do to speed it up?

Well, maybe there is. One of the big reasons why downloads go too slow is that the BitTorrent client hasn't been given an open port.
If your setup is anything like mine, your computer (or computers, plural) is "behind a router." That phrase simply means that on your home network there is a piece of gear between your computer and the cable modem or DSL modem that affords it access to the Internet. The computer "talks to" that piece of gear, called a router, the router talks to the modem, and the modem talks to the Internet.

In my own case, the "router" is actually one of Apple's Airport Extreme Base Stations, a wireless (or Wi-Fi) "access point." Your access point may be a "real" router from Belkin or Linksys or any of the other companies that make them. And you may not be using wireless technology at all. Instead, each computer may connect to the router via Ethernet cable. No matter. You're still behind a router, and you probably need to "forward" a network "port" from the router to the computer you'll be using to run the BitTorrent client.

Azureus has a so-called NAT/Server Port Test which can tell you whether a certain network port is open:


Here, I've tested port 22222, a number which I chose at random. I'll talk more in a while about how to select a port number. In my test in Azureus, port 22222 was "OK," meaning it was open and being properly port-forwarded from my router to my computer. Since it was in fact open, I clicked the Apply button to make it Azureus' official "listen port," as then revealed in the Azureus Preferences/Options pane, under Connection:



I had to set up the port forwarding myself, before running that test in Azureus. It was the first tricky part of these proceedings.

The first thing you need to do to port forward to Azureus what is to become its official listen port is give your computer a manually assigned, static IP address. If you have a Mac such as mine, open the Network panel in System Preferences, show AirPort (if that's how you connect to your access point/router, otherwise show Built-in Ethernet), and select the TCP/IP tab:



Right now, chances are you're setting Configure IPv4: to "Using DHCP." It needs to be changed to "Manually." Then, fill in the text boxes as I have done. You can change the "202" in the IP address field to any number from 201 to 255, as long as no other computer on your home network uses that number.

You also have to replace the DNS server addresses I'm using with the one(s) your Internet service provider provides. This is one of the tricky but essential steps. If you have an Apple base station, open it for manual setup in AirPort Utility. Go to the Internet Connection page:


The DNS server IP address(es) assigned to your Internet connection by your ISP appear in gray, meaning that you needn't enter them yourself in this AirPort Utility panel. Instead, the ISP (in my case, Comcast) filled in the addresses for you.

When you direct your Web browser to, say, the URL http://www.google.com, a DNS server is the piece of the Internet that translates the supplied URL to an actual IP (Internet Protocol) address for Google. Without access to a DNS server, your browser would be confined to using the literal IP address.

When your computer uses a static IP address such as 10.0.1.202, it no longer can take advantage of the DNS server addresses that your router knows about ... unless you copy those DNS server addresses from the router's Internet Connection panel in AirPort Utility to your computer's Network Preferences panel. So do so before going any further, and apply the changes.


While you are still in AirPort Utility, you need to "port map," or "port forward," the listen port number you've decided on, which in my example is 22222. To do that, click on the Advanced icon and then on the Port Mapping tab:



Here you see what my setup looked like before I port forwarded 22222. I had forwarded ports 6881, 11111, 25670, and 9090 earlier, each of these port numbers to exactly one of my two computers. (You can't port forward any single port number to more than one computer.)

To port forward 22222, I clicked on the + sign and entered the following:



22222 was entered as both the public port number and the private port number, and the private IP address I entered was 10.0.1.202, the address which I assigned to my BitTorrent client computer earlier, in Network Preferences. The description field is left blank, as you cannot type anything into it.

When I clicked OK, the list of mapped ports shown in the previous figure had one more item added to it:

22222 -> 10.0.1.202:22222

After updating the base station (don't forget to do that by clicking on the Update button and waiting for the base station to restart), I was able to successfully run the Azureus NAT/Server Port Test I discussed earlier.

If you don't have an Apple base station, or if you aren't using a Mac, you can probably find instructions for doing port forwarding in your setup at PortForward.com.


I promised to talk about how you pick the port number to use with your BitTorrent client as its listen port.

The "well known" BitTorrent ports are in the range 6881-6889, so try 6881 first, if you like.

If you run more than one client either on a single computer or on more than one computer behind the same router, then for each of the clients that will be in simultaneous use you have to open up a different port. Accordingly, client number two would use 6882, number three would use 6883, and so on.

There's nothing really magic about the 6881-6889 range. The game in picking port numbers is to avoid using a number that any other application/service uses.

When a port number is used by an Internet application or service, it has to do with traffic coming into your local network from the wider Internet. When a computer or server out in cyberspace initiates contact with your computer, it uses a specific port number associated with the type of service in question. For example, the World Wide Web typically uses port 80. So if you happened to be running your own Web server, you would have to port forward port 80 to it. That way, Web browsers in cyberspace could access it.

When you open a torrent file in a BitTorrent client, it announces itself to the "tracker" whose URL is listed in the torrent file. The tracker is a special kind of server which co-ordinates between all the clients on the Internet using that same torrent file. When your client announces itself, the tracker learns (among other things) the IP address of your router and the listen port of your client. It shares that information with other clients using the same torrent file. Those other clients typically initiate contact with your client.

When they do, they use your router's IP address and your client's listen port. If your listen port isn't properly port forwarded by your router, the remote clients get no valid response from your local client. That means the remote clients won't be able to share pieces of the torrent with your client. The only source of torrent pieces your client will be able to take advantage of will be those remote clients that it, your client, initiates contact with itself. That's why downloads too too slow when the listen port isn't open.

You can actually pick any port number from 1 to 65565 for your listen port, but the numbers below 1024 are all taken, and many of those between 1024 and 49151 are too. The ones between 49152 and 65565 are less likely to be reserved. Check this list if you want to be ultra-careful. But remember: the vast majority of port numbers will probably never get external traffic coming into your network, for the simple reason that you are not running anything that requires it. If you pick a port number that is officially reserved, there may be no real harm done.

Also, some ISPs go out of their way to interfere with BitTorrent users who use the "well known" port numbers. You can learn more about that here.

Thursday, September 20, 2007

BitTorrent Content for Apple TV, Pt. 1

In Apple TV: Getting Content I talked about how BitTorrent technology can be used to exchange Apple TV and other content with other Internet users. Now I'd like to extend those remarks in a series of posts about exactly how that is done.

In the earlier post I said BitTorrent is

a peer-to-peer file sharing (P2P) protocol for distributing large amounts of data widely across cyberspace in a decentralized way. The general idea is that every file — movie or otherwise — is divided up into many, many, many tiny pieces, every one of which can be redundantly stored in different computers on the Internet. The pieces can be downloaded all at a time, in no particular sequence, from any of these locations. The BitTorrent client software that you run on your computer finds a source computer for each piece, downloads all the pieces one by one, and assembles them in their proper order to make a single file on your hard drive which is exactly like the original file.

All that still goes, but I should note that a lot of people don't think of BitTorrent as P2P, since it does things so much differently than earlier P2P methodologies such as LimeWire and KaZaA. For one thing, those earlier technologies didn't break files into pieces to be obtained from multiple sources.

Also, BitTorrent as a protocol for sharing files needs to be distinguished from BitTorrent as one of several available software clients that you can download and use to do file sharing, via the BitTorrent protocol, on the Internet. (That particular client is sometimes referred to as "the Official BitTorrent Client," or OBTC. There are other clients which in my humble opinion are better, such as Azureus. I'll talk about Azureus and some of the others in later posts.)

Furthermore, there is the official BitTorrent website, where the Official BitTorrent Client can be obtained, along with many (legal) torrent files.

And another thing: when we talk about sharing a file as a "torrent," what we really mean is that we are sharing either one individual computer file or a folder/directory containing one or more individual files. Torrents can include multiple files.

For example, a particular torrent may contain, in a single folder: a movie file, in perhaps the .avi format; one or more subtitle files in various languages, which the user may optionally apply to the movie; and perhaps a .jpg file showing DVD cover art for the movie.

(Notice that I used an .avi file as an example of a typical movie torrent's content. That was because a large proportion of movies available in the BitTorrent universe use that file format ... which does not happen to work in iTunes or Apple TV. In Apple TV: Getting Content I talked about that problem, along with some ways to deal with it by converting the .avi files to an iTunes-compatible format. I'll talk more about that concern in future posts.)


That word, "torrent," is a bit ambiguous. It can refer to the data file (or folder of many files) which is the ultimate target of the BitTorrent client. It can refer to a special file called a "torrent file" which describes the target file or folder; this so-called torrent file, which is small, contains "metadata" about the "target file," which is typically big. Or the word "torrent" can simply refer to the whole package, the target data and the metadata.

In order to download a torrent data file, also known as the target file (or folder), you first have to obtain the torrent file — the small file with metadata in it. The way you usually do that is to go to either a torrent search site on the Web or a torrent "tracker site" — more later on what a "tracker" is — find a torrent for (let us say) a movie you want to download, and download that in the "ordinary" way: click on a "Download" hotlink/button in your browser, at which point your browser will do it's customary downloading thing.

Once a copy of the torrent file is present on your desktop (or wherever you put it when you downloaded it), you simply open it in your favorite BitTorrent client software.

When you open the torrent file, the client will probably ask you where on your hard drive you intend to put the downloaded torrent target file it is about to create. Alternatively, the client may have a fixed place into which it is set up to put all its downloads. Chances are, depending on the client, you can select which of those two behavior patterns you prefer. You do this in the client's Preferences/Options. If you select to download to a fixed automatic location, you can also choose the exact location you want while you are in the client's Preferences/Options.

If all goes well, the client will then at this point do some stuff behind the scenes. Then, after a brief delay, you will see the client give you some indication that downloading has successfully begun. (If that doesn't happen pretty quickly, something is wrong. I'll talk about what may be amiss in a later post. But for now, keep in mind that some torrents aren't very popular and for that reason download slowly. If the first torrent you try is balky, try some others.)


What has actually happened when you open a torrent file? Put simply, when you open the torrent file in a BitTorrent client, the client arranges for you to join the "swarm" of "peers" all trying to download the same torrent!

That is, all the peers — in file-sharing protocols, "peers" are treated as equals — are after the same target data file or folder. Each peer presumably has some but not all of the pieces of the target. When you open the torrent metadata file, your client reads it and announces your presence to a "tracker" whose URL is contained therein.

This tracker is software running on a Web server, somewhere on the Internet. When your client announces that you have opened the torrent file, the tracker sends back current information about the "swarm" of peers for that torrent. Specifically, it tells you client what other peers are in the swarm and what the IP (Internet Protocol) address of each is. It's then up to your client to connect to the client software running on the other peers' computers.

When a connection is made by your client with a certain peer, your client and the peer's client exchange information about which pieces of the torrent each already has. When you open a torrent file for the first time, obviously you have yet to possess any of the pieces, so the respective BitTorrent clients treat that as a special situation, in order that you can get started downloading the torrent.

More generally, each peer you connect to will expect you to have some of the pieces it wants and will expect itself to have some of the pieces you want. The two clients will negotiate an exchange.

Notice that once you have downloaded some of the pieces of the torrent, you can expect peers you have yet to initiate contact with to initiate contact with you — that is, with your BitTorrent client. It is simply the mirror image of the process just described, except that it is initiated by one or more remote peers rather than by your "local" client.

Very quickly, as you can see, what you began as a download turns into an upload as well. No matter whether the other peer — the second peer— initiated the connection or you did, torrent pieces you have obtained elsewhere (i.e., from yet other peers) typically will start flying up to that second peer — or, rather, to any number of remote peers.

In fact, the BitTorrent protocol is implemented in such a way that if for any reason your client is not uploading pieces at a decent clip, other clients will "choke" their download speeds to you. Mark that well. If you want decent download speeds, you need to insure decent upload speeds.


Assume you have just begun downloading a torrent. Now it's a matter of waiting for the download to complete. Depending on any number of factors, that can take as little as two or three hours, for a full-length movie, to as much as several days.

Naturally, you want your download to go as fast as possible. One of the things I want to cover in the next post in this series has to do with avoiding a common mistake that can limit download (and upload) speeds. Hint: it has to do with making sure your client is using an "open port."

Using an open port requires that you open a particular "port number" — one of your choosing — and to tell your client what it is. Opening a port number involves, first of all, telling your computer's firewall not to screen out incoming traffic on that network port.

If your computer is "behind" a router on a local area network in your home or office, you also have to "port forward" the chosen port number from the router to your computer.

If the router also has a firewall, you have to open the port number on the router.

More about opening ports next time ...