BitTorrent Architecture Overview

According to its proponents, BitTorrent is “a free speech tool”.

This is indeed the case, as it allows its users to distribute content without a centralized authority, using mainly each user’s network and computing resources in a distributed fashion.

A user’s network and computing resources are somewhat shared with the other users (also called “peers” in this context), so that everyone can benefit from expanded availability and reduced censorship properties of the BitTorrent network. The content can be stored in multiple peers at the same time: so that if a peer goes down, the content can still be obtained from the remaining peers that have a complete or partial copy of the content.

Additionally, BitTorrent allows a user to download content from multiple peers, instead of downloading it from a single server. This feature can often enable faster download speeds for its users.

Privacy can also be enhanced by using BitTorrent with minimal or no logging, whenever possible. This contrasts with downloading content from a server, which is usually logged.

Main components/terms

BitTorrent has a complex terminology. So it will be interesting to clarify its meaning.

Let’s start with:

  • Original file/content
    • The content to be published/shared with other users
  • Downloader
    • Application that implements the BitTorrent protocols and specifications
      • there are several such applications
      • examples: uTorrent, qbittorrent
  • “.torrent” file (a.k.a. Metadata file)
    • a “summary” file that summarizes the original file contents
      • the user can share the metadata file with other users, so that they can download the shared content
    • “info_hash” is a SHA hash for the “info” section in the “.torrent” file
      • this hash is used to identify the torrent and for searching for peers seeding the torrent
  • Magnet URI
    • a URI for a metadata file
      • even simpler to share than a file
      • the user can share the magnet URI with other users, so that they can download the shared content
  • Piece
    • a segment of the original file
      • the original file is split in multiple “pieces” for downloading and uploading
  • Peer
    • another user with whom to share file “pieces”
  • Swarm
    • a group of peers sharing the same “torrent”
      • downloading
      • or uploading (also known as “seeding”)
    • identified by the torrrent’s “info_hash”
  • Seed
    • a peer that has the entire “torrent” contents
      • has already downloaded all the “pieces” (or is the original publisher)
  • Tracker
    • an auxiliary service that behaves as a kind of “name server”
    • maps torrent’s “info_hashes” in lists of peers that are seeding each torrent
    • peers “announce” that their are “interested” in the torrent identified by a given “info_hash” and simultaneously receive a list of peers that also “interested”
      • “interested” peers are willing to upload and download “pieces” of the torrent
    • trackers are setup and maintained by voluntary entities
  • DHT – Distributed Hash Table
    • another auxiliary service implemented mostly by the full set of BitTorrent nodes
    • also behaves as a kind of “name server”
    • peers “announce” that their are “interested” in the torrent identified by a given “info_hash” and simultaneously receive a list of peers that also “interested”

Step 1. Creating & sharing a new torrent + swarm

screen-shot-10-21-16-at-06-33-pm

In order to publish a new torrent, a user typically has to perform the following steps:

  • 1. Create the “.torrent” metadata file
    • select which content (local files) to include in the new “.torrent”
    • optionally select which tracker(s) will be used to announce the new torrent
    • optionally select if the torrent will be private (not announced in the DHT nor in public trackers) or if it will be public (announced in the DHT and in public trackers)
    • optionally select “web seeds” for the content
      • (these are just HTTP URLs pointing to some web server that is also serving the same content via HTTP)
  • 2. Save the “.torrent” metadata file and “Magnet URI” for later use
  • 3. Announce and Seed the new “.torrent”
    • Announce
      • In the “Trackers” defined in the metadata
      • In the DHT
      • For Local Peers (in the same LAN)
    • Seed
      • allow any interested peer to download from the initial seeder
    • the “info_hash” is used to uniquely identify the “.torrent” file
  • 4. Share the “.torrent” file or Magnet URI with the intended peer audience using means external to the BitTorrent network
    • web sites, emails, chat, sms, …

Example using uTorrent (Windows):

screen-shot-10-20-16-at-07-50-pm

Example using qbittorrent (Linux):

screen-shot-10-20-16-at-07-20-pm

Step 2. Finding and Joining peers for a given torrent/swarm

screen-shot-10-22-16-at-12-28-pm

In order to find and join a torrent, a user typically has to perform the following steps:

  • 1. Search for interesting “.torrent” or Magnet URIs using means external to the BitTorrent network
    • web sites, emails, chat, sms, …
    • NOTE: BitTorrent does not provide a “content search” mechanism, as some of its predecessors did1.
  • 2. Download and Save the “.torrent” file or Magnet URI
  • 3. Calculate the “info_hash” for the “.torrent” file or Magnet URI
  • 4. Query for peers seeding the torrent and Join the Swarm
    • using the calculated “info_hash
    • announce interest in the torrent
    • get lists of peers
    • through the Trackers, the DHT, Local Peer Discovery
    • ( and also through Peer Exchange, after some peers have been found through the other mechanisms )
  • 5. Download and Upload content “Pieces” from/to other Peers
  • 6. Reassemble the original file by assembling all downloaded pieces together
  • 7. Become a Seed
    • Keep seeding the contents
    • seeding all “pieces”

Adding a new torrent file example:

Screen Shot 10-22-16 at 01.18 PM.PNG

Screen Shot 10-22-16 at 01.19 PM.PNG

screen-shot-10-22-16-at-01-19-pm-001

Trackers example:

screen-shot-10-19-16-at-12-15-pm

DHT Example:

screen-shot-10-19-16-at-12-16-pm

Step 3. Downloading and Uploading torrent/swarm pieces

During the download and even after a full download, the “Downloader” also “seeds” the “pieces” it has already downloaded. This means that other peers can download these “pieces” from it. This allows for extra availability and extra bandwidth, when there are many peers in a swarm.

When one or more peers go offline, some of the torrent “pieces” may become “not  available”. This means that other peers which still don’t have those “pieces” will no be able to download the full torrent until those “pieces” become available again.

As previously mentioned:

  • 5. Download and Upload content “Pieces” from/to other Peers
  • 6. Reassemble the original file by assembling all downloaded pieces together
  • 7. Become a Seed
    • Keep seeding the contents
    • seeding all “pieces”

Download Examples (with uTorrent)

 

Screen Shot 10-23-16 at 12.27 PM.PNG

Download status for a specific torrent/file

 

screen-shot-10-21-16-at-05-07-pm

Peers in the swarm

screen-shot-10-21-16-at-05-06-pm

Peers in the swarm

screen-shot-10-19-16-at-12-21-pm

Fully downloaded torrent/file

screen-shot-10-19-16-at-12-18-pm

Peers in the swarm with decoded countries

screen-shot-10-19-16-at-12-20-pm

Fully downloaded torrent/file in “Seeding” status

 

Final remarks

BitTorrent is a very powerful and popular free speech tool.

I will be describing more protocol details in future posts.

References

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s