BitTorrent Protocol (a.k.a. Peer Protocol) examples

Here I present some examples of BitTorrent protocol interactions.

Wireshark can be used to analyze BitTorrent protocol interactions in TCP/IP.

Remember that BitTorrent’s peer protocol operates over TCP or uTP. At the time of writing, Wireshark could identify correctly a uTP connection, but unfortunately would not decode its contents as a BitTorrent protocol session. It decodes it fine for TCP/IP connections.

Message flow/sequence

screen-shot-10-21-16-at-04-34-pm

Handshake example

The Handshake message flows in both directions, this means that each peer sends an handshake message to the other.

screen-shot-10-25-16-at-03-54-pm

“Extended” message examples

In these messages we can see which extensions are supported by a peer / downloader.

uTorrent

d
1:m
d1
1:upload_only
i3e
11:lt_donthave
i7e
12:ut_holepunch
i4e
11:ut_metadata
i2e
6:ut_pex
i1e
10:ut_comment
i6e
e
13:metadata_size
i401e
1:p
i20111e
4:reqq
i255e
1:v
15:Torrent 3.4.8
2:yp
i6881e
6:yourip4:
e

qbittorent

d
1:m
d1
1:lt_donthave
i5e
11:upload_only
i2e1
1:ut_metadata
i15e
6:ut_pex
i1e
e
13:metadata_sizei401e
4:reqq
i250e
1:v
18:qBittorrent v2.9.8
6:yourip
4:
e

 

Port, Interested, Unchoke example

Screen Shot 10-12-17 at 11.34 AM

Request+Piece example

A request for a piece of a file:

Screen Shot 10-12-17 at 11.36 AM

The reply with the piece’s data contents:

Screen Shot 10-12-17 at 11.37 AM

Not Interested example

Screen Shot 10-12-17 at 11.37 AM 001

 

Downloader Peers screenshots

Usually, when a peer is connected to another one, the remote peer appears in the “Peers” tab for a torrent.

Example

screen-shot-10-19-16-at-12-18-pm

Downloader Ports configuration

uTorrent

screen-shot-10-19-16-at-12-25-pmscreen-shot-10-19-16-at-12-23-pm

 

screen-shot-10-20-16-at-07-26-pm

References

 

 

 

Advertisements

Zero Net : P2P websites without central web server

ZeroNet is a very interesting distributed platform for content publishing and sharing. It has some strong properties, which are very interesting to bypass censorship and online hosting costs.

According to its proponents, it provides:

Open, free and uncensorable websites,
using Bitcoin cryptography and BitTorrent network

In this post we will see how ZeroNet can provide these properties.

Interesting features

  • Password-less BIP32 based authorization
    • Your account is protected by the same cryptography as your Bitcoin wallet.
    • You will need your private key to make changes to a given site
    • the site’s address is also BIP32-based
  • Real-time updated sites
    • visitors are notified of changes
    • and they update their browser contents if needed
  • Namecoin .bit domains support
    • for human intelligible site names
  • SQL Database support
    • runs locally
    • dedicated API for database access
  • Anonymity:
    • Full Tor network support with .onion hidden services instead of ipv4 addresses
  • TLS encrypted connections.
  • Automatic, uPnP port opening.
    • can be interesting for visitors using TCP/IP connections
  • Works with any browser/OS.

How does it work ?

According to its proponents, when you vist a ZeroNet site, this is what happens:

ad-hoc-and-p2p_1014841493414094001

ad-hoc-and-p2p_7615479204197627471

Digging a little bit deeper into detail, we find the following architecture:

screen-shot-02-05-17-at-06-51-pm-001

  1. Create a new site: a BIP-32 address and its respective public and private keys are created
  2. A BitTorrent compatible “.torrent” is created for announcing purposes
  3. The BIP-32 address is announced in the BitTorrent network trackers
  4. The BIP-32 address can be shared with the public
  5. A new visitor asks the BitTorrent trackers for other visitors serving the BIP-32 address
  6. The new visitor connects to the first ZeroNet server (using TCP/IP or TOR) and asks for “content.json“.
  7. After validating the “content.json” contents, it asks for the remaining site contents and validates its signature. It stores all the site’s contents locally
  8. The new visitor announces that it is also serving the site

Site Name

ZeroNet uses a BIP-32 address for the site name. This address can be concatenated to the local ZeroNet server.

For example:

Domain names

Namecoin .bit domains are supported: these addresses can be concatenated to the local ZeroNet server.

  • Example:
  • Other examples:

    screen-shot-11-02-16-at-12-16-pm

  • https://namecoin.org/
    • «Namecoin was the first fork of Bitcoin and still is one of the most innovative “altcoins”. It was first to implement merged mining and a decentralized DNS. Namecoin was also the first solution to Zooko’s Triangle, the long-standing problem of producing a naming system that is simultaneously secure, decentralized, and human-meaningful.»
    • «Namecoin is an experimental open-source technology which improves decentralization, security, censorship resistance, privacy, and speed of certain components of the Internet infrastructure such as DNS and identities.»

Other Benefits

Some additional benefits include:

  • Transparent sites: all the site code for all ZeroNet sites is available and can be audited if needed
  • One click cloning
    • it is very easy to clone an existing site and modify it
  • No back-end code
    • everything runs locally, there is no central server
      • even the database runs locally
    • this means the code runs very fast
    • and that there is no single point of failure or censorship
    • API for local database access
    • offline access is naturally supported
  • Instant CDN (Content Delivery Network)
    • the system behaves naturally as a CDN because it naturally produces multiple site copies (one for each visitor)
  • It is able to run on non-internet networks
    • Mesh-networks, Bluetooth, WiFi-direct, for example
  • Zero discrimination
    • everyone can build a site for a zero cost infrastructure
  • Zero trust
    • it is impossible to modify a site without the original private key (for the BIP32 address)
    • NOTE that it is not impossible to make a copy of the site, it’s just impossible to change a site at a given address without the private key

Other Features and limitations

  • No torrent-like file splitting for big file support
  • Support for anonymous communication (built-in full Tor support added)
  • File transactions are not compressed
  • Encrypted communication (TLS encryption added)
  • No private sites
  • Local ZeroNet service uses 15441 TCP/IP port (or a TOR hidden service) for inbound connections from other visitors

Demo

Step 0 – Install ZeroNet

  • Decompress the package and run the script:
    • ZeroNet.cmd (win),
    • ZeroNet(.app) (osx),
    • ZeroNet.sh (linux)
root@osboxes:~/Downloads# cd ZeroBundle/
root@osboxes:~/Downloads/ZeroBundle# ls -l
lib  
LICENSE.txt  
Python  
ssl  
ZeroNet.sh
root@osboxes:~/Downloads/ZeroBundle# ./ZeroNet.sh 
Downloading https://github.com/HelloZeroNet/ZeroNet/archive/master.zip to ZeroNet directory.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Downloaded.
Extracting... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Done.
Starting ZeroNet/"start.py"...
- Starting ZeroNet...
[15:24:11] - OpenSSL loaded, version: 01000207F
[15:24:12] - Version: 0.4.1 r1536, Python 2.7.11 |Continuum Analytics, Inc.| (default, Dec  6 2015, 18:08:32) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)], Gevent: 1.0.2
[
15:24:12] - Creating FileServer....
[15:24:12] TorManager Tor controller connect error: AssertionError: Tor version >=0.2.7.5 required, found: 0.2.2.35 in TorManager.py line 159
[15:24:12] - Creating UiServer....
[15:24:12] Site:1Name2..hM9F Content.json not exist: data/1Name2NXVi1RDPDgf5617UoW7xA6YrhM9F/content.json
[15:24:12] - Removing old SSL certs...
[15:24:12] - Starting servers....
[15:24:12] Ui.UiServer --------------------------------------
[15:24:12] Ui.UiServer Web interface: http://127.0.0.1:43110/
[15:24:12] Ui.UiServer --------------------------------------
[15:24:12] - Opening browser: default_browser...
[15:24:12] Site:1Name2..hM9F Content.json not exist: data/1Name2NXVi1RDPDgf5617UoW7xA6YrhM9F/content.json
[15:24:16] FileServer Checking port 15441 using portchecker.co...
[15:24:21] FileServer [BAD :(] Port closed: Port 15441 is closed.
[15:24:21] FileServer Trying to open port using UpnpPunch...
[15:24:34] Site:1HeLLo..Tf3D Content.json not exist: data/1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D/content.json
[15:24:36] FileServer UpnpPunch run error: UpnpError: Failed to communicate with igd using port 15441 on local machine after 3 tries. in FileServer.py line 73 > UpnpPunch.py line 319 > UpnpPunch.py line 310


screen-shot-11-02-16-at-12-11-pm

screen-shot-11-02-16-at-12-12-pm

Step 1 – Create a new site

You can create an empty site through the command-line, as follows:

# zeronet.cmd siteCreate

- Starting ZeroNet...
 - OpenSSL loaded, version: 01000201F
 - Version: 0.4.1 r1536, Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:19:
 22) [MSC v.1500 32 bit (Intel)], Gevent: 1.1.2
 - Generating new privatekey...
 - ----------------------------------------------------------------------
 - Site private key: 5KfnCgnyqNZWrRJ8tWeg1c5jaXXXXXXXXXXXXXXXXX
 -                   !!! ^ Save it now, required to modify the site ^ !!!
 - Site address:     1KVaAeeVhhW43FsYPWiEK7FSTuXXXXXXXX
 - ----------------------------------------------------------------------
 ? Have you secured your private key? (yes, no) >

Alternatively, you can clone an existing site (much easier option to start with).

Step 2 – View the new site contents

  • In a browser, we can access the new site with the following URL:
  • http://127.0.0.1:43110/1KVaAeeVhhW43FsYPWiEK7FSTuXXXXXXXX
  • Note that the URL to access a ZeroNet site is composed of the local service URL concatenated with site’s BIP-32 address

screen-shot-11-02-16-at-12-10-pm

Step 3 – Update the new site contents

ZeroNet provides some site templates which you can use to quickly add your content.

For example, the Blog and Talk templates allow the content files to be changed through the web interface without any coding. You can clone the ZeroBlog site to create a new blog.

screen-shot-11-02-16-at-01-24-pm

screen-shot-11-02-16-at-03-41-pm

Accessing the new site from another host:

screen-shot-11-02-16-at-04-01-pm

If you want to update your site files manually, you will have to perform some more steps from the command-line:

  • Update the site files located in data/[your site address key] 
    (eg:1KVaAeeVhhW43FsYPWiEK7FSTuXXXXXXXX).
  • When your site is ready run:
$ zeronet.py siteSign 1KVaAeeVhhW43FsYPWiEK7FSTuXXXXXXXX 
- Signing site: 1KVaAeeVhhW43FsYPWiEK7FSTuXXXXXXXX
... Private key (input hidden): 
<enter your private key>

  • in order to inform peers about the changes you made you need to run:
$ zeronet.py sitePublish 1KVaAeeVhhW43FsYPWiEK7FSTuXXXXXXXX
... Site:13DNDk..bhC2 Publishing to 3/10 peers... 
Site:13DNDk..bhC2 Successfuly published to 3 peers 
- Serving files.... 

Stats

You can check your local service’s stats using the following URL:

rev1536 | 88.210.64.173 | Opened: False | Crypt: ['tls-rsa'] | In: 7.23MB, Out: 0.20MB | Peerid: -ZN0041-sbW9XT0WjF9k |
Connections (4, total made: 109):
id proto type ip open crypt ping buff bad idle open delay out in last sent waiting version sites
86 v2 out 112.115.5.254:15441 True ECDHE-RSA-AES128-GCM-SHA256 0.515 0 0 716 6149 0.000 2kB 1kB listModified [] 0.4.1 r1536 3
88 v2 out 78.213.146.86:15441 True AES128-GCM-SHA256 0.140 0 0 716 6130 0.000 2kB 1kB pex [] 0.3.6 r915 3
94 v2 out 78.135.51.152:15441 True ECDHE-RSA-AES128-GCM-SHA256 0.374 0 0 716 6105 0.000 1kB 0kB listModified [] 0.4.1 r1536 2
96 v2 out 179.8.205.91:15441 True ECDHE-RSA-AES128-GCM-SHA256 0.359 0 0 613 6103 0.000 0kB 0kB listModified [] 0.4.1 r1536 2

Tor hidden services (status: OK (7 onion running)):
- 1KVaAeeVhhW43FsYPWiEK7FSTuHdGnbeoX: hyvex7lfikk277kq
- global : bkgn22ygvetxb3ze
- 1BLogC9LN4oPDcruNz3qo1ysa133E9AGg8: 2mtnv2xrii6rrym4
- 131a4VbWAo21oeLffGfty9DVy4SnnPHCND: xzykj3o5yi6vd2xc
- 1Q9xoVam2Kx4Y5r76QkPDCLVMq9t4jRqZc: bol2gf6sil5jbf3n
- 1Name2NXVi1RDPDgf5617UoW7xA6YrhM9F: 6j2u2jiecp3l2d3b
- 1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D: ifnzspvmmwuori33

Db:
- 128.979s: data/content.db
- 127.196s: data/1BLogC9LN4oPDcruNz3qo1ysa133E9AGg8/data/zeroblog.db
- 128.952s: data/1Q9xoVam2Kx4Y5r76QkPDCLVMq9t4jRqZc/data/zeroblog.db

Sites:
address connected peers content.json out in
131a4VbWAo21oeLffGfty9DVy4SnnPHCND [] 0/0/0 0 (loaded: 0) 0kB 0kB
1BLogC9LN4oPDcruNz3qo1ysa133E9AGg8 [88, 86] 2/2/376 465 (loaded: 52) 0kB 3458kB
1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D [94, 96, 86, 88] 4/4/630 1 (loaded: 1) 0kB 807kB
1KVaAeeVhhW43FsYPWiEK7FSTuHdGnbeoX [] 0/0/0 1 (loaded: 1) 0kB 0kB
1Name2NXVi1RDPDgf5617UoW7xA6YrhM9F [94, 96, 86, 88] 4/4/647 1 (loaded: 1) 0kB 818kB
1Q9xoVam2Kx4Y5r76QkPDCLVMq9t4jRqZc [] 0/0/0 3 (loaded: 3) 0kB 0kB

Final Remarks

ZeroNet is a promising technology for the Zero Marginal Cost Society, as it enables democratic access to publishing resources for creative commons sharing, without having to use any private enterprise infrastructure site to host your site content.

This means that ZeroNet is a very interesting technology  if you are worried that your site could be censored by any public authority or private entity, or if you would simple like to use it freely, as it does not imply any fee or payment to external entities.

References

BitTorrent Protocols Analysis

The “BitTorrent Protocol” is, in fact, a set of protocols, used in different stages, such as torrent discovery, peer discovery, download, seeding, and so on.

Let’s take a look at the following diagram.

screen-shot-10-27-16-at-11-26-am

We can see the different protocols in action.

Let’s take a closer look at them

  • HTTP Tracker Protocol
    • This is the oldest (and original) Tracker Protocol
    • Typically, it consists of an HTTP GET request for a given torrent/swarm with the following arguments, announcing the peer’s interest in the swarm and querying the tracker for more interested peers
      • info_hash: 20 byte sha1 hash of the bencoded form of the info value from the metainfo file
      • peer_id: string of length 20 which the downloader uses as its id
      • example
        • GET /announce?peer_id=aaaaaaaaaaaaaaaaaaaa&info_hash=aaaaaaaaaaaaaaaaaaaa&port=6881&left=0&downloaded=100&uploaded=0&compact=1
    • The HTTP response will usually contain a list of peers in the requested swarm
      • Each peer entry in the list contains:
        • peer id
        • ip
        • port
    • URLs for this protocol take the form: http://tracker:port/announce?peer_id=X1&info_hash=X2&port=X3&left=X4&downloaded=100&uploaded=0&compact=1
  • UDP Tracker Protocol
    • it is similar to the HTTP Tracker Protocol, but it is a binary protocol and it is UDP-based
    • it is usually lighter and faster than the HTTP version
    • URLs for this protocol take the form: udp://tracker:port
  • Local Peer Discovery
    • this protocol introduces a mechanism  to announce the presence of swarm to potential peers in the same LAN, using “http over udp-multicast”
    • a peer can broadcast its swarms to these multicast groups:
      • A) 239.192.152.143:6771 (org-local)
      • B) [ff15::efc0:988f]:6771 (site-local)
    • An announce broadcast message looks like the following:
      • BT-SEARCH * HTTP/1.1
        Host: <host>
        Port: <port>
        Infohash: <ihash>
        cookie: <cookie (optional)>
    • Usually, a peer broadcasts these messages every 5 minutes
  • DHT Protocol – “Distributed sloppy hash table”
    • The purpose of this protocol is similar to the previous ones: to find peers interested in a given swarm/torrent, but with an interesting twist:
      • it is implemented inside each Downloader, so that no trackers are needed
      • therefore, torrents announced in the DHT are potentially easier to find, albeit kind of “more public”, because their public “visibility” is not restricted to a particular set of trackers
    • Node
      • in the BitTorrent terminology, a Node is an entity that implements the DHT protocol
      • typically, a Downloader contains a DHT Node that implements the DHT protocol, cooperating with the other Nodes
      • each node is assigned a globally unique identifier, the “node id
    • The protocol is based on Kademlia (A Peer-to-peer DHT algorithm Based on the XOR Metric)
    • DHT Queries
      • ping
        • to keep the connection alive between two nodes
      • find_node
        • “id” containing the node ID of the querying node
        • “target” containing the ID of the node sought
        • this query can useful for torrents that specify specific nodes instead of trackers
      • get_peers
        • get peers associated with a torrent infohash
        • if the queried node knows some peers with the infohash, they are returned
        • otherwise, the node returns a list of nodes that are “closer” to the queried infohash
          • in this case, the querying node should continue by querying these nodes
      • announce_peer
        • Announce that the peer controlling the querying node is interested in the torrent with the given infohash
  • BitTorrent Protocol (a.k.a. Peer Protocol )
    • as the name implies, this is the most important BitTorrent protocol
    • it is used for symmetrical communication between peers, including
      • data transfer (torrent “pieces”)
      • metadata transfer (for torrents)
      • extended protocol data and metadata exchange
    • it can run over TCP/IP or using a BitTorrent specific transport layer, uTP over UDP
      • uTP can be used to improve congestion management
    • typical message flow includes:
      • handshake + extensions
        • each peer sends an handshake message to the other
          • the handshake includes the desired torrent’s “infohash”
            • note that when contacting a new peer, the originator peer believes that the target peer has some of the desired torrent’s pieces
              • because it found it through some tracking mechanism based on the infohash
        • followed by
          • extensions supported
          • and a never-ending stream of length-prefixed messages
      • base messages
        • 0 – choke, 1 – unchoke
          • for bandwidth control
        • 2 – interested, 3 – not interested
          • specifies if the peer is interested in the pieces that the other peer has available for downloading
        • 4 – have
          • a peer can inform the other peer of which torrent pieces it already has downloaded
        • 6 – request, 7 – piece
          • a peer can “request” a given piece
          • the other peer returns the “piece”
      • a peer will keep requesting the “pieces” it hasn’t downloaded yet to the other peer
        • the other peer will return them
        • if a peer detects that the other one is not allowing it to download as many pieces as it should, it may “choke” the other peer, which means it will not allow it to download more pieces until “unchoked”
        • if a peer detects that the other peer only has pieces that it has already downloaded, it should send it a “not interested” message
    • the BitTorrent protocol can be extended with new messages, which peers can use to check for extra functionality in other peers
      • extension examples:
        • DHT node support
          • (was not present in the initial protocol version)
        • Torrent Metadata exchange support
          • (was not present in the initial protocol version)
        • Peer Exchange (PEX)
          • (was not present in the initial protocol version)
    • Encryption (or more precisely, Obfuscation )
      • a weak encryption can be used in this protocol, to try to hide the BitTorrent protocol from ISPs that block or perform traffic shaping over this traffic
      • an encapsulation protocol (called Message Stream Encryption or PHE) can be used for this purpose
        • it uses a completely random header and a D-H key exchange in order to accomplish its purpose
      • a downloader can announce encryption support to a tracker by using the following extra arguments
        • supportcrypto=1, requirecrypto=1, cryptoport=X

Conclusion

This concludes this quick overview of the protocols used by the BitTorrent network.

In the following posts, I will show some low-level protocol examples.

References

 

 

 

 

 

 

 

 

 

 

BitTorrent Architecture Overview

According to its proponents, BitTorrent is “a free speech tool”.

This is indeed the case, as it allows its users to distribute content without a centralized authority, using mainly each user’s network and computing resources in a distributed fashion.

A user’s network and computing resources are somewhat shared with the other users (also called “peers” in this context), so that everyone can benefit from expanded availability and reduced censorship properties of the BitTorrent network. The content can be stored in multiple peers at the same time: so that if a peer goes down, the content can still be obtained from the remaining peers that have a complete or partial copy of the content.

Additionally, BitTorrent allows a user to download content from multiple peers, instead of downloading it from a single server. This feature can often enable faster download speeds for its users.

Privacy can also be enhanced by using BitTorrent with minimal or no logging, whenever possible. This contrasts with downloading content from a server, which is usually logged.

Main components/terms

BitTorrent has a complex terminology. So it will be interesting to clarify its meaning.

Let’s start with:

  • Original file/content
    • The content to be published/shared with other users
  • Downloader
    • Application that implements the BitTorrent protocols and specifications
      • there are several such applications
      • examples: uTorrent, qbittorrent
  • “.torrent” file (a.k.a. Metadata file)
    • a “summary” file that summarizes the original file contents
      • the user can share the metadata file with other users, so that they can download the shared content
    • “info_hash” is a SHA hash for the “info” section in the “.torrent” file
      • this hash is used to identify the torrent and for searching for peers seeding the torrent
  • Magnet URI
    • a URI for a metadata file
      • even simpler to share than a file
      • the user can share the magnet URI with other users, so that they can download the shared content
  • Piece
    • a segment of the original file
      • the original file is split in multiple “pieces” for downloading and uploading
  • Peer
    • another user with whom to share file “pieces”
  • Swarm
    • a group of peers sharing the same “torrent”
      • downloading
      • or uploading (also known as “seeding”)
    • identified by the torrrent’s “info_hash”
  • Seed
    • a peer that has the entire “torrent” contents
      • has already downloaded all the “pieces” (or is the original publisher)
  • Tracker
    • an auxiliary service that behaves as a kind of “name server”
    • maps torrent’s “info_hashes” in lists of peers that are seeding each torrent
    • peers “announce” that their are “interested” in the torrent identified by a given “info_hash” and simultaneously receive a list of peers that also “interested”
      • “interested” peers are willing to upload and download “pieces” of the torrent
    • trackers are setup and maintained by voluntary entities
  • DHT – Distributed Hash Table
    • another auxiliary service implemented mostly by the full set of BitTorrent nodes
    • also behaves as a kind of “name server”
    • peers “announce” that their are “interested” in the torrent identified by a given “info_hash” and simultaneously receive a list of peers that also “interested”

Step 1. Creating & sharing a new torrent + swarm

screen-shot-10-21-16-at-06-33-pm

In order to publish a new torrent, a user typically has to perform the following steps:

  • 1. Create the “.torrent” metadata file
    • select which content (local files) to include in the new “.torrent”
    • optionally select which tracker(s) will be used to announce the new torrent
    • optionally select if the torrent will be private (not announced in the DHT nor in public trackers) or if it will be public (announced in the DHT and in public trackers)
    • optionally select “web seeds” for the content
      • (these are just HTTP URLs pointing to some web server that is also serving the same content via HTTP)
  • 2. Save the “.torrent” metadata file and “Magnet URI” for later use
  • 3. Announce and Seed the new “.torrent”
    • Announce
      • In the “Trackers” defined in the metadata
      • In the DHT
      • For Local Peers (in the same LAN)
    • Seed
      • allow any interested peer to download from the initial seeder
    • the “info_hash” is used to uniquely identify the “.torrent” file
  • 4. Share the “.torrent” file or Magnet URI with the intended peer audience using means external to the BitTorrent network
    • web sites, emails, chat, sms, …

Example using uTorrent (Windows):

screen-shot-10-20-16-at-07-50-pm

Example using qbittorrent (Linux):

screen-shot-10-20-16-at-07-20-pm

Step 2. Finding and Joining peers for a given torrent/swarm

screen-shot-10-22-16-at-12-28-pm

In order to find and join a torrent, a user typically has to perform the following steps:

  • 1. Search for interesting “.torrent” or Magnet URIs using means external to the BitTorrent network
    • web sites, emails, chat, sms, …
    • NOTE: BitTorrent does not provide a “content search” mechanism, as some of its predecessors did1.
  • 2. Download and Save the “.torrent” file or Magnet URI
  • 3. Calculate the “info_hash” for the “.torrent” file or Magnet URI
  • 4. Query for peers seeding the torrent and Join the Swarm
    • using the calculated “info_hash
    • announce interest in the torrent
    • get lists of peers
    • through the Trackers, the DHT, Local Peer Discovery
    • ( and also through Peer Exchange, after some peers have been found through the other mechanisms )
  • 5. Download and Upload content “Pieces” from/to other Peers
  • 6. Reassemble the original file by assembling all downloaded pieces together
  • 7. Become a Seed
    • Keep seeding the contents
    • seeding all “pieces”

Adding a new torrent file example:

Screen Shot 10-22-16 at 01.18 PM.PNG

Screen Shot 10-22-16 at 01.19 PM.PNG

screen-shot-10-22-16-at-01-19-pm-001

Trackers example:

screen-shot-10-19-16-at-12-15-pm

DHT Example:

screen-shot-10-19-16-at-12-16-pm

Step 3. Downloading and Uploading torrent/swarm pieces

During the download and even after a full download, the “Downloader” also “seeds” the “pieces” it has already downloaded. This means that other peers can download these “pieces” from it. This allows for extra availability and extra bandwidth, when there are many peers in a swarm.

When one or more peers go offline, some of the torrent “pieces” may become “not  available”. This means that other peers which still don’t have those “pieces” will no be able to download the full torrent until those “pieces” become available again.

As previously mentioned:

  • 5. Download and Upload content “Pieces” from/to other Peers
  • 6. Reassemble the original file by assembling all downloaded pieces together
  • 7. Become a Seed
    • Keep seeding the contents
    • seeding all “pieces”

Download Examples (with uTorrent)

 

Screen Shot 10-23-16 at 12.27 PM.PNG

Download status for a specific torrent/file

 

screen-shot-10-21-16-at-05-07-pm

Peers in the swarm

screen-shot-10-21-16-at-05-06-pm

Peers in the swarm

screen-shot-10-19-16-at-12-21-pm

Fully downloaded torrent/file

screen-shot-10-19-16-at-12-18-pm

Peers in the swarm with decoded countries

screen-shot-10-19-16-at-12-20-pm

Fully downloaded torrent/file in “Seeding” status

 

Final remarks

BitTorrent is a very powerful and popular free speech tool.

I will be describing more protocol details in future posts.

References