|
|
| Line 1: |
Line 1: |
| '''Présentation : '''<br /><br />
| |
| '''BitTorrent ''' est un [http://fr.wikipedia.org/wiki/Protocole_réseau protocole réseau] de [http://fr.wikipedia.org/wiki/Partage_de_fichiers_en_pair_à_pair partage de fichier en pair-à-pairs] (P2P) créé par '''Bram Cohen''' en avril 2001, qui est utilisé pour la distribution de grande quantité de données. BitTorrent est l'un des protocoles le plus utilisé pour le téléchargement des fichiers de grande taille. En février 2009, des observatoires ont estimés que le protocole représentait entre 27 et 55% du [http://fr.wikipedia.org/wiki/Trafic_Internet trafic internet] selon le lieu géographique.
| |
|
| |
|
| ==Description==
| |
| Le protocole BitTorrent permet de distribuer de large fichier sans alourdir la charge processeur de la source, ni la charge du réseau. Il repose sur une architecture réseau semi-centralisé. Pour tenter d'assurer de meilleure vitesse de transferts, BitTorrent se base sur une limitation du nombre d'utilisateurs vers lesquels sont envoyé les données tout en favorisant l'envoi de données aux utilisateurs depuis lesquels on en reçois le plus. <br />
| |
| Plutôt que de télécharger depuis une source unique, l'utilisateur doit rejoindre un "essaim" d'hôtes, pour à son tour télécharger et envoyer simultanément vers les autres hôtes qui constituent cet essaim.
| |
| Le protocole BitTorrent est une méthode alternative à l'ancienne source unique, aux serveurs miroir de distribution de données et peut fonctionner sur des réseaux dont la bande passante est réduite, ainsi que sur de petits ordinateurs, tel que des téléphones mobile sont en mesure de distribuer efficacement des fichiers à plusieurs destinataires.
| |
| Un utilisateur qui souhaite diffuser un fichier doit tout d'abord créer un petit fichier de description nommé "[[Fichier Torrent|*.torrent]]", qu'il distribue ensuite de façon conventionnel (via internet, email, etc...).
| |
|
| |
| L'utilisateur peut ainsi rendre le fichier disponible via un nœud ([[Terminologie de BitTorrent#tracker|tracker]]) BitTorrent agissant comme une ''[[Terminologie de BitTorrent#seed|semence]]'' ou source ('''seed''' en anglais). Ainsi ceux qui possèdent le descripteur de fichier ([[Fichier Torrent|*.torrent]]) peuvent distribuer via le tracker BitTorrent, en agissant en tant que ''[[Terminologie de BitTorrent#Peer|pairs (seeds)]]''ou''[[Terminologie de BitTorrent#Leech | leechers]]''.
| |
|
| |
| The file being distributed is divided into segments called ''pieces''. As each peer receives a new piece of the file it becomes a source of that piece to other peers, relieving the seed from having to send a copy to every peer. With BitTorrent, the task of distributing the file is shared by those who want it; it is entirely possible for the seed to send only a single copy of the file itself to an unlimited number of peers.
| |
|
| |
| Each piece is protected by a [[cryptographic hash]] contained in the torrent descriptor.<ref>{{cite web |url=http://www.bittorrent.org/beps/bep_0003.html |title=The BitTorrent Protocol Specification |author=Bram Cohen |date=10-Jan-2008 |work=BitTorrent.org |accessdate=20 November 2010}}</ref> This prevents nodes from maliciously modifying the pieces they pass on to other nodes. If a node starts with an authentic copy of the torrent descriptor, it can verify the authenticity of the actual file it has received.
| |
|
| |
| When a peer completely downloads a file, it becomes an additional seed. This eventual shift from peers to seeders determines the overall "health" of the file (as determined by the number of times a file is available in its complete form).
| |
|
| |
| This distributed nature of BitTorrent leads to a [[Flooding algorithm|flood like]] spreading of a file throughout peers. As more peers join the swarm, the likelihood of a successful download increases. Relative to standard Internet hosting, this provides a significant reduction in the original distributor's [[hardware]] and bandwidth resource costs. It also provides [[wikt:redundancy|redundancy]] against system problems, reduces dependence on the original distributor<ref>[http://arxiv4.library.cornell.edu/abs/1004.0395v2 Estimating Self-Sustainability in Peer-to-Peer Swarming Systems] by D. Menasche, A. Rocha, E. de Souza e Silva, R. M. Leao, D. Towsley, A. Venkataramani</ref> and provides a source for the file which is generally temporary and therefore harder to trace than when provided by the enduring availability of a host in standard file distribution techniques.
| |
|
| |
| ==Operation==
| |
| [[Image:Torrentcomp small.gif|thumb|600px|alt=Animation showing 7 remote computers exchanging data with an 8th (local) computer over a network. |In this animation, the colored bars beneath all of the 7 clients in the upper region above represent the file, with each color representing an individual piece of the file. After the initial pieces transfer from the seed (large system at the bottom), the pieces are individually transferred from client to client. The original ''seeder'' only needs to send out one copy of the file for all the clients to receive a copy.]]
| |
|
| |
| A BitTorrent client is any program that implements the BitTorrent protocol. Each client is capable of preparing, requesting, and transmitting any type of [[computer file]] over a network, using the protocol. A peer is any computer running an instance of a client.
| |
|
| |
| To share a file or group of files, a peer first creates a small file called a "[[Torrent file|torrent]]" (e.g. MyFile.torrent). This file contains [[metadata]] about the files to be shared and about the [[BitTorrent tracker|tracker]], the computer that coordinates the file distribution. Peers that want to download the file must first obtain a torrent file for it and connect to the specified tracker, which tells them from which other peers to download the pieces of the file.
| |
|
| |
| Though both ultimately transfer files over a network, a BitTorrent download differs from a classic download (as is typical with an [[Hypertext Transfer Protocol|HTTP]] or [[File Transfer Protocol|FTP]] request, for example) in several fundamental ways:
| |
|
| |
| * BitTorrent makes many small data requests over different [[Transmission Control Protocol|TCP]] connections to different machines, while classic downloading is typically made via a single TCP connection to a single machine.
| |
| * BitTorrent downloads in a random or in a "rarest-first"<ref name="Rarest First and Choke Algorithms Are Enough">{{cite web| url=http://www.imconf.net/imc-2006/papers/p20-legout.pdf| title=Rarest First and Choke Algorithms Are Enough | month=December | year=2006| last=Urvoy-Keller| publisher=imconf.net | accessdate=2007-11-10|format=PDF}}</ref> approach that ensures high availability, while classic downloads are sequential.
| |
|
| |
| Taken together, these differences allow BitTorrent to achieve much lower cost to the content provider, much higher redundancy, and much greater resistance to abuse or to "[[Slashdot effect|flash crowds]]" than regular [[Server (computing)|server software]]. However, this protection, theoretically, comes at a cost: downloads can take time to rise to full speed because it may take time for enough peer connections to be established, and it may take time for a node to receive sufficient data to become an effective uploader. This contrasts with regular downloads (such as from an HTTP server, for example) that, while more vulnerable to overload and abuse, rise to full speed very quickly and maintain this speed throughout.
| |
|
| |
| In general, BitTorrent's non-contiguous download methods have prevented it from supporting "progressive downloads" or "streaming playback". However, [http://torrentfreak.com/interview-with-bram-cohen-the-inventor-of-bittorrent comments made by Bram Cohen in January 2007] suggest that streaming torrent downloads will soon be commonplace and [http://torrentfreak.com/bittorrent-launches-ad-supported-streaming-071218 ad supported streaming] appears to be the result of those comments. In January 2011 Cohen demonstrated an early version of BitTorrent streaming, saying the feature will be available by summer 2011.<ref name="Rarest First and Choke Algorithms Are Enough">{{cite web| url=http://torrentfreak.com/bittorrent-p2p-live-streaming-110119/| title=BitTorrent Inventor Demos New P2P Live Streaming Protocol | month=January | year=2011 | first=Bram | last=Cohen | publisher=torrentfreak.com | accessdate=2011-01-19}}</ref>
| |
|
| |
| ===Creating and publishing torrents===
| |
| The peer distributing a data file treats the file as a number of identically sized pieces, usually with byte sizes of a power of 2, and typically between 32 kB and 16 MB each. The peer creates a [[Hash Function|hash]] for each piece, using the [[SHA-1]] hash function, and records it in the torrent file. Pieces with sizes greater than 512 kB will reduce the size of a torrent file for a very large payload, but is claimed to reduce the efficiency of the protocol.<ref>[http://wiki.theory.org/index.php/BitTorrentSpecification Theory.org]</ref> <!-- Wikis are not citable sources — find a better source -Lexein --> When another peer later receives a particular piece, the hash of the piece is compared to the recorded hash to test that the piece is error-free.<ref name = "Protocol1.0" /> Peers that provide a complete file are called seeders, and the peer providing the initial copy is called the initial seeder.
| |
|
| |
| The exact information contained in the torrent file depends on the version of the BitTorrent protocol. By convention, the name of a torrent file has the suffix <code>.torrent</code>. Torrent files have an "announce" section, which specifies the [[Uniform Resource Locator|URL]] of the tracker, and an "info" section, containing (suggested) names for the files, their lengths, the piece length used, and a [[SHA-1]] [[hash code]] for each piece, all of which are used by clients to verify the integrity of the data they receive.
| |
|
| |
| Torrent files are typically published on websites or elsewhere, and registered with at least one tracker. The tracker maintains lists of the clients currently participating in the torrent.<ref name="Protocol1.0">{{cite web| url=http://www.bittorrent.org/beps/bep_0003.html| title=BitTorrent Protocol 1.0 | month=October | year=2002| first=Bram| last=Cohen| publisher=BitTorrent.org | accessdate=2008-10-27}}</ref> Alternatively, in a ''trackerless system'' (decentralized tracking) every peer acts as a tracker. Azureus was the first{{Citation needed|date=November 2009}} BitTorrent client to implement such a system through the [[distributed hash table]] (DHT) method. An alternative and incompatible DHT system, known as Mainline DHT, was later developed and adopted by the [[BitTorrent (software)|BitTorrent]] (Mainline), [[µTorrent]], [[Transmission (BitTorrent client)|Transmission]], [[rTorrent]], [[KTorrent]], [[BitComet]], and [[Deluge (BitTorrent client)|Deluge]] clients.
| |
|
| |
| After the DHT was adopted, a "private" flag — analogous to the [[broadcast flag]] — was unofficially introduced, telling clients to restrict the use of decentralized tracking regardless of the user's desires.<ref>{{cite web| url=http://wiki.theory.org/BitTorrentSpecification#Info_Dictionary| title=Unofficial BitTorrent Protocol Specification v1.0| accessdate=2009-10-04}}</ref> The flag is intentionally placed in the info section of the torrent so that it cannot be disabled or removed without changing the identity of the torrent. The purpose of the flag is to prevent torrents from being shared with clients that do not have access to the tracker. The flag was requested for inclusion in the official specification in August, 2008, but has not been accepted.<ref>{{cite web| url=http://bittorrent.org/beps/bep_0027.html| publisher= Bittorrent.org| title=Private Torrents| accessdate=2009-10-04}}</ref> Clients that have ignored the private flag were banned by many trackers, discouraging the practice.<ref>{{cite web| url=http://www.slyck.com/news.php?story=1021| title=BitComet Banned From Growing Number of Private Trackers| accessdate=2009-10-04}}</ref>
| |
|
| |
| === Downloading torrents and sharing files ===
| |
| Users browse the web to find a torrent of interest, download it, and open it with a BitTorrent client. The client connects to the tracker(s) specified in the torrent file, from which it receives a list of peers currently transferring pieces of the file(s) specified in the torrent. The client connects to those peers to obtain the various pieces. If the swarm contains only the initial seeder, the client connects directly to it and begins to request pieces.
| |
|
| |
| Clients incorporate mechanisms to optimize their download and upload rates; for example they download pieces in a random order to increase the opportunity to exchange data, which is only possible if two peers have different pieces of the file.
| |
|
| |
| The effectiveness of this data exchange depends largely on the policies that clients use to determine to whom to send data. Clients may prefer to send data to peers that send data back to them (a [[tit for tat]] scheme), which encourages fair trading. But strict policies often result in suboptimal situations, such as when newly joined peers are unable to receive any data because they don't have any pieces yet to trade themselves or when two peers with a good connection between them do not exchange data simply because neither of them takes the initiative. To counter these effects, the official BitTorrent client program uses a mechanism called "optimistic unchoking", whereby the client reserves a portion of its available bandwidth for sending pieces to random peers (not necessarily known good partners, so called preferred peers) in hopes of discovering even better partners and to ensure that newcomers get a chance to join the swarm.<ref name = "Tamilmanistudy" >{{cite web| url=http://mnl.cs.stonybrook.edu/home/karthik/BitTorrent/Robustness_of_BT.doc | title=Studying and enhancing the BitTorrent protocol | first=Karthik | last=Tamilmani | coauthors=| publisher=Stony Brook University| date=2003-10-25| accessdate=2006-05-06|format=DOC |archiveurl = http://web.archive.org/web/20041119150847/http://mnl.cs.stonybrook.edu/home/karthik/BitTorrent/Robustness_of_BT.doc |archivedate = 2004-11-19}}</ref><!-- mnl.cs.stonybrook.edu down, subst archive.org link -->
| |
|
| |
| Although swarming scales well to tolerate flash crowds for popular content, it is less useful for unpopular content. Peers arriving after the initial rush might find the content unavailable and need to wait for the arrival of a seed in order to complete their downloads. The seed arrival, in turn, may take long to happen (this is termed the seeder promotion problem). Since maintaining seeds for unpopular content entails high bandwidth and administrative costs, this runs counter to the goals of publishers that value BitTorrent as a cheap alternative to a client-server approach. This occurs on a huge scale; measurements have shown that 38% of all new torrents become unavailable within the first month.<ref>[http://eprints.comp.lancs.ac.uk/2281/1/P2P10.pdf Unraveling BitTorrent's File Unavailability:Measurements and Analysis] by Sebastian Kaune, Ruben Cuevas Rumin, Gareth Tyson, Andreas Mauthe, Ralf Steinmetz</ref> A strategy adopted by many publishers which significantly increases availability of unpopular content consists of [[bundling]] multiple files in a single swarm.<ref>[http://conferences.sigcomm.org/co-next/2009/papers/Menasche.pdf Content Availability and Bundling in Swarming Systems] by D. Menasche, A. Rocha, B. Li, D. Towsley, A. Venkataramani</ref> More sophisticated solutions have also been proposed; generally, these use cross-torrent mechanisms through which multiple torrents can cooperate to better share content.<ref>[http://www.dcs.kcl.ac.uk/staff/tysong/files/ICCCN09.pdf The Seeder Promotion Problem: Measurements, Analysis and Solution Space] by Sebastian Kaune, Gareth Tyson, Konstantin Pussep, Andreas Mauthe, Aleksandra Kovacevic and Ralf Steinmetz</ref>
| |
|
| |
| BitTorrent does not offer its users anonymity. It is possible to obtain the [[IP address]]es of all current and possibly previous participants in a swarm from the tracker. This may expose users with insecure systems to attacks.<ref name = "Tamilmanistudy" /> It may also expose users to the risk of being sued, if they are distributing files without permission from the copyright holder(s). However, there are ways to promote anonymity; for example, the [[OneSwarm]] project layers privacy-preserving sharing mechanisms on top of the original BitTorrent protocol.
| |
|
| |
| == Adoption ==
| |
| A growing number of individuals and organizations are using BitTorrent to distribute their own or licensed material. Independent adopters report that without using BitTorrent technology and its dramatically reduced demands on their private networking hardware and bandwidth, they could not afford to distribute their files.<ref>See, for example, [http://tasvideos.org/WhyBitTorrent.html Why Bit Torrent] at http://tasvideos.org tasvideos.org.</ref>
| |
|
| |
| === Film, video and music ===
| |
| *[[BitTorrent Inc.]] has amassed a number of licenses from Hollywood studios for distributing popular content from their websites.
| |
| *[[Sub Pop]] Records releases tracks and videos via BitTorrent Inc.<ref>{{cite web| url=http://www.bittorrent.com/users/subpoprecords/| title=Sub Pop page on BitTorrent.com| accessdate=2006-12-13}}</ref> to distribute its 1000+ albums. [[Babyshambles]] and [[The Libertines]] (both bands associated with [[Pete Doherty]]) have extensively used torrents to distribute hundreds of demos and live videos. US [[industrial rock]] band [[Nine Inch Nails]] frequently distributes albums via BitTorrent.
| |
| *[[Podcast]]ing software is starting to integrate BitTorrent to help podcasters deal with the download demands of their [[MP3]] "radio" programs. Specifically, [[Juice (podcasting)|Juice]] and [[Miro Media Player|Miro]] (formerly known as Democracy Player) support automatic processing of .torrent files from [[RSS]] feeds. Similarly, some BitTorrent clients, such as [[µTorrent]], are able to process [[web feeds]] and automatically download content found within them.
| |
| *[[DGM Live!]] purchases are provided via BitTorrent.<ref>[http://www.dgmlive.com/help.htm#whatisbittorrent DGMlive.com]</ref>
| |
|
| |
| === Broadcasters ===
| |
| *In 2008, the [[Canadian Broadcasting Corporation|CBC]] became the first public broadcaster in North America to make a full show (''[[Canada's Next Great Prime Minister]]'') available for download using BitTorrent.<ref>{{cite news
| |
| | title = CBC to BitTorrent Canada's Next Great Prime Minister:
| |
| | date = 19 March 2008
| |
| | url = http://www.cbc.ca/nextprimeminister/blog/2008/03/canadas_next_great_prime_minis.html
| |
| | accessdate = 2008-03-19
| |
| | work=CBC News}}</ref>
| |
| *The [[Norwegian Broadcasting Corporation]] (NRK) has since March 2008 experimented with bittorrent distribution, available online.<ref>[http://nrkbeta.no/bittorrent/ Bittorrent<!-- Bot generated title -->]</ref> Only selected material in which NRK owns all royalties are published. Responses have been very positive, and NRK is planning to offer more content.
| |
| * The Dutch [[VPRO]] broadcasting organization released three documentaries under a [[Creative Commons]] license using the content distribution feature of the [[Mininova]] tracker.
| |
|
| |
| === Personal material ===
| |
| *The [[Amazon S3]] "Simple Storage Service" is a scalable Internet-based storage service with a simple [[web service]] interface, equipped with built-in BitTorrent support.
| |
| *[[Blog Torrent]] offers a simplified BitTorrent tracker to enable [[bloggers]] and non-technical users to host a tracker on their site. Blog Torrent also allows visitors to download a "stub" loader, which acts as a BitTorrent client to download the desired file, allowing users without BitTorrent software to use the protocol.<ref>{{cite web| url=http://grep.law.harvard.edu/article.pl?sid=04/08/26/0236209| title=Blog Torrent and Participatory Culture|date=26 August 2004| first=Roger E. |last=Rustad|publisher=Grep Law| accessdate=2006-05-09}}</ref> This is similar to the concept of a [[self-extracting archive]].
| |
|
| |
| === Software ===
| |
| *[[Blizzard Entertainment]] uses BitTorrent (via a proprietary client called the "Blizzard Downloader") to distribute most content for [[StarCraft II]] and [[World of Warcraft]], including the games themselves.<ref>{{cite web | title = Blizzard Downloader | publisher = Curse Inc. | date = 4 November 2010 | url = http://www.wowpedia.org/Blizzard_Downloader | accessdate = 2010-11-04 }}</ref>
| |
| *Many software games, especially those whose large size makes them difficult to host due to bandwidth limits, extremely frequent downloads, and unpredictable changes in network traffic, will distribute instead a specialized, stripped down bittorrent client with enough functionality to download the game from the other running clients and the primary server (which is maintained in case not enough peers are available).
| |
| *Many major [[open source]] and [[free software]] projects encourage BitTorrent as well as conventional downloads of their products (via [[HTTP]], [[FTP]] etc.) to increase availability and to reduce load on their own servers, especially when dealing with larger files.<ref>{{cite web | title = Complete Download Options List – BitTorrent | url = http://www.ubuntu.com/getubuntu/downloadmirrors#bt | accessdate = 2009-05-07 }}</ref>
| |
|
| |
| ===Government===
| |
| *The UK government used BitTorrent to distribute details about how the tax money of UK citizens was spent.<ref>[http://data.gov.uk/dataset/coins Combined Online Information System | data.gov.uk<!-- Bot generated title -->]</ref><ref>{{cite web | url=http://torrentfreak.com/uk-government-uses-bittorrent-to-share-public-spending-data-100604/ | title=UK Government Uses BitTorrent to Share Public Spending Data}}</ref>
| |
|
| |
| ===Others===
| |
| *[[Facebook]] uses BitTorrent to distribute updates to Facebook servers.<ref>{{cite web | url=http://torrentfreak.com/facebook-uses-bittorrent-and-they-love-it-100625/ | title=Facebook Uses Bittorrent And They Love It}}</ref>
| |
| *[[Twitter]] uses BitTorrent to distribute updates to Twitter servers.<ref>{{cite web | url=http://torrentfreak.com/twitter-uses-bittorrent-for-server-deployment-100210/ | title=Twitter Uses BitTorrent for Server Deployment}}</ref><ref>{{cite web | url=http://torrentfreak.com/bittorrent-makes-twitters-server-deployment-75-faster-100716/ | title=BitTorrent Makes Twitters Server Deployment 75 Faster}}</ref>
| |
|
| |
| == Network impact ==
| |
| {{update|type=section|date=November 2010}}
| |
| [[CableLabs]], the research organization of the North American cable industry, estimates that BitTorrent represents 18% of all broadband traffic.<!-- this reference should be replaced by the original paper by Terry Shaw and Jim Martin, which I couldn't locate --><ref name="Ellis">{{cite web| url=http://www.multichannel.com/article/CA6332098.html| date=8 May 2006| title=BitTorrent's Swarms Have a Deadly Bite On Broadband Nets| first=Leslie| last=Ellis| publisher=Multichannel News| accessdate=2006-05-08}}</ref>{{Update after|2010|11}} In 2004, CacheLogic put that number at roughly 35% of all traffic on the Internet.<ref name="Pasick">{{cite web
| |
| |url=http://www.interesting-people.org/archives/interesting-people/200411/msg00078.html
| |
| |date=4 November 2004| title=LiveWire — File-sharing network thrives beneath the radar
| |
| |first=Adam
| |
| |last=Pasick
| |
| |publisher=Yahoo! News
| |
| |accessdate=2006-05-09}}</ref>{{Update after|2010|11}} The discrepancies in these numbers are caused by differences in the method used to measure P2P traffic on the Internet.<ref name="Sevcik">{{cite web| url=http://www.bcr.com/opinion/next_generation_networks/peer-to-peer_traffic_internet_myth_born_20051101647.htm| title= Peer-to-Peer Traffic: Another Internet Myth Is Born| first=Peter| last=Sevcik| date=1 November 2005| publisher=Business Communication Review| accessdate=2006-05-09 |archiveurl = http://web.archive.org/web/20060410005017/http://www.bcr.com/opinion/next_generation_networks/peer-to-peer_traffic_internet_myth_born_20051101647.htm <!-- Bot retrieved archive --> |archivedate = 2006-04-10}}</ref>
| |
|
| |
| Routers that use [[network address translation]] (NAT) must maintain tables of source and destination IP addresses and ports. Typical home routers are limited to about 2000 table entries while some more expensive routers have larger table capacities. BitTorrent frequently contacts 300–500 servers per second rapidly filling the NAT tables. This is a common cause of home routers locking up.<ref>{{cite web|title=uTorrent's FAQ page|url=http://www.utorrent.com/faq.php#Modems_routers_that_are_known_to_have_problems_with_P2P}}</ref>
| |
|
| |
| ==Indexing==
| |
| The BitTorrent protocol provides no way to index torrent files. As a result, a comparatively small number of websites have hosted a large majority of torrents, many linking to copyrighted material without the authorization of copyright holders, rendering those sites especially vulnerable to lawsuits.<ref>{{cite web| url=http://torrentfreak.com/publicbt-tracker-set-to-patch-bittorrents-achilles-heel-090712/| title=PublicBT Tracker Set To Patch BitTorrent's Achilles' Heel| date=12 July 2009| accessdate=14 July 2009}}</ref> Several types of websites support the discovery and distribution of data on the BitTorrent network.
| |
|
| |
| Public torrent hosting sites such as [[The Pirate Bay]] allow users to search and download from their collection of torrent files. Users can typically also upload torrent files for content they wish to distribute. Often, these sites also run [[BitTorrent tracker]]s for their hosted torrent files, but these two functions are not mutually dependent: a torrent file could be hosted on one site and tracked by another, unrelated site.
| |
|
| |
| Private host/tracker sites operate like public ones except that they restrict access to registered users and keep track of the amount of data each user uploads and downloads, in an attempt to reduce [[leech (computing)|leeching]].
| |
|
| |
| [[Search engine]]s allow the discovery of torrent files that are hosted and tracked on other sites; examples include [[Mininova]], [[BTJunkie]], [[Torrentz]], [[The Pirate Bay]], [[Eztorrent]] and [[isoHunt]]. These sites allow the user to ask for content meeting specific criteria (such as containing a given word or phrase) and retrieve a list of links to torrent files matching those criteria. This list can often be sorted with respect to several criteria, relevance (seeders-leechers ratio) being one of the most popular and useful (due to the way the protocol behaves, the download bandwidth achievable is very sensitive to this value). Bram Cohen launched a BitTorrent search engine on http://www.bittorrent.com/search that co-mingles licensed content with search results.<ref>{{cite web| url=http://www.betanews.com/article/BitTorrent_Creator_Opens_Online_Search/1117065427| title=BitTorrent Creator Opens Online Search| first=David| last=Worthington| coauthors=Nate Mook| publisher=BetaNews| date=25 May 2015| accessdate=2006-05-09}}</ref> [[Metasearch engine]]s allow one to search several BitTorrent indices and search engines at once.
| |
|
| |
| == Technologies built on BitTorrent ==
| |
| The BitTorrent protocol is still under development and therefore may still acquire new features and other enhancements such as improved efficiency.
| |
|
| |
| === Distributed trackers ===
| |
| On May 2, 2005, Azureus 2.3.0.0 (now known as [[Vuze]]) was released,<ref>{{cite web|title=Vuze Changelog|url=http://azureus.sourceforge.net/changelog.php|publisher=Azureus.sourceforge.net}}</ref> introducing support for "trackerless" torrents through a system called the "distributed database." This system is a [[Distributed hash table|DHT]] implementation which allows the client to use torrents that do not have a working [[BitTorrent tracker]]. The following month, [[BitTorrent, Inc.]] released version 4.2.0 of the Mainline BitTorrent client, which supported an alternative DHT implementation (popularly known as "Mainline DHT") that is incompatible with that of Azureus. Current versions of the official [[BitTorrent client]], [[µTorrent]], [[BitComet]], and BitSpirit all share compatibility with Mainline DHT. Both DHT implementations are based on [[Kademlia]].<ref>[http://khashmir.sourceforge.net/ Khashmir.Sourceforge.net]</ref> As of version 3.0.5.0, Azureus also supports Mainline DHT in addition to its own distributed database through use of an optional application plugin.<ref>[http://azureus.sourceforge.net/plugin_details.php?plugin=mlDHT Azureus.sourceforge.net]</ref> This potentially allows the Azureus client to reach a bigger swarm.
| |
|
| |
| Another idea that has surfaced in Vuze is that of ''virtual torrents''. This idea is based on the distributed tracker approach and is used to describe some web resource. Currently, it is used for [[instant messaging]]. It is implemented using a special messaging protocol and requires an appropriate plugin. [[Anatomic P2P]] is another approach, which uses a decentralized network of nodes that route traffic to dynamic trackers.
| |
|
| |
| Most BitTorrent clients also use [[Peer exchange]] (PEX) to gather peers in addition to [[BitTorrent tracker|trackers]] and [[Distributed hash table|DHT]]. Peer exchange checks with known peers to see if they know of any other peers. With the 3.0.5.0 release of Vuze, all major BitTorrent clients now have compatible peer exchange.
| |
|
| |
| === Web seeding ===
| |
| Web seeding was implemented in 2006 as the ability of BitTorrent clients to download torrent pieces from an HTTP source in addition to the swarm. The advantage of this feature is that a website may distribute a torrent for a particular file or batch of files and make those files available for download from that same web server; this can simplify long-term seeding and [[Load balancing (computing)|load balancing]] through the use of existing, cheap, web hosting setups. In theory, this would make using BitTorrent almost as easy for a web publisher as creating a direct HTTP download. In addition, it would allow the "web seed" to be disabled if the swarm becomes too popular while still allowing the file to be readily available.
| |
|
| |
| This feature has two distinct and incompatible specifications.
| |
|
| |
| The first was created by John "TheSHAD0W" Hoffman, who created [[BitTornado]].<ref>{{cite web| url=http://bittornado.com/docs/webseed-spec.txt| title=HTTP-Based Seeding Specification| format=TXT| accessdate=2006-05-09}}</ref> From version 5.0 onward, the Mainline BitTorrent client also supports web seeds, and the BitTorrent web site had<ref>{{cite web| url=http://www.bittorrent.com/btusers/nowplaying/?| title=The Torrent Entertainment Network has closed}}</ref> a simple publishing tool that creates web seeded torrents.<ref>{{cite web| url=http://www.bittorrent.com/publish | title=Publish – BitTorrent |archiveurl = http://web.archive.org/web/20070526065412/http://www.bittorrent.com/publish |archivedate = 2007-05-26}} (archived page from May 26, 2007, web.archive.org)</ref> [[µTorrent]] added support for web seeds in version 1.7. [[BitComet]] added support for web seeds in version 1.14. This first specification requires running a web service that serves content by info-hash and piece number, rather than filename.
| |
|
| |
| The other specification is created by [[GetRight]] authors and can rely on a basic HTTP download space (using [[byte serving]]).<ref>{{cite web| url=http://www.getright.com/seedtorrent.html|title=HTTP/FTP Seeding for BitTorrent| accessdate=2010-03-18}}</ref>
| |
|
| |
| In September 2010, a new service named Burnbit was launched which generates a torrent from any URL using webseeding. <ref name="tf-article">[http://torrentfreak.com/burn-any-web-hosted-file-into-a-torrent-with-burnbit-100913/]</ref>
| |
|
| |
| There exist server-side solutions that provide initial seeding of the file from the webserver via standard Bittorrent protocol and when the number of external seeders reach a limit, they stop serving the file from the original source. <ref name="web-seeders">[http://php-tracker.org/]</ref>
| |
|
| |
| === RSS feeds ===
| |
| {{Main|Broadcatching}}
| |
|
| |
| A technique called [[Broadcatching]] combines [[RSS (file format)|RSS]] with the BitTorrent protocol to create a content delivery system, further simplifying and automating content distribution. [[Steve Gillmor]] explained the concept in a column for [[Ziff-Davis]] in December, 2003.<ref>Gillmore, Steve. [http://www.eweek.com/article2/0,1895,1413403,00.asp BitTorrent and RSS Create Disruptive Revolution] ''EWeek.com,'' 13 December 2003. Retrieved on 22 April 2007.</ref> The discussion spread quickly among bloggers (Ernest Miller,<ref>[http://www.corante.com/importance/ Corante.com]{{dead link|date=December 2010}}</ref> [[Chris Pirillo]], etc.). In an article entitled ''Broadcatching with BitTorrent'', Scott Raymond explained:
| |
|
| |
| {{quote|I want RSS feeds of BitTorrent files. A script would periodically check the feed for new items, and use them to start the download. Then, I could find a trusted publisher of an [[Alias (TV Show)|Alias]] RSS feed, and "subscribe" to all new episodes of the show, which would then start downloading automatically — like the "season pass" feature of the [[TiVo]].|Scott Raymond| scottraymond.net<ref>Raymond, Scott: [http://web.archive.org/web/20040213093750/http://scottraymond.net/archive/4745 Broadcatching with BitTorrent]. ''scottraymond.net'': 16 December 2003.</ref>}}
| |
|
| |
| The RSS feed will track the content, while BitTorrent ensures content integrity with [[cryptography|cryptographic]] [[Hash function|hashing]] of all data, so feed subscribers will receive uncorrupted content.
| |
|
| |
| One of the first and popular software clients ([[free software|free]] and [[open source software|open source]]) for ''broadcatching'' is [[Miro Media Player|Miro]]. Other free software clients such as [[PenguinTV]] and [[KatchTV]] are also now supporting broadcatching.
| |
|
| |
| The BitTorrent web-service MoveDigital had the ability to make torrents available to any web application capable of parsing [[XML]] through its standard [[Representational State Transfer|REST]]-based interface,<ref>{{cite web| url=http://www.movedigital.com/docs/index.php/MoveDigital_API| title=Move Digital REST API| publisher=Move Digital| accessdate=2006-05-09}} Documentation.</ref> although this has since been discontinued.<!-- Could someone cleanup and clarify the preceding sentence? I don't even understand it. --> Additionally, Torrenthut is developing a similar torrent [[API]] that will provide the same features, as well as further intuition to help bring the torrent community to [[Web 2.0]] standards. Alongside this release is a first [[PHP]] application built using the API called PEP, which will parse any [[Really Simple Syndication]] (RSS 2.0) feed and automatically create and seed a torrent for each enclosure found in that feed.<ref>{{cite web|url=http://prodigem.com/code/pep/pep.txt | title=Prodigem Enclosure Puller(pep.txt)| publisher=Prodigem.com| format=TXT| accessdate=2006-05-09 |archiveurl = http://web.archive.org/web/20060526130219/http://prodigem.com/code/pep/pep.txt |archivedate = 2006-05-26}} via [[Internet Wayback Machine]]</ref> <!-- <ref>{{cite web| url=http://wi-fizzle.com/code/pep/pep.txt| title=pep.txt| publisher=Prodigem.com| format=TXT| accessdate=2007-01-30}} Archival access.</ref> – wi-fizzle unreachable 2007-4-15 — found in Google cache — identical to archive.org copy. --Lexein -->
| |
|
| |
| === Throttling and encryption===
| |
| {{Main|BitTorrent protocol encryption}}
| |
|
| |
| Since BitTorrent makes up a large proportion of total traffic, some [[ISP]]s have chosen to throttle (slow down) BitTorrent transfers to ensure network capacity remains available for other uses. For this reason, methods have been developed to disguise BitTorrent traffic in an attempt to thwart these efforts.<ref>{{cite web| url=http://torrentfreak.com/encrypting-bittorrent-to-take-out-traffic-shapers/| title=Encrypting Bittorrent to take out traffic shapers| date=2006-02-05| publisher=Torrentfreak.com| accessdate=2006-05-09}}</ref>
| |
|
| |
| Protocol header encrypt (PHE) and [[BitTorrent protocol encryption|Message stream encryption/Protocol encryption (MSE/PE)]] are features of some BitTorrent clients that attempt to make BitTorrent hard to detect and throttle. At the moment [[Vuze]], [[Bitcomet]], [[KTorrent]], [[Transmission (BitTorrent client)|Transmission]], [[Deluge (BitTorrent client)|Deluge]], [[µTorrent]], MooPolice, Halite, [[rTorrent]] and the latest official BitTorrent client (v6) support MSE/PE encryption.
| |
|
| |
| In September 2006 it was reported that some software could detect and throttle BitTorrent traffic masquerading as [[HTTP]] traffic.<ref name="HTTP_Masquerading">{{cite web| url=http://www.studlife.com/archives/News/2006/09/27/ResTechsolvesnetworkissues/| title=ResTech solves network issues | month=September | year=2006| first=Ben| last=Sales| publisher=studlife.com }}</ref>
| |
|
| |
| Reports in August 2007 indicated that [[Comcast]] was preventing BitTorrent seeding by monitoring and interfering with the communication between peers. Protection against these efforts is provided by [[proxy server|proxying]] the client-tracker traffic via an [[tunneling protocol|encrypted tunnel]] to a point outside of the Comcast network.<ref>[http://torrentfreak.com/comcast-throttles-bittorrent-traffic-seeding-impossible/ Comcast Throttles BitTorrent Traffic, Seeding Impossible], ''TorrentFreak'', 17 August 2007</ref> Comcast has more recently called a "truce" with [[BitTorrent, Inc.]] with the intention of shaping traffic in a protocol-agnostic manner.<ref>[http://www.news.com/8301-10784_3-9904494-7.html Comcast and Bittorrent Agree to Collaborate]</ref> Questions about the ethics and legality of Comcast's behavior have led to renewed debate about [[Network neutrality|net neutrality]] in the United States.<ref>[http://www.cnet.com/8301-13739_1-9769645-46.html Is Comcast's BitTorrent filtering violating the law?]</ref>
| |
|
| |
| In general, although encryption can make it difficult to determine ''what'' is being shared, BitTorrent is vulnerable to [[traffic analysis]]. Thus even with MSE/PE, it may be possible for an ISP to recognize BitTorrent and also to determine that a system is no longer downloading but only uploading data, and terminate its connection by injecting [[Transmission Control Protocol|TCP]] RST (reset flag) packets.
| |
|
| |
| === Multitracker ===
| |
| Another unofficial feature is an extension to the BitTorrent metadata format proposed by John Hoffman<ref>{{cite web| url=http://www.bittornado.com/docs/multitracker-spec.txt| title=Multitracker Metadata Entry Specification| format=TXT| accessdate=2006-05-09| publisher=Bittornado.com}}</ref> and implemented by several indexing websites. It allows the use of multiple trackers per file, so if one tracker fails, others can continue to support file transfer. It is implemented in several clients, such as [[BitComet]], [[BitTornado]], [[BitTorrent (software)|BitTorrent]], [[KTorrent]], [[Transmission (BitTorrent client)|Transmission]], [[Deluge (software)|Deluge]], [[µTorrent]], [[rtorrent]], and [[Vuze]]. Trackers are placed in groups, or tiers, with a tracker randomly chosen from the top tier and tried, moving to the next tier if all the trackers in the top tier fail.
| |
|
| |
| Torrents with multiple trackers<ref>Called MultiTorrents by indexing website [http://www.mybittorrent.com/ myBittorrent.com]</ref> can decrease the time it takes to download a file, but also has a few consequences:
| |
|
| |
| * Poorly implemented<ref>{{cite web| url=http://wiki.depthstrike.com/index.php/P2P:Protocol:Specifications:Multitracker#Bad_Implementations| title=P2P:Protocol:Specifications:Multitracker| accessdate=2009-11-13| publisher=wiki.depthstrike.com}}</ref> clients may contact multiple trackers, leading to more overhead-traffic.
| |
| * Torrents from closed trackers suddenly become downloadable by non-members, as they can connect to a seed via an open tracker.
| |
|
| |
| === Decentralized keyword search ===
| |
| Even with distributed trackers, a third party is still required to find a specific torrent. This is usually done in the form of a hyperlink from the website of the content owner or through indexing websites like [[The Pirate Bay]] or [[Torrentz]].
| |
|
| |
| The [[Tribler]] BitTorrent client is the first to incorporate decentralized search capabilities. With Tribler, users can find .torrent files that are hosted among other peers, instead of on a centralized index sites. It adds such an ability to the BitTorrent protocol using a [[gossip protocol]], somewhat similar to the [[eXeem]] network which was shut down in 2005. The software includes the ability to recommend content as well. After a dozen downloads the Tribler software can roughly estimate the download taste of the user and recommend additional content.<ref>[https://www.tribler.org/DecentralizedRecommendation DecentralizedRecommendation – tribler.org<!-- Bot generated title -->]</ref>
| |
|
| |
| In May 2007 [[Cornell University]] published a paper proposing a new approach to searching a peer-to-peer network for inexact strings,<ref>
| |
| {{cite web
| |
| |url=http://www.cs.cornell.edu/People/egs/papers/hyperspaces.pdf
| |
| |title=Hyperspaces for Object Clustering and Approximate Matching in Peer-to-Peer Overlays
| |
| |format=PDF
| |
| |accessdate=2008-05-26
| |
| |publisher=Cornell University}}</ref> which could replace the functionality of a central indexing site. A year later, the same team implemented the system as a plugin for [[Vuze]] called Cubit<ref>
| |
| {{cite web
| |
| |url=http://www.cs.cornell.edu/~bwong/cubit/index.html
| |
| |title=Cubit: Approximate Matching for Peer-to-Peer Overlays
| |
| |accessdate=2008-05-26
| |
| |publisher=Cornell University}}</ref> and published a follow-up paper reporting its success.<ref>
| |
| {{cite web
| |
| |url=http://www.cs.cornell.edu/~bwong/cubit/tr-cubit.pdf
| |
| |title=Approximate Matching for Peer-to-Peer Overlays with Cubit
| |
| |format=PDF
| |
| |accessdate=2008-05-26
| |
| |publisher=Cornell University}}</ref>
| |
|
| |
| A somewhat similar facility but with a slightly different approach is provided by the [[BitComet]] client through its "Torrent Exchange"<ref>[http://wiki.bitcomet.com/Torrent_Exchange Torrent Exchange]. The torrent sharing feature of BitComet. Retrieved 2010-01-31.</ref> feature. Whenever two peers using BitComet (with Torrent Exchange enabled) connect to each other they exchange lists of all the torrents (name and info-hash) they have in the Torrent Share storage (torrent files which were previously downloaded and for which the user chose to enable sharing by Torrent Exchange).
| |
|
| |
| Thus each client builds up a list of all the torrents shared by the peers it connected to in the current session (or it can even maintain the list between sessions if instructed). At any time the user can search into that Torrent Collection list for a certain torrent and sort the list by categories. When the user chooses to download a torrent from that list, the .torrent file is automatically searched for (by info-hash value) in the [[Distributed hash table|DHT Network]] and when found it is downloaded by the querying client which can after that create and initiate a downloading task.
| |
|
| |
| == Implementations ==
| |
| {{Main|Comparison of BitTorrent clients}}
| |
|
| |
| The BitTorrent specification is free to use and many clients are [[open source]], so BitTorrent clients have been created for all common [[operating systems]] using a variety of [[programming languages]]. The [[BitTorrent (software)|official BitTorrent client]], [[µTorrent]], [[Vuze]], [[Transmission (BitTorrent client)|Transmission]], and [[BitComet]] are some of the most popular clients. {{Citation needed|date=March 2009}}
| |
|
| |
| Some BitTorrent implementations such as [[MLDonkey]] and [[Torrentflux]] are designed to run as servers. For example, this can be used to centralize file sharing on a single dedicated server which users share access to on the network.<ref>[http://www.turnkeylinux.org/torrentserver Torrent Server combines a file server with P2P file sharing]</ref> Server-oriented BitTorrent implementations can also be hosted by [[Internet hosting service|hosting providers]] at [[colocation centre|co-located]] facilities with high bandwidth Internet connectivity (e.g., a datacenter) which can provide dramatic speed benefits over using BitTorrent from a regular home broadband connection.
| |
|
| |
| Services such as [[ImageShack]] can download files on BitTorrent for the user, allowing them to download the entire file by [[HTTP]] once it is finished.
| |
|
| |
| The [[Opera (web browser)|Opera]] [[web browser]] supports BitTorrent,<ref>{{cite web| url=http://arstechnica.com/news.ars/post/20070201-8750.html| title=Does network neutrality mean an end to BitTorrent throttling?| first=Nate| last=Anderson| date=1 February 2007| publisher=Ars Technica, LLC| accessdate=2007-02-09}}</ref> as does [[Wyzo]]. [[BitLet]] allows users to download Torrents directly from their browser using a [[Java applet]]. Sites such as [http://xfiles.technihelp.net/ xFiles] and [http://www.dushare.com/# DuShare] allow to transfer big files directly using bittorrent inside adobe Flash.
| |
|
| |
| An increasing number of hardware devices are being made to support BitTorrent. These include routers and NAS devices containing BitTorrent-capable firmware like [[OpenWrt]].
| |
|
| |
| Proprietary versions of the protocol which implement [[Digital rights management|DRM]], encryption, and authentication are found within managed clients such as [[Pando (application)|Pando]].
| |
|
| |
| == Development ==
| |
| An unimplemented ({{As of|2008|2|lc=1}}) unofficial feature is [[Similarity Enhanced Transfer]] (SET), a technique for improving the speed at which peer-to-peer file sharing and content distribution systems can share data. SET, proposed by researchers Pucha, Andersen, and Kaminsky, works by spotting chunks of identical data in files that are an exact or near match to the one needed and transferring these data to the client if the "exact" data are not present. Their experiments suggested that SET will help greatly with less popular files, but not as much for popular data, where many peers are already downloading it.<ref>{{cite web|url=http://www.cs.cmu.edu/~dga/papers/nsdi2007-set/|title= Exploiting Similarity for Multi-Source Downloads Using File Handprints|author=Himabindu Pucha, David G. Andersen, Michael Kaminsky|month=April | year=2007|accessdate=2007-04-15|publisher=[[Purdue University]], [[Carnegie Mellon University]], Intel Research Pittsburgh}}</ref> Andersen believes that this technique could be immediately used by developers with the BitTorrent file sharing system.<ref>{{cite news|url=http://news.bbc.co.uk/2/hi/technology/6544919.stm |publisher=BBC News|title=Speed boost plan for file-sharing|date=12 April 2007|accessdate=2007-04-21}}</ref>
| |
|
| |
| {{As of|2008|12}}, BitTorrent, Inc. is working with Oversi on new Policy Discover Protocols that query the ISP for capabilities and network architecture information. Oversi's ISP hosted NetEnhancer box is designed to "improve peer selection" by helping peers find local nodes, improving download speeds while reducing the loads into and out of the ISP's network.<ref>[http://arstechnica.com/news.ars/post/20081209-bittorrent-has-new-plan-to-shape-up-p2p-behavior.html Arstechnica.com]</ref>
| |
|
| |
| == Legal issues ==
| |
| {{Main|Legal issues with BitTorrent}}
| |
| There has been much controversy over the use of BitTorrent trackers. BitTorrent metafiles themselves do not store file contents. Whether the publishers of BitTorrent metafiles violate copyrights by linking to copyrighted material without the authorization of copyright holders is controversial.
| |
|
| |
| Various jurisdictions have pursued legal action against websites that host BitTorrent trackers. High-profile examples include the closing of [[Suprnova.org]], [[Torrentspy]], [[LokiTorrent]], [[Mininova]] and [[Oink's Pink Palace|OiNK.cd]]. [[The Pirate Bay]] torrent website, formed by a Swedish group, is noted for the "legal" section of its website in which letters and replies on the subject of alleged copyright infringements are publicly displayed. On 31 May 2006, The Pirate Bay's servers in Sweden were raided by Swedish police on allegations by the MPAA of copyright infringement;<ref>{{cite web| url=http://torrentfreak.com/the-piratebay-is-down-raided-by-the-swedish-police/| year=31.05.2006| publisher=TorrentFreak| title=The Piratebay is Down: Raided by the Swedish Police| accessdate=2007-05-20}}</ref> however, the tracker was up and running again three days later.
| |
|
| |
| == BitTorrent and malware ==
| |
| Several studies on BitTorrent have indicated that a large portion of files available for download via BitTorrent contain [[malware]]. In particular, one small sample<ref>{{cite web | url=http://www.docstoc.com/docs/14960461/Searching-for-Malware-in-Bit-Torrent | title=Searching for Malware in Bit Torrent}}</ref> indicated that 18% of all executable programs available for download contained malware. Another study <ref>{{Citation | title=Where Only Fools Dare to Tread: An Empirical Study on the Prevalence of Zero-day Malware | author=Håvard Vegge, Finn Michael Halvorsen and Rune Walsø Nergård | publisher=2009 Fourth International Conference on Internet Monitoring and Protection | year=2009}}</ref> claims that as much as 14.5% of BitTorrent downloads contain [[Zero-day attack|zero-day malware]], and that BitTorrent was used as the distribution mechanism for 47% of all zero-day malware they have found.
| |
|
| |
| == See also ==
| |
| {{Portal|Free software}}
| |
| * [[BitTorrent tracker software]]
| |
| * [[Bencode]]
| |
| * [[Comparison of BitTorrent clients]]
| |
| * [[Comparison of BitTorrent sites]]
| |
| * [[FastTrack]]
| |
| * [[Kazaa]]
| |
| * [[Magnet URI scheme]]
| |
| * [[Segmented downloading]]
| |
| * [[Similarity Enhanced Transfer]]
| |
| * [[Simple file verification]]
| |
| * [[Super-seeding]]
| |
| * [[Terminology of BitTorrent]]
| |
| * [[Torrent file]]
| |
| * [[Micro Transport Protocol|µTP (Micro Transport Protocol)]]
| |
| * [[Anti-Counterfeiting Trade Agreement]]
| |
| * [[Torrent poisoning]]
| |
|
| |
| == References ==
| |
|
| |
| {{refs|colwidth=30em}}
| |