--- ChangeLog.orig Sat Jan 14 23:52:22 2006 +++ ChangeLog Sun Sep 3 15:36:13 2006 @@ -2,6 +2,27 @@ Enhanced CTorrent Change Log _________________________________________________________________ + Changes for "dnh2.1" Release + + Code Fixes + * Fixed interaction with CTCS under Linux, resulting in "Operation + now in progress" errors. + * Fixed a flaw in optimistic unchoke logic implementation. + * Fixed problem with return value in _btf_ftruncate() related to my + update for the vfat issue [btfiles patch]. + * Fixed handling of 301/302 redirect response from the tracker. + + Operational Enhancements + * Attempt to reconnect to successful peers that disconnect + unexpectedly, if possible. + * Changed the unchoke choice algorithm when seeding. The new method + is based on page 4 of the paper "Rarest First and Choke Algorithms + Are Enough" by Arnaud Legout et al. An exception is that CTorrent + chooses new unchokes using its original method (based on wait + time) rather than randomly. + See [1]the paper for details. + _________________________________________________________________ + Changes for "dnh2" Release Patches @@ -23,9 +44,10 @@ for. Optimization - * Use fewer call to random() by shifting the previously unused bits. + * Use fewer calls to random() by shifting the previously unused + bits. * Time() calls have been greatly reduced; a global timestamp - variable "now" is set once per main loop interation and referenced + variable "now" is set once per main loop iteration and referenced in functions that need a timestamp (except the caching I/O routines which were left alone). * Overall current bandwidth rates are now computed only once per @@ -150,7 +172,7 @@ parameters. * Support/display tracker warning message * Now able to handle torrents with more than 255 files. - * Support for [1]CTorrent Control Server, an application and + * Support for [2]CTorrent Control Server, an application and protocol for monitoring and managing multiple Enhanced CTorrent clients. The "-S" option is used to connect to CTCS, as in "-S localhost:2780" if CTCS is listening at port 2780 on the local @@ -239,7 +261,7 @@ Patches * Incorporates the following patches. The number is the Request ID - from the [2]SourceForge patches page, which you can reference for + from the [3]SourceForge patches page, which you can reference for the details of each patch. The name in brackets is the name of the patch file or a name I chose to refer to the patch. Some of these names are used below (in brackets) to describe a fix or change to @@ -368,12 +390,13 @@ This provides more precise DL rate measurement, and helps insure that we receive a productive amount of data (i.e. a complete slice) even if we are unchoked by a peer for only one cycle. See - [3]http://groups.yahoo.com/group/BitTorrent/message/1260 for more + [4]http://groups.yahoo.com/group/BitTorrent/message/1260 for more discussion/analysis on this. * Added -v (verbose) option for additional debugging output. References - 1. http://www.rahul.net/dholmes/ctorrent/ctcs.html - 2. http://sourceforge.net/tracker/?atid=598034&group_id=91688&func=browse - 3. http://groups.yahoo.com/group/BitTorrent/message/1260 + 1. http://www.eurecom.fr/~michiard/pubs/bt_experiments_techRepINRIA-00001111_VERSION1_13FEBRUARY2006.pdf + 2. http://www.rahul.net/dholmes/ctorrent/ctcs.html + 3. http://sourceforge.net/tracker/?atid=598034&group_id=91688&func=browse + 4. http://groups.yahoo.com/group/BitTorrent/message/1260 --- README-DNH.TXT.orig Sun Jan 15 00:44:38 2006 +++ README-DNH.TXT Sun Sep 3 15:35:15 2006 @@ -8,8 +8,8 @@ Overview [8]CTorrent is a [9]BitTorrent client implemented in C++ to be - lightweight and quick. It has fallen a little behind in updates and - bug fixes though. + lightweight and quick. The original has fallen a little behind in + updates and bug fixes though. The files here contain the good work of those who wrote the original CTorrent base code and a number of patches that provide fixes and @@ -54,9 +54,9 @@ D: Total number of peers in the swarm, as last reported by the tracker. E: Number of pieces of the torrent that you have completed. - F: Number of pieces currently available from you and your connected + F: Total number of pieces in the torrent. + G: Number of pieces currently available from you and your connected peers. - G: Total number of pieces in the torrent. H: Total amount of data you have downloaded. I: Total amount of data you have uploaded. J: Your current total download rate. @@ -89,33 +89,71 @@ News + 2006-09-03 + Version dnh2.1 is [12]released, along with [13]CTCS 1.1. Both + updates fix issues when using CTCS on Linux, among other + things. + + 2006-04-26 + The [14]CTCS protocol is finally documented. + + 2006-04-25 + A [15]patch is available to fix a bug in my solution for the + vfat filesystem issue. This bug can cause the client to crash + when creating a file on any filesystem type; the patch is + recommended for all users. + 2006-01-15 Version dnh2 is released! This version includes a number of - significant [12]changes, including large file support, piece + significant [16]changes, including large file support, piece selection, tuned request queue depth, and support for - [13]CTorrent Control Server. + [17]CTorrent Control Server. _________________________________________________________________ Changes For a list of changes in the current and previous versions, see the - [14]ChangeLog file. + [18]ChangeLog file. _________________________________________________________________ Download + Release dnh2.1 + + [19]dnh2 to dnh2.1 patch file + A patch file of changes to release dnh2 to bring it up to dnh2.1. + + [20]Source distribution + A complete source distribution for all platforms. + ___________________________________ + + Patch for vfat bug + + [21]btfiles patch + This fixes a coding bug in my solution for the vfat filesystem issue. + This bug can cause the client to crash when creating a file on any + filesystem type; the patch is recommended for all users of release + dnh2. + + To apply the patch, download the file into the ctorrent source + directory and run the command: + patch < patch-btfiles.cpp.diff + + in that directory. Then recompile the program. + ___________________________________ + Release dnh2 - The patch files for this version are significantly larger - than in previous releases. It will be faster and easier to just - download the patched source distribution below. + The patch files for this version are significantly larger than in + previous releases. It will be faster and easier to just download the + patched source distribution below. - [15]dnh1.2 to dnh2 patch file + [22]dnh1.2 to dnh2 patch file A patch file of changes to release dnh1.2 to bring it up to dnh2. - [16]Patch file + [23]Patch file A patch file of changes to the CTorrent 1.3.4 base. - [17]Patched source + [24]Patched source A complete source distribution for all platforms. ___________________________________ @@ -124,40 +162,40 @@ want to try the FreeBSD patch/version even if you are not using FreeBSD. - [18]dnh1.1 to dnh1.2 patch file + [25]dnh1.1 to dnh1.2 patch file A patch file of changes to release dnh1.1 to bring it up to dnh1.2. - [19]FreeBSD patch file + [26]FreeBSD patch file A patch file of changes to the CTorrent 1.3.4 base, including the patches from the FreeBSD ports tree. - [20]Patch file + [27]Patch file A patch file of changes to the CTorrent 1.3.4 base. - [21]FreeBSD patched source + [28]FreeBSD patched source This includes the patches from the FreeBSD ports tree. - [22]Linux/Windows/Other patched source - Please [23]let me know if you encounter any portability issues, as I + [29]Linux/Windows/Other patched source + Please [30]let me know if you encounter any portability issues, as I don't have a test environment set up for these platforms. ___________________________________ Bitfield::Invert patch - [24]Bitfield::Invert patch + [31]Bitfield::Invert patch See notes in the change log; this is needed if you are using dnh1.1, dnh1, or ctorrent-1.3.4. ___________________________________ Release dnh1.1 - [25]dnh1 to dnh1.1 patch file + [32]dnh1 to dnh1.1 patch file A patch file of changes to release dnh1 to bring it up to dnh1.1. ___________________________________ Release dnh1 - [26]FreeBSD patch file + [33]FreeBSD patch file A patch file of changes to the CTorrent 1.3.4 base, including the patches from the FreeBSD ports tree. Note: Thanks to Florent Thoumie, as of 29 Jul 2005 this patchset is @@ -165,38 +203,38 @@ least net/ctorrent) and install from there, you will have these updates without downloading the file and patching manually. - [27]Patch file + [34]Patch file A patch file of changes to the CTorrent 1.3.4 base. - [28]FreeBSD patched source + [35]FreeBSD patched source This includes the patches from the FreeBSD ports tree. - [29]Linux/Windows/Other patched source - Please [30]let me know if you encounter any portability issues, as I + [36]Linux/Windows/Other patched source + Please [37]let me know if you encounter any portability issues, as I don't have a test environment set up for these platforms. _________________________________________________________________ Resources - [31]CTorrent Home Page + [38]CTorrent Home Page Outdated, but you may find some useful info (particularly the FAQ). - [32]CTorrent SourceForge Project - Hosts the CTorrent codebase, bug reports, patches, and forum. + [39]CTorrent SourceForge Project + Hosts the original CTorrent codebase, bug reports, patches, and forum. - [33]Custom CTorrent + [40]Custom CTorrent A page by the author of the "get1file" patch and other fixes. It contains a custom version and a GUI for CTorrent. - [34]BitTorrent + [41]BitTorrent The official BitTorrent home page. - [35]BitTorrent wiki + [42]BitTorrent wiki Various documentation. - [36]BitTorrent protocol specification (official version) + [43]BitTorrent protocol specification (official version) - [37]BitTorrent protocol specification (wiki version) + [44]BitTorrent protocol specification (wiki version) References @@ -211,29 +249,36 @@ 9. http://www.bittorrent.com/ 10. http://sourceforge.net/projects/ctorrent/ 11. http://www.rahul.net/dholmes/ctorrent/ctcs.html - 12. http://www.rahul.net/dholmes/ctorrent/changelog.html + 12. http://www.rahul.net/dholmes/ctorrent/index.html#download 13. http://www.rahul.net/dholmes/ctorrent/ctcs.html - 14. http://www.rahul.net/dholmes/ctorrent/changelog.html - 15. http://www.rahul.net/dholmes/ctorrent/patchset-ctorrent-dnh1.2-dnh2.diff - 16. http://www.rahul.net/dholmes/ctorrent/patchset-ctorrent-1.3.4-dnh2.diff - 17. http://www.rahul.net/dholmes/ctorrent/ctorrent-1.3.4-dnh2.tar.gz - 18. http://www.rahul.net/dholmes/ctorrent/patchset-ctorrent-dnh1.1-dnh1.2.diff - 19. http://www.rahul.net/dholmes/ctorrent/patchset-ctorrent-1.3.4-dnh1.2-fbsd.diff - 20. http://www.rahul.net/dholmes/ctorrent/patchset-ctorrent-1.3.4-dnh1.2.diff - 21. http://www.rahul.net/dholmes/ctorrent/ctorrent-1.3.4-dnh1.2-fbsd.tar.gz - 22. http://www.rahul.net/dholmes/ctorrent/ctorrent-1.3.4-dnh1.2.tar.gz - 23. mailto:dholmes@ct.boxmail.com - 24. http://www.rahul.net/dholmes/ctorrent/patch-invert.diff - 25. http://www.rahul.net/dholmes/ctorrent/patchset-ctorrent-dnh1-dnh1.1.diff - 26. http://www.rahul.net/dholmes/ctorrent/patchset-ctorrent-1.3.4-dnh1-fbsd.diff - 27. http://www.rahul.net/dholmes/ctorrent/patchset-ctorrent-1.3.4-dnh1.diff - 28. http://www.rahul.net/dholmes/ctorrent/ctorrent-1.3.4-dnh1-fbsd.tar.gz - 29. http://www.rahul.net/dholmes/ctorrent/ctorrent-1.3.4-dnh1.tar.gz + 14. http://www.rahul.net/dholmes/ctorrent/ctcs-protocol.html + 15. http://www.rahul.net/dholmes/ctorrent/index.html#download + 16. http://www.rahul.net/dholmes/ctorrent/changelog.html + 17. http://www.rahul.net/dholmes/ctorrent/ctcs.html + 18. http://www.rahul.net/dholmes/ctorrent/changelog.html + 19. http://www.rahul.net/dholmes/ctorrent/patchset-ctorrent-dnh2-dnh2.1.diff + 20. http://www.rahul.net/dholmes/ctorrent/ctorrent-1.3.4-dnh2.1.tar.gz + 21. http://www.rahul.net/dholmes/ctorrent/patch-btfiles.cpp.diff + 22. http://www.rahul.net/dholmes/ctorrent/patchset-ctorrent-dnh1.2-dnh2.diff + 23. http://www.rahul.net/dholmes/ctorrent/patchset-ctorrent-1.3.4-dnh2.diff + 24. http://www.rahul.net/dholmes/ctorrent/ctorrent-1.3.4-dnh2.tar.gz + 25. http://www.rahul.net/dholmes/ctorrent/patchset-ctorrent-dnh1.1-dnh1.2.diff + 26. http://www.rahul.net/dholmes/ctorrent/patchset-ctorrent-1.3.4-dnh1.2-fbsd.diff + 27. http://www.rahul.net/dholmes/ctorrent/patchset-ctorrent-1.3.4-dnh1.2.diff + 28. http://www.rahul.net/dholmes/ctorrent/ctorrent-1.3.4-dnh1.2-fbsd.tar.gz + 29. http://www.rahul.net/dholmes/ctorrent/ctorrent-1.3.4-dnh1.2.tar.gz 30. mailto:dholmes@ct.boxmail.com - 31. http://ctorrent.sourceforge.net/ - 32. http://sourceforge.net/projects/ctorrent/ - 33. http://customctorrent.ifreepages.com/ - 34. http://bittorrent.com/ - 35. http://wiki.theory.org/CategoryBitTorrent - 36. http://www.bittorrent.com/protocol.html - 37. http://wiki.theory.org/BitTorrentSpecification + 31. http://www.rahul.net/dholmes/ctorrent/patch-invert.diff + 32. http://www.rahul.net/dholmes/ctorrent/patchset-ctorrent-dnh1-dnh1.1.diff + 33. http://www.rahul.net/dholmes/ctorrent/patchset-ctorrent-1.3.4-dnh1-fbsd.diff + 34. http://www.rahul.net/dholmes/ctorrent/patchset-ctorrent-1.3.4-dnh1.diff + 35. http://www.rahul.net/dholmes/ctorrent/ctorrent-1.3.4-dnh1-fbsd.tar.gz + 36. http://www.rahul.net/dholmes/ctorrent/ctorrent-1.3.4-dnh1.tar.gz + 37. mailto:dholmes@ct.boxmail.com + 38. http://ctorrent.sourceforge.net/ + 39. http://sourceforge.net/projects/ctorrent/ + 40. http://customctorrent.ifreepages.com/ + 41. http://bittorrent.com/ + 42. http://wiki.theory.org/CategoryBitTorrent + 43. http://www.bittorrent.com/protocol.html + 44. http://wiki.theory.org/BitTorrentSpecification --- btconfig.h.orig Sat Jan 14 12:20:10 2006 +++ btconfig.h Sun Sep 3 13:48:55 2006 @@ -12,7 +12,7 @@ extern size_t cfg_req_queue_length; #define MAX_PF_LEN 8 #define PEER_ID_LEN 20 -#define PEER_PFX "-CD0200-" +#define PEER_PFX "-CD0201-" extern size_t cfg_cache_size; --- btfiles.cpp.orig Sun Jan 1 18:41:21 2006 +++ btfiles.cpp Sun Mar 19 19:54:53 2006 @@ -170,11 +170,13 @@ return write(fd, &c, 1); #else // ftruncate() not allowed on [v]fat under linux - if( ftruncate(fd,length) < 0 ) { + int retval = ftruncate(fd,length); + if( retval < 0 ) { char c = (char)0; if(lseek(fd,length - 1, SEEK_SET) < 0 ) return -1; return write(fd, &c, 1); } + else return retval; #endif } --- ctcs.cpp.orig Thu Jan 5 20:51:00 2006 +++ ctcs.cpp Tue Aug 15 21:06:00 2006 @@ -161,7 +161,7 @@ buf[CTCS_BUFSIZE-2] = '\n'; buf[CTCS_BUFSIZE-1] = '\0'; } - r = out_buffer.Put(m_sock, buf, len+1); + r = out_buffer.PutFlush(m_sock, buf, len+1); if( r<0 ) Reset(1); } return r; @@ -428,15 +428,16 @@ else if( r == -2 ) m_status = T_CONNECTING; else{ m_status = T_READY; - if( Send_Protocol() != 0 ){ + if( Send_Protocol() != 0 && errno != EINPROGRESS ){ fprintf(stderr,"warn, send protocol to CTCS failed. %s\n",strerror(errno)); return -1; } - if( Send_Auth() != 0) { + if( Send_Auth() != 0 && errno != EINPROGRESS ) { fprintf(stderr,"warn, send password to CTCS failed. %s\n",strerror(errno)); return -1; } - if( Send_Torrent(BTCONTENT.GetPeerId(), arg_metainfo_file) != 0 ){ + if( Send_Torrent(BTCONTENT.GetPeerId(), arg_metainfo_file) != 0 && + errno != EINPROGRESS ){ fprintf(stderr,"warn, send torrent to CTCS failed. %s\n",strerror(errno)); return -1; } @@ -488,15 +489,16 @@ return -1; }else{ m_status = T_READY; - if( Send_Protocol() != 0 ){ + if( Send_Protocol() != 0 && errno != EINPROGRESS ){ fprintf(stderr,"warn, send protocol to CTCS failed. %s\n",strerror(errno)); return -1; } - if( Send_Auth() != 0) { + if( Send_Auth() != 0 && errno != EINPROGRESS ) { fprintf(stderr,"warn, send password to CTCS failed. %s\n",strerror(errno)); return -1; } - if( Send_Torrent(BTCONTENT.GetPeerId(), arg_metainfo_file) == 0 ){ + if( Send_Torrent(BTCONTENT.GetPeerId(), arg_metainfo_file) == 0 + && errno != EINPROGRESS ){ fprintf(stderr,"warn, send torrent to CTCS failed. %s\n",strerror(errno)); return -1; } --- peer.cpp.orig Mon Jan 2 14:40:24 2006 +++ peer.cpp Tue Aug 15 22:02:00 2006 @@ -134,6 +134,7 @@ m_prev_dlrate = 0; m_health_time = m_receive_time = m_choketime = m_last_timestamp; m_bad_health = 0; + m_want_again = m_connect = 0; } int btPeer::SetLocal(unsigned char s) @@ -866,6 +867,10 @@ if( r >= 0){ if( stream.in_buffer.PickUp(68) < 0 ) return -1; m_status = P_SUCCESS; + m_want_again = 1; + // When seeding, new peer starts at the end of the line. + if( BTCONTENT.pBF->IsFull() ) // i am seed + m_unchoke_timestamp = now; } return r; } @@ -964,8 +969,10 @@ if( r < 0 && r != -2 ) return -1; - else if ( r == -2 ) + else if ( r == -2 ){ // seed<->seed f_peer_closed = 1; + m_want_again = 0; + } r = stream.HaveMessage(); for( ; r;){ --- peer.h.orig Mon Jan 2 14:28:59 2006 +++ peer.h Sat Jan 28 19:52:31 2006 @@ -93,7 +93,10 @@ unsigned char m_status:4; unsigned char m_bad_health:1; unsigned char m_standby:1; - unsigned char m_reserved:1; + unsigned char m_want_again:1; // attempt reconnect if lost + + unsigned char m_connect:1; // we initiated the connection + unsigned char m_reserved:7; BTSTATUS m_state; @@ -158,6 +161,9 @@ void CloseConnection(); + int WantAgain() const { return (m_connect && m_want_again) ? 1 : 0; } + void DontWantAgain() { m_want_again = 0; } + void SetConnect() { m_connect = 1; } int AreYouOK(); --- peerlist.cpp.orig Mon Jan 9 11:16:21 2006 +++ peerlist.cpp Fri Aug 18 23:03:43 2006 @@ -115,6 +115,7 @@ if( !peer ) goto err; #endif + peer->SetConnect(); peer->SetAddress(addr); peer->stream.SetSocket(sk); peer->SetStatus( (-2 == r) ? P_CONNECTING : P_HANDSHAKE ); @@ -246,6 +247,11 @@ m_seeds_count = 0; for(p = m_head; p;){ if( PEER_IS_FAILED(p->peer)){ + if( p->peer->WantAgain() ){ // connect to this peer again + struct sockaddr_in addr; + p->peer->GetAddress(&addr); + IPQUEUE.Add(&addr); + } if( pp ) pp->next = p->next; else m_head = p->next; delete p->peer; delete p; @@ -733,6 +739,7 @@ PEERNODE *p = m_head; for( ; p; p = p->next) if(p->peer->bitfield.IsFull()) { + p->peer->DontWantAgain(); if(arg_verbose) fprintf(stderr, "close: seed<->seed\n"); p->peer->CloseConnection(); } @@ -748,7 +755,8 @@ unsigned long rndbits; int r=0; - if (m_opt_timestamp) no_opt = 1; + if(m_opt_timestamp) no_opt = 1; + if(f_seed) no_opt = 1 - no_opt; // Find my 3 or 4 fastest peers. // The MAX_UNCHOKE+1 (4th) slot is for the optimistic unchoke when it happens. @@ -763,8 +771,15 @@ if(cancel_idx == i) continue; if(f_seed){ - // compare upload rate. - if(peer_array[cancel_idx]->RateUL() > peer_array[i]->RateUL()) + // compare time unchoked + if( (!peer_array[i]->Is_Local_UnChoked() && + (peer_array[cancel_idx]->Is_Local_UnChoked() || + peer_array[cancel_idx]->GetLastUnchokeTime() < + peer_array[i]->GetLastUnchokeTime())) || + (peer_array[i]->Is_Local_UnChoked() && + peer_array[cancel_idx]->Is_Local_UnChoked() && + peer_array[i]->GetLastUnchokeTime() < + peer_array[cancel_idx]->GetLastUnchokeTime()) ) cancel_idx = i; }else{ // compare download rate. @@ -782,7 +797,14 @@ if( (btPeer*) 0 != peer_array[cancel_idx] && PEER_IS_SUCCESS(peer_array[cancel_idx]) ){ if(f_seed){ - if(peer->RateUL() > peer_array[cancel_idx]->RateUL()){ + if( (!peer_array[cancel_idx]->Is_Local_UnChoked() && + (peer->Is_Local_UnChoked() || + peer->GetLastUnchokeTime() < + peer_array[cancel_idx]->GetLastUnchokeTime())) || + (peer_array[cancel_idx]->Is_Local_UnChoked() && + peer->Is_Local_UnChoked() && + peer_array[cancel_idx]->GetLastUnchokeTime() < + peer->GetLastUnchokeTime()) ){ loster = peer_array[cancel_idx]; peer_array[cancel_idx] = peer; }else @@ -829,7 +851,8 @@ ( peer_array[MAX_UNCHOKE]->Is_Local_UnChoked() || loster->GetLastUnchokeTime() < peer_array[MAX_UNCHOKE]->GetLastUnchokeTime() )) || - (peer_array[MAX_UNCHOKE]->Is_Local_UnChoked() && + (loster->Is_Local_UnChoked() && + peer_array[MAX_UNCHOKE]->Is_Local_UnChoked() && peer_array[MAX_UNCHOKE]->GetLastUnchokeTime() < loster->GetLastUnchokeTime()) ){ // if current is empty and loser is not, loser gets 25% chance; --- tracker.cpp.orig Sun Jan 1 20:40:31 2006 +++ tracker.cpp Mon Aug 14 22:35:00 2006 @@ -276,19 +276,21 @@ return -1; } - strcpy(tmppath,m_path); + if(!strstr(m_path, "info_hash=")) { + strcpy(tmppath,m_path); - if(strchr(m_path, '?')) - format=REQ_URL_P1A_FMT; - else format=REQ_URL_P1_FMT; + if(strchr(m_path, '?')) + format=REQ_URL_P1A_FMT; + else format=REQ_URL_P1_FMT; - if(MAXPATHLEN < snprintf(m_path,MAXPATHLEN,format, - tmppath, - Http_url_encode(ih_buf, (char*)BTCONTENT.GetInfoHash(), 20), - Http_url_encode(pi_buf, (char*)BTCONTENT.GetPeerId(), 20), - cfg_listen_port, - m_key)){ - return -1; + if(MAXPATHLEN < snprintf(m_path,MAXPATHLEN,format, + tmppath, + Http_url_encode(ih_buf, (char*)BTCONTENT.GetInfoHash(), 20), + Http_url_encode(pi_buf, (char*)BTCONTENT.GetPeerId(), 20), + cfg_listen_port, + m_key)){ + return -1; + } } return Connect();