Searching/queued/etc.

[tharrison1]

I just found Shareaza again -- it seems to keep moving around -- and updated from something like 2.2.1.0 to 2.5.3.0.

Sorry to hear about your problems with domain hijackers.

2.5.3.0 mostly seems to be a vast improvement over 2.2.1.0, but ...

1. I don't like the alternation between Searching and Pending/Queued/whatever every few seconds. One issue with that is that you can't sort properly by status any more, e.g. all the queued or pending together. Can this be changed, or at least made optional? I'd rather the statuses were

Paused - only when paused
Queued - only when locally queued and inactive, but not paused
Searching - when none of the above, and looking for more sources
Pending - when none of the above, and remote host busy
In Line - remotely queued
Downloading

Alternatively, drop Searching as a status entirely and show a magnifying glass next to the status (e.g. Pending) if it's searching, too.

2. The way it handles Queued isn't very smart. If there's a large number of downloads it Queues all the new ones. It won't resume them, ever, on its own, unless one of the first 25 or so files downloads. If you manually resume them, after a while it will randomly Queue some other group of files and the same then applies.

First of all, it can Queue ed2k files in A4AF state in preference to everything else, and resume one of them automatically if the existing ed2k download from one of its sources completes. (Even smarter, it can try to download ed2k files one at a time starting with the one with the most sources and Queue the rest, unless there are ones with nonoverlapping sets of sources; one from each then.)

Second, it can Queue files with no sources, since aside from intermittent requeries these don't do anything anyway. I take it there's a limit on how many you want Pending at one time?

Third, for the rest it can rotate them in a round-robin fashion. It can try the first 25 files, then the next 25, and so on, then return to the start of the list, requesting the files from their known sources.

If Queued is about limiting the number of gnutella files Pending at one time, it can also completely ignore bt and ed2k files.

Maybe it's better to just try file 1 (and requery if no known sources), then try file 2, then try file 3, etc., and at the end of the list return to file 1 (with a wait if there are so few files that it would be trying file 1 every few seconds). The more files you have pending, the less frequently it tries any one of them. But it will try them all if you wait long enough, even if none of them download.

3. It forgets push sources at the drop of a hat. A regular source is busy, it will remember it and try it again in a while. A push source is busy and it gets dropped straight away. That doesn't seem right.

4. I have a bunch of ed2k downloads languishing at anywhere from 50% to over 99% done. All the progress on these got made several days ago, and since then, nothing. Now and again one shows as Active or even Downloading rather than Pending, but it doesn't actually seem to make any progress -- even if I see 14KB/s and 00:01 remaining on one of them that's at 99.7%, if I check back on it later I'll see it as Active or Pending and still at 99.7%! It seems these downloads have gotten "stuck" somehow. I don't know what's wrong in this case. I have highID and pass the connection test so I doubt it's a router/firewall problem at my end (and what kind of router/firewall problem would let me get HALF of a file and stop me getting the rest, rather than simply slow it down throughout or stop it entirely?)

[old_death]

There is a possibility to set the number of parallel active downloads. You could try to set it to a higher value...

As for the status, I totally agree to you. There should be a name for that status, and not 2 names (Searching and Queued) that do switch all the time.

As for eDonkey, that's something I can't help you with. There have been several reports in the past noticing slower downloads on that network but we have never been able to tell whether this had a cause we could correct.

[ailurophobe]

Searching used to correspond to Shareaza actually actively searching for the file, but changes were made to way the searches are handled and for some reason I have either forgotten or never knew this was removed. Apparently the dev in question didn't want people thinking Shareaza has stopped doing automatic searches so he did this periodic cycling to "searching" to show that Shareaza is actively searching for the file even if it is not happening right now. I doubt anyone has ever been happy with this solution, but it has never been nuisance enough for anyone to code something better.

As OldDeath said you should just increase the maximum files. The problem is that having too many downloads active increases network overhead and the optimal maximum files limit varies depending on what you are connected to and what you are downloading. The default is often too low if you use ED2K much.

I can't tell if there is a problem with push sources or not from your description. Since you can't connect to a push source directly the connection is routed thru other systems with a link to the push source such the G2 hubs or ED2K servers it is using. These routes are not permanent and expire automatically after some time. Obviously if you are not going to query the source again before the route expires you might as well forget the source immediately. But I can't really tell if that is what you are seeing.

The ED2K downloads completion problem seems similar to what another user reported some time back. But that one was never actually solved that I know of. What we asked him to do was to try the debug builds since there is a good chance this has been fixed, but that was not practical for him and we never found out if that would have worked. Is this enough of a nuisance for you that you are willing to try installing a debug build?

[tharrison1]

If it searches infrequently for more sources for all waiting files, then there's really no use for the intermittent "Searching" notification; it can go away and the manual can note that it automatically seeks more sources for files, but not continually.

The push source thing you describe is clearly a problem. If a push source is busy, it will be impossible to ever download a file from it if it's not possible to keep retrying it until it is not busy because something "expires" before you will have had time to get to the front of the line. If that's happening then a better method has to be found to keep track of push sources for however long it takes.

Perhaps you can add a new query type to the network that searches not for a specific file but for a specific leaf node. Search for the push source itself, then ask it again for the file, every so often until it's non-busy. This will tax the network a lot less than searching for the file itself, which is what happens if it forgets all sources. There are a lot fewer leaves than files on the network, probably by a factor of thousands. Each hub is being asked if some IP is in its leaf list (maybe 300 items) instead of to do a more complex query or even to pass that query on to all maybe-as-many-as-300 leaves, and then a specific one known to have the file gets asked for the file.

Another thing to do would be to promote the stability of push source to hub connections even at the expense of non-push leaf to hub connections, so that the former "expire" more slowly on average.

As for "debug versions", I'd have to be pretty desperate to risk using nightly builds of actively developed software rather than use the current or even an older released version. Nightlies presumably are sometimes going to crash, trash your library, things like that.

[siavoshkc]

I'm going to tell you my idea about issues orderly one by one:
1) I like it the way it is right now but can't deny your logic. I like verbosity as a dev.
2) We should stick to the ones that upload to us. If one file is downloading it is not wise to stop downloading and going for a queued download. Because the download won't start immediately and we should wait in uploaders queue. You should set your number of simultaneous downloads to 70 or something. When an item is queued Shareaza don't mention it like paused items but unlike paused ones it will automatically change them to pending when needed. Exception is when it has no source which I will discuss in another thread.
Second with files with no source we should always search for new sources.
3) X
4) I think you don't have connection problem. If you are sure downloads are valid (not fake). Then be sure eventually they will complete. I often see downloads that don't have some parts in whole network. Usually they are fake and to pollute network.

One thing to note is that some changes in code may be harder than it seems to, and may not worth being done. Maybe like adding a magnifying glass.

[tharrison1]

Of course I wasn't suggesting the round-robin thing should actually interrupt a file when it's downloading. If a file is downloading it should keep doing so until it's done or the uploading sources all have quit for any reason.

And it makes sense for Shareaza to prioritize files with a recent history of successful downloading, and sources with a recent history of successfully uploading to it. After that though it should prioritize the files that have been quiescent the longest. They may have few, busy sources or sources that aren't online very frequently, and Shareaza and the sources could keep missing each other if none are "there" very often. So Shareaza needs to check these fairly frequently in the hopes of catching a source online and available to upload to it.

[old_death]

I think, we should change something on the way the files in the Searching/Queued state are dealt with. First of all, these files should only show "Searching" while they are actually searching for sources (like files in any other status should do). Second, while they are shown as Queued (while they are not searching for sources and not dealing with other active sources), they should definitely be dealt with as if they actually were queued.
The current annoyance I do see from time to time is that I would like to restart some queued downloads and therefore I do select like 20-30 of them, use the right-click menu on them and the restart button is disabled, because I selected one or several files of the annoying Searching/Queued group together with the other files. The problem is that even if these files are shown as queued a big part of the time, they are not dealt with if they were queued, but they are dealt with as active downloads.
Therefore I'd like to suggest that all download that is in the "Searching/Queued" state and will not search for sources (or do anything else) for several minutes should be migrated into the Queued group of downloads and be migrated back into the appropriate group once an event actually makes this necessary (either the times says there should be an other search for sources or an event that adds a source to the download). Like this, Shareaza could deal much more efficiently with long lists of downloads as the more inactive downloads are currently queued, the more downloads that have sources and can download could start directly instead of waiting in the group of queued downloads for these inactive downloads to finish.

BTW, tharrison1, our debug builds are relatively stable ATM, so they do crash now and then, but in average, that's less than one crash per week. Also, there have been measures taken to prevent the library from getting screwed up or loosing downloads, so that's something that happens more often while upgrading from one (stable) version to an other then because of debug build crashes. So, what I want to tell you is: The general advice we give most people that come here to report problems is to use the debug builds, as they are secure enough that we can assure that there shouldn't be major problems (which means that the possibility there are such problems is relatively small, not that it doesn't exist.)

[siavoshkc]

[old_death]

[tharrison1]

[old_death]

I was talking about updating between major versions. From time to time, there are changes in the way partial files are stored. Then and only then there is a risk of damaging partial files. This means you do run almost the same risk by using one debug build after the other than you would by only using major versions.

[siavoshkc]

Whats the Searching/Queued file? The file is either Queued or Pending (Searching). Right?
Maybe you mean status Queued is shown for two different things. Or better say we have two types of Queued file. one that's Shareaza is searching for more sources and one that is suspended really. At least i know it is true for download with no sources.

But the fact is resume options is available for both types (Aka any file with Queued status).

[old_death]

[siavoshkc]

[old_death]

[siavoshkc]

If you mean Downloads.MinSources, I think its the minimum sources a file should have to be able to change into Pending state (start downloading).

[old_death]

It is the minimum amount of sources Shareaza requires before aggressively searching for sources every now and then - or at least, that's my information.

See: https://sourceforge.net/apps/mediawiki/ ... #Downloads

[siavoshkc]