Request: Need to detect chuncks whith 0

Discuss Shareaza features.
Forum rules
Home | Wiki | Rules

Request: Need to detect chuncks whith 0

Postby ivan386 » 13 Jan 2010 20:26

Some users share incomlete files. Shareaza must detect zero chuncks in file by ed2k or tiger set. And show warning to user that file is incomplete. If user chuse to download it than not need to download that chuncks (We alredy know what in it).
ivan386
 
Posts: 261
Joined: 17 Jun 2009 14:08

Re: Request: Need to detect chuncks whith 0

Postby raspopov » 19 Feb 2010 22:45

Zero chunk can be normal part of file. So declined.
User avatar
raspopov
Project Admin
 
Posts: 945
Joined: 13 Jun 2009 12:30

Re: Request: Need to detect chuncks whith 0

Postby old_death » 19 Feb 2010 22:54

User avatar
old_death
 
Posts: 1950
Joined: 13 Jun 2009 16:19

Re: Request: Need to detect chuncks whith 0

Postby raspopov » 19 Feb 2010 23:04

Anyway its impossible to detect zero chunks with 100%.
Use zip to compress real zero chunks.
User avatar
raspopov
Project Admin
 
Posts: 945
Joined: 13 Jun 2009 12:30

Re: Request: Need to detect chuncks whith 0

Postby ivan386 » 20 Feb 2010 09:38

Green line (hash checked) on white part (not downloaded) will be warning for advanced user.

Whith ed2k hash set we can only detect parts 9728000 bytes. [PATCH] "empty" ed2k block detector.
Whith tiger tree hash set we can detect parts from 1024 bytes to 8388608 bytes.

If user see that file incomlete but have some usefull parts he can search complete copy and join already downloaded data whith this new download.

Stadart whay to detect empty parts is "Assume file is 100% complete and reverify". But it may lose incomplete downloaded parts and it to slow for that easy task.


No need to stay in queue to get "empty" part.
ivan386
 
Posts: 261
Joined: 17 Jun 2009 14:08

Re: Request: Need to detect chuncks whith 0

Postby raspopov » 20 Feb 2010 09:53

"No need to stay in queue to get "empty" part" is different from "Some users share incomlete files" please don't change topic of feature request!
BTW how many zero chunks you saw on eDonkey?
User avatar
raspopov
Project Admin
 
Posts: 945
Joined: 13 Jun 2009 12:30

Re: Request: Need to detect chuncks whith 0

Postby ivan386 » 20 Feb 2010 10:03

> 1000 chuncks

some files already marked as corrupt/fake some not.

Some files have same name and size but diffrent hash.

It's greate problem for big files.
incomlete file.gif
ivan386
 
Posts: 261
Joined: 17 Jun 2009 14:08

Re: Request: Need to detect chuncks whith 0

Postby raspopov » 20 Feb 2010 10:30

Its Gnutella 2 and its just icomplete downloads.
User avatar
raspopov
Project Admin
 
Posts: 945
Joined: 13 Jun 2009 12:30

Re: Request: Need to detect chuncks whith 0

Postby ivan386 » 20 Feb 2010 10:36

At the end of first file big "empty" part.

magnet:?xt=urn:bitprint:HYWC5ILQPVVOZOSUHRKOMKZ5YWTJP2S5.F7YN3X4LKE4CECOVBNCTXBKWRUL5NN2XTZWB3SI&xt=urn:ed2khash:b4c8aeccd780261e7015e4bbb1eaed98&xl=1523968732&dn=CBT%20NUGGETS%20LINUX%20SERIES%20(VIDEO%20TUTORIAL).7z

when you get hash set use "Assume file is 100% complete and reverify".
Last edited by ivan386 on 20 Feb 2010 10:39, edited 1 time in total.
ivan386
 
Posts: 261
Joined: 17 Jun 2009 14:08

Re: Request: Need to detect chuncks whith 0

Postby raspopov » 20 Feb 2010 10:38

I don't thinks so. And we speaking about eDonkey and completed downloads.
User avatar
raspopov
Project Admin
 
Posts: 945
Joined: 13 Jun 2009 12:30

Re: Request: Need to detect chuncks whith 0

Postby ivan386 » 20 Feb 2010 10:40

just chek it.
ivan386
 
Posts: 261
Joined: 17 Jun 2009 14:08

Re: Request: Need to detect chuncks whith 0

Postby brov » 20 Feb 2010 13:13

I checked that file and I can confirm ~180MB verified chunk without any downloading :D
brov
 
Posts: 87
Joined: 05 Jul 2009 12:15

Re: Request: Need to detect chuncks whith 0

Postby mojo85 » 20 Feb 2010 18:29

Just checked it as well. Ivan is correct that this does pose a problem if a person did believe it to be a complete file, when in fact it was just an incomplete file. The hash and file go with each other, but I can see this being of great frustration to an end user and I see this as a potential for a spam exploit. We should definately have a warning system for this scenario, the end user has to be smart enoug to decide whether to continue downloading or to cancel.
mojo85
 
Posts: 115
Joined: 27 Sep 2009 05:35

Re: Request: Need to detect chuncks whith 0

Postby old_death » 20 Feb 2010 18:46

User avatar
old_death
 
Posts: 1950
Joined: 13 Jun 2009 16:19

Re: Request: Need to detect chuncks whith 0

Postby mojo85 » 20 Feb 2010 20:11

mojo85
 
Posts: 115
Joined: 27 Sep 2009 05:35

Re: Request: Need to detect chuncks whith 0

Postby ailurophobe » 21 Feb 2010 01:19

That is the code for detecting files that have no complete sources, which is useful and should be added to Shareaza (if it isn't already there). But what ivan was talking about was some people sharing files as complete when they are actually not and contain large empty parts.

But I am glad you brought this up, because adding a filter check-box that hides files without complete sources would be a good idea. Not sure if all networks report this data though.
ailurophobe
 
Posts: 709
Joined: 11 Nov 2009 05:25

Re: Request: Need to detect chuncks whith 0

Postby mojo85 » 21 Feb 2010 07:49

Yeah that is what I'm saying, not sure how eMule handles this during the download, but this is how it handles things during the search. I haven't used the eD2k network in a while, especially while using eMule (maybe 5 years ago?). But I kind of remember that servers keep a history of the last time they saw such a file complete, I can't find the lugdunum website, but these servers should have extensions that should help us identify some files. But yes this is definately worth adding to Shareaza's routine search filters if it only enhances it a bit.

Well the simplest but most brute force method is to check upon download import. The instant any file is set to download, it waits to get a hashset and assumes complete to rehash and continue downloading ... but the progress bar would show a different color for regions/ chunks/ parts that are a continuous zero. Maybe worth adding as an advanced settings feature as it would just tax cpu.

Can anyone think of a clever way of doing this? Just for the record this is a problem as it leaves room for spam and fake corrupt files to easily proliferate. Also a small file that is like this is ok, the problem is when it is 100mb and over. Than you are just wasting time and bandwidth to download something that could have been prevented.
mojo85
 
Posts: 115
Joined: 27 Sep 2009 05:35

Re: Request: Need to detect chuncks whith 0

Postby old_death » 21 Feb 2010 17:57

User avatar
old_death
 
Posts: 1950
Joined: 13 Jun 2009 16:19


Return to Features

Who is online

Users browsing this forum: No registered users and 1 guest