What you think about virtual files?

[ivan386]

I think this a great idea but not right realized.

Shareaza must share 2 files (1 real, 2 virtual (part of real)).

First file avalible by name and by hash.
Second (virtual) only by hash.

Real file download have new header.

X-Virtual: [URN's] [Range's in real file]

[old_death]

I don't understand what positive effect this could bring... (and I don't understand the idea, BTW )

[ivan386]

Virtual file is a part of file.

In MP3 file is a MP3 Data without ID3 tag. If you change Tag file hash will chenge. Hash of virtual file is stay same. Tag is not part of Virtual file.

[ocexyz]

[ivan386]

It alow get many sources for virtual file part.

[ailurophobe]

Right. So add file metadata that gives the hash and offset for the metadata stripped version of the file so you can use sources with versions of the same file with identical data but different metadata. Would mainly be useful for mp3. I think Shareaza already supports sharing both the full and the stripped version of mp3 files. No idea if it can share hash and offset data about alternate versions with same data. (With 'offset' I mean that it would be necessary to share data about the sizes of headers before Shareaza could use data from versions with different headers.)

[kevogod]

[ailurophobe]

Right. All I remembered was that Shareaza can strip the tags and share the stripped version and that there was talk about also sharing the tagged data. I never was that interested in sharing music, so my recollections of the discussions are vague.

[punkmaister]

*removed by moderator*
Stop hijacking topics without adding something useful to the discussion.

mfg,
Old

[old_death]

Well, if that's your idea, ivan386, it's a cool one. Shareaza already supports sharing either only the stripped file, or only the complete file. Therefore I think adding your virtual files wold be useful (but only with sha1, not the other hashes, as this would (at least at the beginning) be only for the G2 network).

[ailurophobe]

The same logic might be usable for sharing other alternate hashes. Mainly I am thinking that after BT DHT support is working it should be possible to search G2 with the btih to get files with sha1 and ed2k hashes (I think this already happens, just rarely, since Shareaza is pretty rare seed in BT) and then get alternate btih hashes (for torrents that contain the same file) from the sources found using sha1 hash. Also storing alternate hashes in metadata would allow sharing files by btih over G2 while not seeding over BT. This doesn't seem to be happening currently since the btih is only linked to the torrent file. Again, not familiar with 2.5 changes, many of which were in BT handling.

[old_death]

See my comments on this over here: viewtopic.php?f=7&t=358

Virtual files can be used a) to share the metadata-stripped mp3 file hash & b) share the BITH batch torrent hash of torrents that contain one or several files found in the library.

[mojo85]

I don't think this will add any benefit unless downloading from a G1 client. It would add a new layer of complication for too little a gain. It only works as a limited scope "mp3" files, and those files are small enough that 12 original sources are enough as opposed to 100. It also defeats the reason for implementing a tagless hashing for mp3's ... the real metadata was constantly changing and fragmenting the original sources ... it may have started as 100 sources but as time progresses you get 75 under original hash1 25 under new hash2. A couple of months later this changes again ... and more peers are able to share but some under totally new hashes like hash(3,4,5,...n). This was the reason ... now we go back to constantly changing hashes.

The bittorrent idea sounds neat ... but I got an alternative. Instead of re-inventing the wheel (a virtual file to me sounds like a container file). Could we not utilize collections for this sake? It is our equivalent to a Torrent but it is much more powerful and useful. We could add BTIH urn support, and Tracker support. Each file would have it's own sha1/ eD2k/ and TTH hashes ... but on top of that the whole set of files in the collection could also have batch download support (sha1/ TTH/ and BTIH with Tracker), and sha1/ eD2k/ tth hashes for each file within it could be passed to the Downloader as a virtual file. We can modify TorrentWizard to spit out Collections ... this would move us in the right step and also create a semi standard for how future Torrents should behave with respect to batch file. At this point we can't wait for BT to write the rules for us, we have to set an example and they can follow if need be. But it would make Shareaza a better BT client.

If the above is implemented on the collections side of things, we have to suport it on the download side of things. I think the best approach is to use the virtual idea at this point. A batch download would appear as a Folder a drop down menu containing the files within the folder, and there individual progress. Another drop down menu under each file would show the peers we are downloading from. Now each file is technically a virtual file because in the partials it appears as one .sd file ... but because we are treating it as a virtual file ... it contains the information necessary to download from G2, G1, and eD2k (hashes, file offsets, file size, file name, etc...) ... bittorrent would only require the tracker and btih from the collection to download.

Now for progress bars, we can color them in individually for each downloaded part because we know it's offsets and so when downloading we can color based on those ranges with respect to the offsets ... but the parent batch download folder can be colored in based on overall downloaded percentage instead of downloaded pieces.

On uploading ... the BT upload runs as usual but now we can support partial file support for each child file under a batch download folder.

[ocexyz]

Good idea IMHO.

[ivan386]

If you create torrent in TorrentWizard SHA1 ED2K and MD5 will be included for each file. But Shareaza AFAIK don't use them for multi-file torrent.

[old_death]

[rolandas]

The major problem for virtual file hashes is their support. The term "virtual" is an abstract conception. It means that one day the virtual portion of file can become false "virtual", because the tagging changes from day to day, and many proprietary players use their own hackish tags, which no one understands. If you take into account that there's no absolute "tagger" nowadays, then you will get why it's hard to achieve this. Shareaza can not recognize all MPC tags, and that means we can not remove them, or all tags which come before it (if is present in footer), or after it (if is present in header).
There can be APE, MPC, various flavours of ID3 tags in any sequence. There are even LAME encoder tags with its version number embedded. The majority of taggers don't even touch that information. In the past I wanted to implement LAME tag detection, I believe the code is in the codebase, but it was not completed. It's a big headacke with the encoder tags... Some of them are comercial ones, and you never know how to detect them.

[ivan386]

Wrong logic. Only need to find MP3 frames.

I use code that read technical info from MP3 frames (CLibraryBuilderInternals::ReadMP3Frames). It's alredy implemented in shareaza.

"Detect virtual Mp3"