Constant G2 "lack of traffic" errors - can't stay connected

[dew]

For me, the daily build r9358 had a CPU spike after about 15 minutes, but this time it didn't end. After about an hour, I killed it. I'll experiment more tomorrow.

[Lanigiro]

And now after restarting it and it running for just two hours, it's back to CPU surging again. Somehow, the bug that had gone away has magically reappeared after a few times of quitting and restarting it.

OK. That does it. I am officially putting my foot down and ending this. You will tell me exactly what buttons to push or "Advanced Settings" to twiddle to put it back exactly the way it was three weeks ago today. Simple step-by-step instructions that require no more than five minutes to perform. I will then implement those instructions and afterward, everything will work again in the same manner that it did three weeks before the date on this post, including all aspects of G2 functionality in particular, including the typical lifetime of G2 connections. These CPU surges will also not occur.

Am I making myself clear?

I want my copy of Shareaza back as it was three weeks ago, and I believe I am entitled to have it back in that state. My hardware, my copy of the software, my say-so. I don't know how it got screwed up, and it this point neither do I care. I just want it fixed. Now. I'm willing to fix it myself, given simple step-by-step instructions; I'm not asking for hand-holding here. But I want my damn Shareaza working properly again, without any more of this run-around.

And as the owner of my hardware and of the particular copy of Shareaza at issue here, I have the authority, under the property rights laws in my country, to issue this edict. As of this moment and the immediately preceding utterance, it has become the case that my not having my copy of Shareaza functioning correctly again within 24 hours of now, where by "correctly" I mean "within approximately the same parameters that it was functioning within as of January 28, 2014", will constitute a violation of said property rights, to wit an illegal act of conversion in which an item of my property has effectively been wrested from my control by an unauthorized third party despite not having physically been taken from my possession, and where said third party has been explicitly placed on notice that the conversion in question is not authorized by the property owner. Such an act is potentially subject to civil and even criminal penalties.

Now, instructions on how to configure my copy of Shareaza so as to restore its pre-February-2014 state of functionality please. The 24 hours starts as of the date on this post.

[Lanigiro]

You have neither pushed out another debug version, pushed out a new stable version, nor pushed some button over at G2 HQ to make the ten-minute CPU surges stop -- one of them just happened to me again just now.

You have failed.

I will now begin investigating my exact legal options regarding what seem to constitute repeated remote seizures of my property in contravention of the law and my rights. The person responsible for causing the ten minute CPU surges will certainly be going to prison for violations of the Computer Fraud and Abuse Act perpetrated against mine and others' computers, and other persons may also be civilly or criminally liable, depending.

If any of the responsible persons is reading this, I have this to say to you: Enjoy your day in court.

[dew]

[Winterkeeper]

[Lanigiro]

[Lanigiro]

By now, it should be clear to anyone keeping an eye on the "G2 Network Size" page (http://crawler.doxu.org/history.html) that there are oscillations in the hub/leaf ratio with a period of about one hour, or some shorter time that produces a strong beat with a one-hour period when sampled at whatever rate the Doxu crawler samples. Oscillations which have grown to the point of being clearly visible in the graph there, when they weren't at first, and are presumably still growing. There was a further degradation in the overall performance of the network a week ago, and if the oscillations continue growing, the network may eventually tear itself apart.

Are you going to actually start doing anything about this situation anytime soon?

[raspopov]

[old_death]

You might want to try out the latest debug build. It contains some protection against G2 overloading:
http://shareaza.sourceforge.net/?id=debug

thx rasporov!

[Lanigiro]

[Winterkeeper]

[Winterkeeper]

[raspopov]

Can you (or anybody) debug a this issue? On my computer I can't reproduce this not a single time...

[dew]

[Lanigiro]

The short-lived hub problem is caused by 2.7.1.0.

Check out the graphs at http://crawler.doxu.org/history.html ... lower right quadrant. G2 started acting especially aberrant right around the start of February, but the orange "leaves per hub" line started trending downward much earlier, near the end of November 2013, a sudden departure from tracking very closely with the yellow line until then.

2.7.1.0 was released on Nov. 24, 2013.

I don't think that's a coincidence. I think something buggy in 2.7.1.0 made 2.7.1.0 hubs short-lived, or made them do something that affected the other hubs they were connected to, probably the latter (as I see short-lived 2.6.0.0 and 2.7.0.0 hubs). The orange line began dipping when 2.7.1.0 came out and perhaps continued to dip as 2.7.1.0 uptake continued, until some kind of nonlinear threshold was reached in February, perhaps when hubs could expect more 2.7.1.0 hubs connected than they could deal with due to whatever toxicity 2.7.1.0 emits to adjacent hubs.

Check the 2.7.1.0 G2 networking code, specifically stuff with diffs from 2.7.0.0 and probably specifically hub-to-hub (including hub-to-anything-including-another-hub), and I think you'll find whatever the problem is in short order.

[Lanigiro]

[raspopov]

Probably we found a problem: an experimental G2 client named GTKG (maintained by Raphael Manfredi) sends a query packets in a little different format that Shareaza expects. All Shareaza versions are affected (bug is older than 2006). GTKG client was patched to be gentle with Shareaza, and this bug fixed in Shareaza r9360. Use latest daily build please.

[ram]

I'm sorry that gtk-gnutella (aka GTKG) caused these problems to Shareaza users. The packet was legit, I could not have imagined it would be incorrectly parsed by Shareaza. Fortunately, we were able to cooperate to identify the bug and I changed gtk-gnutella to emit a packet that is slightly different to avoid that parsing bug and block every G2 user out there.!

Note that this means a malicious person could use that parsing bug to shutdown all legacy Shareaza nodes, so it is critical that each person running Shareaza upgrade to the latest version immediately.

[ram]

This issue has been solved: it was due to a parsing bug in Shareaza, causing it to incorrectly parse the query messages that gtk-gnutella sent to it. I added G2 querying in gtk-gnutella on January 27th, but did not understand it was causing problems until I started sending the queries to neighbouring hubs (via TCP), and witnessed that after sending these messages, I was no longer getting any traffic on the connection. It took me a while to realize the problem was not in gtk-gnutella but could be a bug in Shareaza and I contacted the devels, with a hexadecimal dump of the messages I was sending, which helped diagnose and fix the bug.

Note that gtk-gnutella was not doing anything wrong, the packets sent were completely valid, and the problems stemmed from a parsing bug in Shareaza.

As such, it means a malicious person exploiting this parsing bug could in effect disrupt all running Shareaza nodes. Therefore, all Shareaza users must upgrade to the latest version immediately.

Meanwhile, I have slightly altered gtk-gnutella to send a query that Shareaza has no problems to parse, removing the problem for now. I am deeply sorry to have been the (involuntary) cause of this disruption, but no program should misbehave when handling its inputs.

[Lanigiro]

This is excellent news.

And, in a way, not too surprising. There are other incompatibilities between gtk-gnutella and Shareaza that I've noticed. Notably, sometimes I get "wonky hits" in my G2 searches, which exhibit one or more abnormal behaviors under Shareaza:

1. Add with no sources, needing to have the IP:port that sent the hit added as a source manually afterward.

2. Do not respect "files you have already" filter. The file doesn't disappear from search hits when that filter is on and the file is successfully downloaded. Normally that could happen if the download was corrupted and no longer matched the uploaded file, but I've seen instances where the file persists in the search hits but produces the "already in library" popup on an attempt to re-download it, meaning the file is NOT corrupted but is for some reason not being recognized by the "files you have already" filter. It ALSO means the "already in library" popup is not using the same logic as the "files you have already" filter -- which would make this another Shareaza bug, not a gtk-gnutella bug.

3. Duplicate instead of re-adding sources if download hit repeatedly. So you can select the item in search results, have an empty download list, click download in search results ten times, and get ten copies of the item all pending instead of just one like normal.

These all seem to happen frequently with sources self-identifying as gtk-gnutella; item 1 also infrequently occurs with items identified as Shareaza. I suspect bugs on Shareaza's end in cases 1 and 2 -- 1 because it sometimes happens with Shareazas at both ends and 2 because Shareaza is not using the same logic for two different checks for files you already have, when it probably should be ("Don't repeat yourself" principle of coding). 3 I'm less sure of. On the other hand that this happens a lot with gtkg at the other end means gtkg is very good at provoking bugs in Shareaza! And maybe in some cases is actually straying outside the rules of the G2 protocol. Shareaza definitely is sometimes construing it more narrowly than it should, or at least was, though.

I will try the new build tomorrow sometime. Let's hope this works. It's nice to see some good old-fashioned "be liberal in what you accept and conservative in what you send out" internet problem-solving...

[raspopov]

[Lanigiro]

[raspopov]

[Lanigiro]

[skinvista]

I can confirm that just adding "if ( str.IsEmpty() ) break;" in that loop had seemed to fix the problem.

Although buried in a much larger commit r105 (page=13),
and just a few days testing a problem that was random, and also recently unmentioned here.

[ram]

[Lanigiro]

[Lanigiro]

I see 2.7.2.0 is now out. I've been running it for the past 40 minutes and it seems to be running very smoothly. Did the G2 parsing bug impede anything else besides communicating properly with GTKG hosts? Because it seems to be downloading avidly, getting lots of files on my pending list that had been languishing. Or is it just coincidence, a good source coming online today?

[old_death]

Nope. The parsing bug did basically interrupt network communication, and that's (probably) why you have noticed problems downloading until recently.

For more information have a look at the network size fluctuations over the past month caused by the bug: http://crawler.doxu.org/history.html