Thanks for that Martin.
Having the network access set to OFF is a good idea for those that can do so.
A web site white listing option would be good for ALL AVs, I think.

Martin, thanks very much for contacting ESET and for your + their investigations.

---

Cpdn news

I\'ve had a chance to do a bit more investigation...

It\'s not necessary to reboot in safe mode to get the transfer to go through, but it does seem to be necessary to temporarily disable the following AVG components

* Link Scanner (all three checkboxes)
* Web Shield (just the \"Enable Web Shield\" checkbox)
* Resident Shield (just the \"Resident Shield active\" checkbox)

If any of these is still enabled then the behaviour is that, when a transfer starts, it very rapidly (a few seconds) runs through to 100% (much too fast to be real data transfer). At the same time data *starts* uploading from the PC (you can see this in Task Manager). After a period (maybe 5-10 minutes, but I haven\'t timed it), the Transfer switches back from \"Uploading\" to \"Retry in hh:mm:ss\". Data continues flowing from the PC for a further few minutes.

I think a plausible explanation is that there are several different AVG components which need to inspect network traffic. If any of these is enabled, AVG installs a transparent proxy which accepts the upload, temporarily storing it locally, and starts transferring it to the destination (perhaps inspecting it in the process). A consequence of this, however, is that after BOINC has transferred all the data (in fact to a local cache, but it doesn\'t know this) there is then a period of 15 minutes or so when it sees no traffic before the response from climateprediction.net is received. This is long enough for BOINC to time out and think the transfer has failed.

I\'ve had a chance to do a bit more investigation...

It\'s not necessary to reboot in safe mode to get the transfer to go through, but it does seem to be necessary to temporarily disable the following AVG components

* Link Scanner (all three checkboxes)
* Web Shield (just the \"Enable Web Shield\" checkbox)
* Resident Shield (just the \"Resident Shield active\" checkbox)

If any of these is still enabled then the behaviour is that, when a transfer starts, it very rapidly (a few seconds) runs through to 100% (much too fast to be real data transfer). At the same time data *starts* uploading from the PC (you can see this in Task Manager). After a period (maybe 5-10 minutes, but I haven\'t timed it), the Transfer switches back from \"Uploading\" to \"Retry in hh:mm:ss\". Data continues flowing from the PC for a further few minutes.

I think a plausible explanation is that there are several different AVG components which need to inspect network traffic. If any of these is enabled, AVG installs a transparent proxy which accepts the upload, temporarily storing it locally, and starts transferring it to the destination (perhaps inspecting it in the process). A consequence of this, however, is that after BOINC has transferred all the data (in fact to a local cache, but it doesn\'t know this) there is then a period of 15 minutes or so when it sees no traffic before the response from climateprediction.net is received. This is long enough for BOINC to time out and think the transfer has failed.

The curious thing to me is that the whole problem is inconsistent. Throughout all this, my computers (I have 4 running CPDN) have never blocked zip uploads for HADSM, only the zips for HADCMs. Apart from that, the symptoms and solutions seem to be identical to those experienced by other crunchers. Puzzling.

If it\'s a proxy/timeout problem (and John\'s analysis sounds horribly plausible), then it will catch long uploads more than short ones - and the slower your (real) upload speed, the smaller the file which will get throuugh successfully. HadCM uploads are big....

Yes -I can confirm the inconsistent part. It appears only to have blocked the AM uploads on my PC. I noticed that a recent SM passed through without a problem.

Bill H

I\'ve just been sitting here trying all the things ESET suggest the next time uploads were available. Nothing short of uninstalling the software works. Putting exceptions for the uploader IP addresses doen\'t work, nor does telling ESET not to block suspicous behavior.

I confirm John\'s observation of the upload behaviour in post 39157, para starting \"If any of these is still enabled...\" However, I\'ve seen this many times before when the server is down. Things then revert to normal when the server comes back on line.

This time however things did not revert to normal, yet before the latest server breakdown, I\'ve never had this sort of problem.

Also I confirm that small files are CURRENTLY uploading, and I think I spotted a 2MB file uploading. However the 20+MB files are stuck.

Back on the 3&4th March when I had large numbers of files stuck in the uploading (21 & 24 respectively), three of them were small 212KB files, and only one (hadam3p #3 212KB) file did sneek through between those two days. Perhaps it was just a question of being in a queue to upload. If I go back to 24 Feb, only files 2.94MB and over were hanging around (no, I don\'t have a brillant memory - took screen dumps.)

A couple of things I wonder.

1. I\'m using Win7 64bit. What about the others with the problems?
2. Could a windows update have effected the AV software?

I\'ll get back to ESET with more files on Monday.
Till then, Martin

---

1. I\'m using Win7 64bit. What about the others with the problems?
2. Could a windows update have effected the AV software?

I too use WIN7 64bit and it\'s kept up to date with updates.
Because I\'m only processing SMs just now, the problem has <gone away>.

Band width issues.
My connection is a 5GB per month service which normally suffices. Occasionally and when running the hadam3p tasks, I need to double the data limit and I sneak through.
Rather surprised today to hit my 5GB limit after 8 days!

The following is my data use since a week or so before the server issue:
Date MB
06-Feb-2010 7
07-Feb-2010 27
08-Feb-2010 74
09-Feb-2010 275
10-Feb-2010 272
11-Feb-2010 51
12-Feb-2010 137
13-Feb-2010 62
14-Feb-2010 39
15-Feb-2010 78
16-Feb-2010 54
17-Feb-2010 29
18-Feb-2010 63
19-Feb-2010 91
20-Feb-2010 230
21-Feb-2010 409
22-Feb-2010 369
23-Feb-2010 465
24-Feb-2010 376
25-Feb-2010 267
26-Feb-2010 184
Total 5205 (from 27Jan)

27-Feb-2010 625
28-Feb-2010 459
01-Mar-2010 959
02-Mar-2010 883
03-Mar-2010 768
04-Mar-2010 1155
05-Mar-2010 140
Today 372
Total 5364

The 4 March was when I uploaded all 24 files, hence the large data number. However, given that no data was supposed to be transferred while the server was down from the 20 Feb, the high traffic numbers since then do surprise me.

The inference I draw is that the data is heading out of my system , but for whatever reason, the transfer not being completed. Just something to complicate the picture.

Guess what? Just turned my BOINC network connection off ;-)

---

MartinNZ said...

# Rather surprised today to hit my 5GB limit after 8 days!

I think that\'s to be expected, given my suggested diagnosis... every time an upload fails, 25MB is transferred from your system. 5GB in 8 days is about one failed upload per hour, which could easily happen if you have a few stuck uploads.

If my suggested diagnosis is right, a possible fix would be to increase BOINC\'s upload inactivity timeout... does anyone know whether this is possible?

I\'m running AVG 9.0.733 on Windows XP SP3 (32 bit), by the way.

Oh... I\'ve just noticed that 9.0.787 is available, and have updated my systems. I won\'t know whether it\'s cured the problem until a WU completes, though.

If my suggested diagnosis is right, a possible fix would be to increase BOINC\'s upload inactivity timeout... does anyone know whether this is possible?

Afraid not. The 5 minute timeout is fixed.

---

"The ultimate test of a moral society is the kind of world that it leaves to its children." - Dietrich Bonhoeffer

I\'ve had a chance to do a bit more investigation...

It\'s not necessary to reboot in safe mode to get the transfer to go through, but it does seem to be necessary to temporarily disable the following AVG components

* Link Scanner (all three checkboxes)
* Web Shield (just the \"Enable Web Shield\" checkbox)
* Resident Shield (just the \"Resident Shield active\" checkbox)

Here I am. Another victim of AVG... solved with this workaround

Thanks ;-)

---

ESET ESS update. Things now work OK.

Just updated the software to v4.2.35 (first update issued by ESET since Dec 09). For the record, I did a clean install, left all settings on default and used learning mode for the firewall (Win7 64bit).

The large hadam3p files seemed to upload OK on completion with none of the drama I had before.

Will have another hadam3p task completing in 25hrs, so if you don\'t hear from me assume that one went through OK as well.

Given that uploads used to work OK, I suspect a MS update changed something which effected the AV software and that has now been fixed - but who knows. Let\'s hope AVG manage the same.

Thanks for everyone\'s help.

Regards,
Martin

---

AV & ESET ESS update 2.

There is a memory leak issue with Windows 7 that may affect some AV Software and the way it interacts with the CPDN large file uploads.

Updating to the latest version of ESET ESS allowed the uploads to work correctly as I mention above, but I found that my PC was slowing down considerably. I noticed my AV software process went from typically 4-6M to 117M in Task Manager.

The NZ agent for ESET suggested trying this MS Hot Fix: KB979223

As you will see in the MS notes, it relates to a memory leak originating from a process that may used by AV software to check large files. So far it seems to have fixed the memory leak on my system, just uploaded hadam3p successfully with 2 more not far away.

Regards,
Martin

---

Thank you, Martin.

---

Cpdn news

Good link. Should be in the first service pack for win 7, when that comes out.

I love how MSE takes 150MB RAM and other anti-virus products use that much only if there\'s a memory leak in Windows. LOL

For AVG users, this problem should have been alleviated. Input from AVG was that the problem would be fixed with AVG product updates in version 9.0.785 and I am now on 9.0.791 . If any AVG user is still experiencing the problem, I suggest they escalate it with AVG once more.