climateprediction.net (CPDN) home page
Thread 'Upload failures'

Thread 'Upload failures'

Message boards : Number crunching : Upload failures
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 . . . 19 · Next

AuthorMessage
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 60030 - Posted: 27 Apr 2019, 14:53:12 UTC - in response to Message 60029.  

OK, but I have seen cases where my PC would not handle certain file transfers well, depending on the setting of the network card. Turning off Flow Control, and also Interrupt Moderation, sometimes helped. It depends on you local network, especially if wireless is used.
ID: 60030 · Report as offensive     Reply Quote
Dave Roberts

Send message
Joined: 15 Jan 11
Posts: 175
Credit: 6,242,691
RAC: 699
Message 60031 - Posted: 27 Apr 2019, 14:53:26 UTC

No problems with uploading SAM50's but not had any EU25s :- W1ndows7.
ID: 60031 · Report as offensive     Reply Quote
Profiletullio

Send message
Joined: 6 Aug 04
Posts: 264
Credit: 965,476
RAC: 0
Message 60032 - Posted: 27 Apr 2019, 15:02:33 UTC - in response to Message 60030.  

Yes, but how come trickles go up? In my humble opionion, I've been blacklisted for unknown reasons. Now I am making a last stand, using my fastest CPU, a Ryzen 5 1400, 24 GB RAM, Windows 10. I've downloaded 8 tasks and they are running.
Tullio
ID: 60032 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60033 - Posted: 27 Apr 2019, 15:53:40 UTC

Trickles have their their own server, at Oxford.
And they're not zips, they're RPCs.
And they're very small, mostly Kilobytes, but some get up to a bit over one Megabyte.

Comparing the zip files with trickles, is like comparing apples with duck eggs. :)
ID: 60033 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 60037 - Posted: 27 Apr 2019, 19:51:30 UTC - in response to Message 60032.  

Sometimes the DNS servers lose it. These work in the U.S. Maybe they are the same in Europe.

OpenDNS
Primary: 208.67.222.222
Secondary: 208.67.220.220
-------------------------------------------------
Google Public DNS (either order):
* 8.8.8.8
* 8.8.4.4
ID: 60037 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60038 - Posted: 27 Apr 2019, 21:29:57 UTC

I just remembered something from last year.

I got some cams (Central America), which seemed to run OK, but there were problems with the zip uploads. (cam and sam models go to a server in Mexico.)

With the first one, zip 3 refused to upload past about 30%. Then zip 6 did the same. I was wondering if zip 9 would be the same, but all of the others were OK.

Every time a new zip was created and I tried to upload it, first the stuck zip uploaded to the fail point, waited until it timed out, and then the new zip got to upload.
In the end I Aborted the model, on the grounds that it had been given enough chances, and if the researchers wanted that particular data set, they could just reissue it as part of a new batch.

The same thing happened with a second cam. There was only one zip failed, forget which. Same process - let anything that wanted to, upload, then Aborted.

A 3rd cam, on the other hand, was no trouble at all.

So, I'm guessing that it's Tullio's sam50s that are not uploading properly.
If so, then there's 2 answers:
1) Do what I did, and give them as much data as will upload, then Abort, or
2) Just Abort the problem sam50 now.

And, as with my cams, it may not be all of them that won't upload.
ID: 60038 · Report as offensive     Reply Quote
ProfileAlan K

Send message
Joined: 22 Feb 06
Posts: 491
Credit: 31,047,220
RAC: 14,652
Message 60039 - Posted: 27 Apr 2019, 22:10:03 UTC - in response to Message 60038.  

Weren't there problems with the Mexico server at one point last year? Wasn't it down for several days?
ID: 60039 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60040 - Posted: 27 Apr 2019, 22:39:29 UTC

This was at a time when it was working.
And when reported, it was looked into, but nothing amiss could be found.
ID: 60040 · Report as offensive     Reply Quote
Profiletullio

Send message
Joined: 6 Aug 04
Posts: 264
Credit: 965,476
RAC: 0
Message 60041 - Posted: 28 Apr 2019, 9:48:47 UTC
Last modified: 28 Apr 2019, 9:49:19 UTC

I am making my last attempt in CPDN omn two PCs, une with Windows 8.1 and one on Windows 10, both with plenty of RAM and 1 TB disks. If they fail tu upload, I shall leave. I have 4 other projects running on Windows, Linux and Android on a cell phone.
Tullio
ID: 60041 · Report as offensive     Reply Quote
Profiletullio

Send message
Joined: 6 Aug 04
Posts: 264
Credit: 965,476
RAC: 0
Message 60093 - Posted: 6 May 2019, 2:55:14 UTC

One task on the Windows 10 PC has uploaded.Two have failed, four have completed and are trying to upload. One on the Windows 8.1 is still running after 15 days.
Tullio
ID: 60093 · Report as offensive     Reply Quote
wolfman1360

Send message
Joined: 18 Feb 17
Posts: 81
Credit: 14,031,919
RAC: 1,370
Message 60096 - Posted: 6 May 2019, 3:38:21 UTC
Last modified: 6 May 2019, 3:42:35 UTC

So out of curiosity, how much internet should I be using *per* task vs. drive space? Is it 2 gb per task which progressively gets smaller on the drive as the task moves along? Additionally I'm guessing that if an upload fails it would show up either a) in the tasks list on the website or b) in the boinc transfers window?
I have my Toshiba laptop connected to wireless internet and sometimes it can be a bit far from the router if I'm working on it. Will this be an issue?


thanks
ID: 60096 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 60098 - Posted: 6 May 2019, 6:54:48 UTC - in response to Message 60096.  

I have my Toshiba laptop connected to wireless internet and sometimes it can be a bit far from the router if I'm working on it. Will this be an issue?


Unlikely to be an issue if you get a reasonable speed by being closer to the router on a regular basis. If I ever get a really fast machine however I could see myself being restricted by ability to transfer data fast enough. However, the dream machine is unlikely to arrive before the alleged date of superfast broadband of 300MB/s arriving in Cambridge. Indeed it is likely to be at least a year later but then I wouldn't be surprised if the superfast is a year or more late!
ID: 60098 · Report as offensive     Reply Quote
Profiletullio

Send message
Joined: 6 Aug 04
Posts: 264
Credit: 965,476
RAC: 0
Message 60100 - Posted: 6 May 2019, 15:09:46 UTC

I have been running 9 WAH2 tasks on two Windows PC, one Windows 10 (7) and one Windows 8.1 (2). Of the Windows 10 7 tasks ,two have failed with computing errors, one has completed and uploaded, four are trying to upload. Of the 2 8.1 tasks one has completed and is trying to upload, the other is still running. This after two completed tasks were trying to upload when the system crashed due to a bad RAM module and were lost. So of a total of 11 tasks only one has uploaded, one is still running but I have little hope of seeing it uploaded. So I gave my best two PCs to CPDN, excluding other projects. My average credit has risen from zero to about 9000 due to the trickles. Goodbye.
Tullio
ID: 60100 · Report as offensive     Reply Quote
wolfman1360

Send message
Joined: 18 Feb 17
Posts: 81
Credit: 14,031,919
RAC: 1,370
Message 60102 - Posted: 6 May 2019, 15:46:07 UTC - in response to Message 60098.  

I have my Toshiba laptop connected to wireless internet and sometimes it can be a bit far from the router if I'm working on it. Will this be an issue?


Unlikely to be an issue if you get a reasonable speed by being closer to the router on a regular basis. If I ever get a really fast machine however I could see myself being restricted by ability to transfer data fast enough. However, the dream machine is unlikely to arrive before the alleged date of superfast broadband of 300MB/s arriving in Cambridge. Indeed it is likely to be at least a year later but then I wouldn't be surprised if the superfast is a year or more late!


I have my dream Mobile workstation arriving soon, Lenovo x1 extreme. I haven't had a laptop that in some instances is more powerful than this desktop. I'm excited.

Am I correct in assuming the above task sizes? E.g 2 gb per task, 50 mb or so for each zip? I'm not sure where to look to see each file size in either boinc logs or my hard disk.
thanks!
ID: 60102 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 60103 - Posted: 6 May 2019, 17:52:40 UTC

If you go to activity and suspend network activity you can then click on the transfer button and see the size of your uploads.
ID: 60103 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60104 - Posted: 6 May 2019, 20:03:21 UTC

Bye Tullio.
Thanks for trying.
ID: 60104 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60105 - Posted: 6 May 2019, 20:19:17 UTC

Wolfman

If you want to see the file sizes, you'll have to set the network to Suspend network activity, and then wait for one to be created.
Once the data is zipped up, it gets returned asap. Then the zip file is deleted.

The files should be in:
ProgramData/BOINC/projects/climateprediction.net

Zip sizes:
Batch 719 sam25 were 92.77 Megs
Batch 785 hadam4 were 17.3 Megs
The current batch 813 sam50 are 74.6 Megs

It all depends on the size of the area of interest, and the total of the amount of data the researcher wants to get.
ID: 60105 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 60108 - Posted: 7 May 2019, 7:16:28 UTC
Last modified: 7 May 2019, 7:18:29 UTC

The files should be in:
ProgramData/BOINC/projects/climateprediction.net


and if you do run a Linux box and want to check at some point once there is work again,
/var/lib/boinc/projects/climateprediction.net You can't actually look there unless you use sudo or some other means to allow root privileges such as su or you have modified permissions.

Edit:Having never used a Mac for CPDN I will leave it to someone who knows rather than searching the Mac fora for the answer.
ID: 60108 · Report as offensive     Reply Quote
Profiletullio

Send message
Joined: 6 Aug 04
Posts: 264
Credit: 965,476
RAC: 0
Message 60110 - Posted: 7 May 2019, 10:32:50 UTC - in response to Message 60104.  

I've enlisted in Science United to see how it works I am now running Milkyway@home on one Windows 8.1 PC. It is also hosting a Virtual Linux PC with SuSE Thimblewood, a development version. BOINC manager from SuSE does not work, so I installed manually Einstein@home and it runs fine. Cheers.
Tullio
ID: 60110 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60111 - Posted: 7 May 2019, 14:03:47 UTC

Getting back on track:
I've started getting upload fails on jasmin, for batch 813 sam50s.

It might be temporary, so I'll leave the net connection off for a few hours and try again in the morning.
ID: 60111 · Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 · 6 . . . 19 · Next

Message boards : Number crunching : Upload failures

©2024 cpdn.org