climateprediction.net (CPDN) home page
Thread 'Upload problem'

Thread 'Upload problem'

Message boards : Number crunching : Upload problem
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · 4 · Next

AuthorMessage
Profileold_user5593

Send message
Joined: 31 Aug 04
Posts: 27
Credit: 1,523,155
RAC: 0
Message 40250 - Posted: 27 Jul 2010, 18:47:39 UTC

since several days I have problems uploading the data.
I get these messages when uploading three different results:

27.07.2010 19:51:06 climateprediction.net Started upload of hadsm3dhet2_jyjn_006607765_4_1.zip
27.07.2010 19:51:06 climateprediction.net Started upload of hadam3p_nb1w_1983_2_006162814_2_2.zip
27.07.2010 19:52:08 Project communication failed: attempting access to reference site
27.07.2010 19:52:08 climateprediction.net Temporarily failed upload of hadsm3dhet2_jyjn_006607765_4_1.zip: HTTP error
27.07.2010 19:52:08 climateprediction.net Backing off 3 hr 36 min 8 sec on upload of hadsm3dhet2_jyjn_006607765_4_1.zip
27.07.2010 19:52:08 climateprediction.net Started upload of famous_u4hm_1799_200_006638661_0_1.zip
27.07.2010 19:52:09 Internet access OK - project servers may be temporarily down.
27.07.2010 19:52:09 climateprediction.net Temporarily failed upload of hadam3p_nb1w_1983_2_006162814_2_2.zip: HTTP error
27.07.2010 19:52:09 climateprediction.net Backing off 1 hr 49 min 22 sec on upload of hadam3p_nb1w_1983_2_006162814_2_2.zip
27.07.2010 19:52:35 climateprediction.net Temporarily failed upload of famous_u4hm_1799_200_006638661_0_1.zip: HTTP error
27.07.2010 19:52:35 climateprediction.net Backing off 3 hr 21 min 15 sec on upload of famous_u4hm_1799_200_006638661_0_1.zip

Any ideas?
These messages are the same since a week or more. On other servers there are no problems.
Also with other projects there are no problems in uploading.
I'm not using a proxy.

ID: 40250 · Report as offensive     Reply Quote
Profilemo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 40262 - Posted: 28 Jul 2010, 22:31:38 UTC
Last modified: 28 Jul 2010, 22:32:08 UTC

Hi Legolas

I had a lot of files to upload yesterday. Most uploaded immediately but some produced messages like:

28/07/2010 00:05:32|climateprediction.net|Temporarily failed upload of famous_u0d0_599_200_006633319_3_9.zip: HTTP error

But the files uploaded successfully 5 or 10 minutes later.

Have you got files stuck in the Transfers tab that cannot upload?
Cpdn news
ID: 40262 · Report as offensive     Reply Quote
Profileold_user5593

Send message
Joined: 31 Aug 04
Posts: 27
Credit: 1,523,155
RAC: 0
Message 40264 - Posted: 29 Jul 2010, 5:36:00 UTC - in response to Message 40262.  

At the moment there are 5 files in the transfer tab, which are to be transferred.
The upload goes to a maximum of 1 MB when it stucks.
There are 3 hadam and 2 famous WUs to upload.

I had the same from time to time but in the past the upload was successful after several tries.

ID: 40264 · Report as offensive     Reply Quote
Profileold_user5593

Send message
Joined: 31 Aug 04
Posts: 27
Credit: 1,523,155
RAC: 0
Message 40265 - Posted: 29 Jul 2010, 8:38:18 UTC - in response to Message 40264.  

One point:
I have two other machines where the upload is ok with client version 6.10.56
The machine that fails has 6.10.58
Can that be the reason?
ID: 40265 · Report as offensive     Reply Quote
Profilemo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 40266 - Posted: 29 Jul 2010, 8:59:31 UTC
Last modified: 29 Jul 2010, 8:59:53 UTC

I don't think the computer's Boinc version can be the problem. I have computers with 6.2.19, 6.10.56 and 6.10.58; they all upload files. But on all three computers there are occasional problems with CPDN HTTP and other server errors. They could be various messages for the same server problem.

CPDN server status is all green. Check that the computer has Network activity always available in the Activity menu. Instead of waiting for Boinc to try again press the Try again button in the Transfers section and see what happens. I would use this button at intervals but not dozens of times repeatedly. Once each time.
Cpdn news
ID: 40266 · Report as offensive     Reply Quote
Profileold_user5593

Send message
Joined: 31 Aug 04
Posts: 27
Credit: 1,523,155
RAC: 0
Message 40267 - Posted: 29 Jul 2010, 9:22:07 UTC - in response to Message 40266.  

That machine is at home - at the moment I'm not there.
But I did that yesterday evening and each time it started uploading with the expected speed of my internet connection and suddenly it stucks after about 1MB uploaded.
Is there a possibility to restart the upload servers although the status is green?
I read in another thread that this solved the problem a few months ago.
ID: 40267 · Report as offensive     Reply Quote
Profileold_user5593

Send message
Joined: 31 Aug 04
Posts: 27
Credit: 1,523,155
RAC: 0
Message 40287 - Posted: 3 Aug 2010, 6:23:38 UTC - in response to Message 40267.  

The number of results to be uploaded is still growing: now I have 8 files waiting for uploading. They all stuck after 1 MB.
what can be done ? :-(
ID: 40287 · Report as offensive     Reply Quote
ProfileJIM

Send message
Joined: 31 Dec 07
Posts: 1152
Credit: 22,363,583
RAC: 5,022
Message 40288 - Posted: 3 Aug 2010, 6:31:00 UTC

Does anyone know when the uploader.oerc will be back up? I already have a 2 zip files stuck in the transfer tab, and will have 2 more by morning. In the meantime I have suspended network communication. I guess that all we can do is wait for the data transfer to be complete.


ID: 40288 · Report as offensive     Reply Quote
Profileapohawk
Avatar

Send message
Joined: 14 Sep 08
Posts: 1
Credit: 156,832
RAC: 0
Message 40290 - Posted: 3 Aug 2010, 7:29:16 UTC - in response to Message 40288.  

I've got two files stuck:

2010-08-03 09:20:33 climateprediction.net Started upload of famous_ulnt_1999_200_006660916_2_18.zip
2010-08-03 09:20:40 Project communication failed: attempting access to reference site
2010-08-03 09:20:40 climateprediction.net Temporarily failed upload of famous_ulnt_1999_200_006660916_2_18.zip: connect() failed
2010-08-03 09:20:40 climateprediction.net Backing off 1 hr 22 min 28 sec on upload of famous_ulnt_1999_200_006660916_2_18.zip
2010-08-03 09:20:41 Internet access OK - project servers may be temporarily down.

and

2010-08-03 09:25:38 climateprediction.net Started upload of famous_ulns_1799_200_006660915_1_15.zip
2010-08-03 09:25:40 Project communication failed: attempting access to reference site
2010-08-03 09:25:40 climateprediction.net Temporarily failed upload of famous_ulns_1799_200_006660915_1_15.zip: connect() failed
2010-08-03 09:25:40 climateprediction.net Backing off 3 hr 58 min 29 sec on upload of famous_ulns_1799_200_006660915_1_15.zip
2010-08-03 09:25:41 Internet access OK - project servers may be temporarily down.

Now those've been lingering for few days now. Is it because uploader.oesc being down or are those task just messed up?
Other uploads went ok.
ID: 40290 · Report as offensive     Reply Quote
Profilemo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 40291 - Posted: 3 Aug 2010, 7:36:45 UTC

Please read the post about uploader.oerc in the forum News thread at the top of this Number Crunching section. As soon as we know more that is where it will be announced.
Cpdn news
ID: 40291 · Report as offensive     Reply Quote
Profileold_user5593

Send message
Joined: 31 Aug 04
Posts: 27
Credit: 1,523,155
RAC: 0
Message 40304 - Posted: 5 Aug 2010, 20:17:22 UTC - in response to Message 40291.  

The uploading is not really better although all upload servers seem to work.
In the meantime two files of FAMOUS 6.11 were uploaded but still 5 files stuck after about 1 MB.
And I have still 2 HADAM and 1 HADSM file to be uploaded - also stuck after about 1 MB.

Here the messages with xfer-debug:

05.08.2010 20:34:07 climateprediction.net [fxd] starting upload, upload_offset -1
05.08.2010 20:34:07 climateprediction.net Started upload of hadam3p_nb1w_1983_2_006162814_2_1.zip
05.08.2010 20:34:07 climateprediction.net [file_xfer_debug] URL: http://uploader.oerc.ox.ac.uk/cpdn_cgi/file_upload_handler
05.08.2010 20:34:08 climateprediction.net [file_xfer_debug] FILE_XFER_SET::poll(): http op done; retval 0
05.08.2010 20:34:08 climateprediction.net [file_xfer_debug] parsing upload response: <data_server_reply> <status>0</status> <file_size>1130358</file_size></data_server_reply>
05.08.2010 20:34:08 climateprediction.net [file_xfer_debug] parsing status: 0
05.08.2010 20:34:08 climateprediction.net [fxd] starting upload, upload_offset 1130358
05.08.2010 20:34:38 Project communication failed: attempting access to reference site
05.08.2010 20:34:38 climateprediction.net [file_xfer_debug] FILE_XFER_SET::poll(): http op done; retval -184
05.08.2010 20:34:38 climateprediction.net [file_xfer_debug] file transfer status -184
05.08.2010 20:34:38 climateprediction.net Temporarily failed upload of hadam3p_nb1w_1983_2_006162814_2_1.zip: HTTP error
05.08.2010 20:34:38 climateprediction.net Backing off 2 hr 18 min 40 sec on upload of hadam3p_nb1w_1983_2_006162814_2_1.zip


ID: 40304 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 40305 - Posted: 5 Aug 2010, 21:21:30 UTC - in response to Message 40304.  
Last modified: 5 Aug 2010, 21:22:50 UTC

It may be something external to your computer:
ISP limit
Business/Educational institute file size limit, or data amount limit.
Proxy server problem.

Or if you have your own router/network, something with that.
Backups: Here
ID: 40305 · Report as offensive     Reply Quote
Profilemo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 40306 - Posted: 5 Aug 2010, 21:58:14 UTC
Last modified: 5 Aug 2010, 22:22:05 UTC

I don't know much about network problems but wonder whether trying this could help:

http://setiathome.berkeley.edu/forum_thread.php?id=47636&sort=6#768589

[Edit: Don't do that yet; just read it. I've sent a message to Gundolf on Seti asking whether he thinks trying that would be a good idea.]
Cpdn news
ID: 40306 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 40307 - Posted: 5 Aug 2010, 23:10:11 UTC - in response to Message 40306.  

That's what I had in mind, but was going to wait and see, before explaining about cc_config.

ID: 40307 · Report as offensive     Reply Quote
Profilemo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 40311 - Posted: 6 Aug 2010, 6:54:30 UTC
Last modified: 6 Aug 2010, 6:54:48 UTC

There may be another member with the same problem plus a person who seems to have the same upload problem on the CPDN Beta project.

Legolas, we don't need to know the model names or task numbers or workunit numbers for the files that cannot upload. But could you please tell us for each file:

* the model type (eg HadAM3P, HadSM, FAMOUS)
* the version (eg 6.10, 6.11)
* the file number (that's the number like _11, _13 at the end of each file name in the Transfers tab. Not the number at the end of the name in the Tasks tab.)

When we know that we can look at our list of which files upload to which server to see if it's a problem with one upload server or more. Please list all the details for all the files because some models upload files to 3 different servers(!).
Cpdn news
ID: 40311 · Report as offensive     Reply Quote
Profilemo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 40313 - Posted: 6 Aug 2010, 8:32:43 UTC
Last modified: 6 Aug 2010, 8:34:19 UTC

I've had a reply on Seti from Gundolf who says:

The symptoms Legolas describes are different from what I have experienced, but he could try the other protocol nonetheless. I can't imagine any problems being introduced by doing so.

I am running HTTP 1.0 for several years now without problems (SETI and Einstein). My problems were probably caused by my ISP.

Legolas should also try a reboot, to flush all (DNS) buffers of unwanted information.


* If you haven't rebooted since this upload problem began try that first.

* If the files still don't upload use Gundolf's suggestion in the Seti forum thread. Before you edit the cc_config file you should completely exit from Boinc by right-clicking on the Boinc icon and selecting Exit.

* If that still doesn't work it would be useful if you could give us the information I listed in my last post.
Cpdn news
ID: 40313 · Report as offensive     Reply Quote
Profileold_user5593

Send message
Joined: 31 Aug 04
Posts: 27
Credit: 1,523,155
RAC: 0
Message 40314 - Posted: 6 Aug 2010, 8:51:50 UTC

Thanks for your answers.
I will try your suggestions this evening.

I did already a reboot since the problem occured first and I also rebooted my rooter. That didn't help so far.
ID: 40314 · Report as offensive     Reply Quote
Profileold_user5593

Send message
Joined: 31 Aug 04
Posts: 27
Credit: 1,523,155
RAC: 0
Message 40317 - Posted: 6 Aug 2010, 16:54:46 UTC - in response to Message 40311.  

I tested the HTTP option and it had no effect :-(
and I also booted my machine.

Here are the models, versions and filenumbers ( 3 different WUs ):

1. HadSM3 6.07 File _1
2. HADAM3P 6.14 Files _1 and _2
3. FAMOUS 6.11 Files _2, _3, _5, _6, _8

The HadSM3 and HADAM3P WU has already been finished.
The FAMOUS files not in the list had the same problem but at least they were uploaded.
Can you see an error on the server when files are uploaded?
ID: 40317 · Report as offensive     Reply Quote
Profilemo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 40321 - Posted: 6 Aug 2010, 18:24:00 UTC
Last modified: 6 Aug 2010, 18:25:08 UTC

Well, you are trying everything.

I can't see the server, nor can any of the other moderators. None of us are in Oxford where CPDN is. Milo, the programmer, can see the servers. But now it's his weekend.

I was hoping that all the files would be allocated to one upload server and Milo would find a problem on it.

But if our upload server lists are correct, HadSM, HadAM3P file _1 and FAMOUS files _3 and _6 go to uploader.oerc, while HadAM3P _2 and FAMOUS _2, _5 & _8 go to upload.1.comlab. If the FAMOUS 6.11 is from a new enough batch all its files will go to kraken. So two or three upload servers need to accept your files.

I think this must be some connection problem at your end. I would be surprised if it's a bug on two or three upload servers. While we think again about this, don't constantly retry the transfers in the Transfers tab. I think the number of upload attempts allowed for each file is 100.

But other files have uploaded OK?

Have you got another computer there with internet access? If so, does it crunch CPDN and do its files all upload?
Cpdn news
ID: 40321 · Report as offensive     Reply Quote
Profileold_user5593

Send message
Joined: 31 Aug 04
Posts: 27
Credit: 1,523,155
RAC: 0
Message 40323 - Posted: 6 Aug 2010, 19:30:28 UTC - in response to Message 40321.  

I have 17 other projects running here all of them without upload problems.
And I have no other computers here, only at work which upload without problems too.
I upgraded to Boinc client 6.10.58 a few day ago.
I will go back to 6.10.56 but I don't think that will help.
It's a try.
Most of the time the upload goes up to 1.09 MB when it stucks, so smaller zip files will upload without problems.
I crunched a lot of CPDN WUs without any uploading problems.

It doesn't matter if the problem is not solved over the weekend if we can get a solution one time.

Have a nice weekend.
ID: 40323 · Report as offensive     Reply Quote
1 · 2 · 3 · 4 · Next

Message boards : Number crunching : Upload problem

©2024 cpdn.org