I have been trying to upload a couple of completed work units for some days. Is there a server issue?

Bill H

-----------------------------------------
21/02/2010 09:36:13|climateprediction.net|Started upload of hadam3p_npdg_2000_2_006583538_1_1.zip
21/02/2010 09:36:13|QMC@HOME|Sending scheduler request: To report completed tasks. Requesting 0 seconds of work, reporting 1 completed tasks
21/02/2010 09:36:18|climateprediction.net|[error] Error reported by file upload server: [hadam3p_npdg_2000_2_006583538_1_1.zip] locked by file_upload_handler PID=23989
21/02/2010 09:36:18|climateprediction.net|Temporarily failed upload of hadam3p_npdg_2000_2_006583538_1_1.zip: transient upload error
21/02/2010 09:36:18|climateprediction.net|Backing off 13 min 40 sec on upload of hadam3p_npdg_2000_2_006583538_1_1.zip
21/02/2010 09:36:18|climateprediction.net|Started upload of hadam3p_npdg_2000_2_006583538_1_2.zip
21/02/2010 09:36:24|QMC@HOME|Scheduler request completed: got 0 new tasks
21/02/2010 09:37:50||Project communication failed: attempting access to reference site
21/02/2010 09:37:50|climateprediction.net|Temporarily failed upload of hadam3p_npdh_2005_2_006583539_0_2.zip: HTTP error
21/02/2010 09:37:50|climateprediction.net|Backing off 2 hr 11 min 32 sec on upload of hadam3p_npdh_2005_2_006583539_0_2.zip
21/02/2010 09:37:53|climateprediction.net|Started upload of hadam3p_npdh_2005_2_006583539_0_1.zip
21/02/2010 09:38:03||General prefs: from rosetta@home (last modified 12-May-2007 22:12:12)
21/02/2010 09:38:03||Host location: none
21/02/2010 09:38:03||General prefs: using your defaults
21/02/2010 09:38:03||Reading preferences override file
21/02/2010 09:38:03||Preferences limit memory usage when active to 4095.13MB
21/02/2010 09:38:03||Preferences limit memory usage when idle to 7780.74MB
21/02/2010 09:38:03||Preferences limit disk usage to 100.00GB
21/02/2010 09:38:05||Internet access OK - project servers may be temporarily down.
21/02/2010 09:41:34||Project communication failed: attempting access to reference site
21/02/2010 09:41:34|climateprediction.net|Temporarily failed upload of hadam3p_npdg_2000_2_006583538_1_2.zip: HTTP error
21/02/2010 09:41:34|climateprediction.net|Backing off 3 hr 0 min 57 sec on upload of hadam3p_npdg_2000_2_006583538_1_2.zip
21/02/2010 09:41:39||Internet access OK - project servers may be temporarily down.
21/02/2010 09:43:09||Project communication failed: attempting access to reference site
21/02/2010 09:43:09|climateprediction.net|Temporarily failed upload of hadam3p_npdh_2005_2_006583539_0_1.zip: HTTP error
21/02/2010 09:43:09|climateprediction.net|Backing off 2 hr 29 min 14 sec on upload of hadam3p_npdh_2005_2_006583539_0_1.zip
21/02/2010 09:43:15||Internet access OK - project servers may be temporarily down.
21/02/2010 09:43:18||Suspending network activity - user request
21/02/2010 09:44:02||Resuming network activity
21/02/2010 09:44:02|climateprediction.net|Started upload of hadam3p_npdg_2000_2_006583538_1_1.zip
21/02/2010 09:44:02|climateprediction.net|Started upload of hadam3p_npdg_2000_2_006583538_1_2.zip
21/02/2010 09:49:19||Project communication failed: attempting access to reference site
21/02/2010 09:49:19|climateprediction.net|Temporarily failed upload of hadam3p_npdg_2000_2_006583538_1_1.zip: HTTP error
21/02/2010 09:49:19|climateprediction.net|Backing off 6 min 41 sec on upload of hadam3p_npdg_2000_2_006583538_1_1.zip
21/02/2010 09:49:19|climateprediction.net|Temporarily failed upload of hadam3p_npdg_2000_2_006583538_1_2.zip: HTTP error
21/02/2010 09:49:19|climateprediction.net|Backing off 1 hr 43 min 47 sec on upload of hadam3p_npdg_2000_2_006583538_1_2.zip
21/02/2010 09:49:19|climateprediction.net|Started upload of hadam3p_npdh_2005_2_006583539_0_1.zip
21/02/2010 09:49:19|climateprediction.net|Started upload of hadam3p_npdh_2005_2_006583539_0_2.zip
21/02/2010 09:49:33||Internet access OK - project servers may be temporarily down.
21/02/2010 09:54:37||Project communication failed: attempting access to reference site
21/02/2010 09:54:37|climateprediction.net|Temporarily failed upload of hadam3p_npdh_2005_2_006583539_0_2.zip: HTTP error
21/02/2010 09:54:37|climateprediction.net|Backing off 2 hr 26 min 4 sec on upload of hadam3p_npdh_2005_2_006583539_0_2.zip
21/02/2010 09:54:40|climateprediction.net|Temporarily failed upload of hadam3p_npdh_2005_2_006583539_0_1.zip: HTTP error
21/02/2010 09:54:40|climateprediction.net|Backing off 3 hr 22 min 1 sec on upload of hadam3p_npdh_2005_2_006583539_0_1.zip
21/02/2010 09:54:55||Internet access OK - project servers may be temporarily down.
21/02/2010 09:56:01|climateprediction.net|Started upload of hadam3p_npdg_2000_2_006583538_1_1.zip
21/02/2010 09:56:09|climateprediction.net|[error] Error reported by file upload server: [hadam3p_npdg_2000_2_006583538_1_1.zip] locked by file_upload_handler PID=24352
21/02/2010 09:56:09|climateprediction.net|Temporarily failed upload of hadam3p_npdg_2000_2_006583538_1_1.zip: transient upload error
21/02/2010 09:56:09|climateprediction.net|Backing off 1 hr 21 min 35 sec on upload of hadam3p_npdg_2000_2_006583538_1_1.zip

Update

Has anyone any ideas?

I now have 4 zip files waiting for upload and I\'m not sure just what its doing. The router lights flash continuously and my (useful) bandwidth has dropped off dramically.

hadam3p_npdg_2000_2_006583538_1_2.zip
hadam3p_npdg_2000_2_006583538_1_1.zip
hadam3p_npdg_2000_2_006583539_0_2.zip
hadam3p_npdg_2000_2_006583539_0_1.zip

All about 20Mb - I\'d hate to waste the work but perhaps I just need to dump the lot!

Bill H

It might be the server, but it\'s still \"the weekend\", so you\'ll have to wait until the project people are back.
I send them a message for when they get in.

The boinc wiki states something about the problem, but no firm solution.

link

Would someone let me know to which server these files are being uploaded, please?

Update

Has anyone any ideas?

I now have 4 zip files waiting for upload and I\'m not sure just what its doing. The router lights flash continuously and my (useful) bandwidth has dropped off dramically.

same problem here :-(

---

Would someone let me know to which server these files are being uploaded, please?

Milo,

Mo posted in the News thread awhile back about which files for hadam3p go to which server. Looks like two servers might be affected if two files are having trouble being uploaded, if Mo is right.

http://climateapps2.oucs.ox.ac.uk/cpdnboinc/forum_thread.php?id=5447&nowrap=true#38669

The problem files go to different servers.

hadam3p_*_1.zip goes to uploader.oerc.ox.ac.uk (as does hadam3p_*_3.zip)

hadam3p_*_2.zip goes to cpdn-upload1.comlab.ox.ac.uk

Some users have had problems uploading those 2 files because they are larger than the limit set on their network.

---

"The ultimate test of a moral society is the kind of world that it leaves to its children." - Dietrich Bonhoeffer

I have on my (paper) list of which model files go to which server that HadAM3P #1 zip file goes to uploader.oerc.ox.ac.uk and #2 goes to cpdn-upload1.comlab. Both are showing green at the moment on the server status page, though that isn\'t a guarantee.

In the past when servers have refused to accept zip file uploads because the disk was nearly full I don\'t remember having seen the Boinc message reported above:

[hadam3p_npdg_2000_2_006583538_1_1.zip] locked by file_upload_handler PID=24352

---

Cpdn news

Nobody should do anything to try to force these files to upload. It\'s a nuisance but Milo will sort things out, though not necessarily immediately.

---

Cpdn news

How do I determine what server?

Looked at Netstat -b and could it be cpdn-Upload1 ?

Bill H

OK - Although in the short term the PC becomes almost unusable on-line while its trying to upload. It appears to be uploading the whole 25Mb and then failing, then trying again. Including the latest message log entries in case it helps!

Bill H

22/02/2010 18:58:38|climateprediction.net|Temporarily failed upload of hadam3p_npdh_2005_2_006583539_0_2.zip: HTTP error
22/02/2010 18:58:38|climateprediction.net|Backing off 3 hr 26 min 50 sec on upload of hadam3p_npdh_2005_2_006583539_0_2.zip
22/02/2010 18:58:42||Internet access OK - project servers may be temporarily down.
22/02/2010 19:19:35|climateprediction.net|Started upload of hadam3p_npdg_2000_2_006583538_1_2.zip
22/02/2010 19:22:18|climateprediction.net|Started upload of hadam3p_npdh_2005_2_006583539_0_1.zip
22/02/2010 19:24:44||Project communication failed: attempting access to reference site
22/02/2010 19:24:44|climateprediction.net|Temporarily failed upload of hadam3p_npdg_2000_2_006583538_1_2.zip: HTTP error
22/02/2010 19:24:44|climateprediction.net|Backing off 2 hr 10 min 3 sec on upload of hadam3p_npdg_2000_2_006583538_1_2.zip
22/02/2010 19:24:57||Internet access OK - project servers may be temporarily down.

When the server sends the files to a computer, included in this is the location of the server for returning the results. This information is stored in BOINC\'s To Do list, aka the client_state.xml file.

Open this file with notepad, click on Find in menu, type in the name of the model (the second group of 4 characters should be sufficient), and then keep clicking on Find next until you\'re just past a line which says uploader. This line has the needed server name.

Exit from Notepad by clicking on the red X at the top right, so as not to change anything.

---

Backups: Here

Here is the file section for one of the four zip files for upload:-
_____________________________________________________________________
<file_info>
<name>hadam3p_npdg_2000_2_006583538_1_1.zip</name>
<nbytes>26429666.000000</nbytes>
<max_nbytes>80000000.000000</max_nbytes>
<md5_cksum>84ef44a0a7849a06c98874b21dd3c96b</md5_cksum>
<generated_locally/>
<status>1</status>
<upload_when_present/>
<url>http://uploader.oerc.ox.ac.uk/cpdn_cgi/file_upload_handler</url>
<persistent_file_xfer>
<num_retries>33</num_retries>
<first_request_time>1266690720.627699</first_request_time>
<next_request_time>1266872589.005557</next_request_time>
<time_so_far>7611.756095</time_so_far>
<last_bytes_xferred>26429666.000000</last_bytes_xferred>
</persistent_file_xfer>
<signed_xml>
<name>hadam3p_npdg_2000_2_006583538_1_1.zip</name>
<generated_locally/>
<upload_when_present/>
<max_nbytes>80000000</max_nbytes>
<url> http://uploader.oerc.ox.ac.uk/cpdn_cgi/file_upload_handler </url>
</signed_xml>
<xml_signature>
498ba8db9391b1d6b0798689ee82ecced61e04f43c0e36cfa1db3e4d5566265c
45c007b739023f6279d8dcfef7cbfb5bb00da7c8fcf81f75cd8df8bda6cda2b7
3e1c9eefdf2fb1a549405b79db22ddcade68a1e78a43f797897249f7f7ee8252
3fb5d0db59795bd4ce7d5903380d90b96345661a3ea802c55a25438a057a82cb
.
</xml_signature>
</file_info>
_____________________________________________________________________
And Another:-

<file_info>
<name>hadam3p_npdh_2005_2_006583539_0_1.zip</name>
<nbytes>26419294.000000</nbytes>
<max_nbytes>80000000.000000</max_nbytes>
<md5_cksum>185c7982ff7f2e20809c4f677135d7c5</md5_cksum>
<generated_locally/>
<status>1</status>
<upload_when_present/>
<url>http://uploader.oerc.ox.ac.uk/cpdn_cgi/file_upload_handler</url>
<persistent_file_xfer>
<num_retries>30</num_retries>
<first_request_time>1266656934.528933</first_request_time>
<next_request_time>1266876801.588503</next_request_time>
<time_so_far>7794.994826</time_so_far>
<last_bytes_xferred>26419294.000000</last_bytes_xferred>
</persistent_file_xfer>
<signed_xml>
<name>hadam3p_npdh_2005_2_006583539_0_1.zip</name>
<generated_locally/>
<upload_when_present/>
<max_nbytes>80000000</max_nbytes>
<url> http://uploader.oerc.ox.ac.uk/cpdn_cgi/file_upload_handler </url>
</signed_xml>
<xml_signature>
40441c50412715f69240e9b884da777843fa697d3373800e866b66606cb6d9ba
2cc41fac1321c1df1addc5106514428e3af0ca4b3a4d06bfb57d79c74abb5e47
7533bc1175f8d6509d576fd174863cb189bdf3e5d94eb74f4f5d60e0263a9871
3ee88dba0278a5d6f668cddc08cd5649e3573b12862635488acc57c882d9c9a1
.
</xml_signature>
</file_info>


Hope that this helps!

Bill H

Thank you very much, Bill. These are the sections for the #1 file from each model. They confirm that this file uploads to the server I said in a previous post, uploader.oerc. (These HadAM3P models generate 3 files and each uploads to a different server; that\'s to spread the load of these big files.)

Bill, if you could post a file section referring to the second file of one of these models we will be able to confirm that it really should upload to cpdn-upload1.comlab.

I\'m not a programmer, but does it matter that line 9 of each section says:

<url>http://uploader.oerc.ox.ac.uk/cpdn_cgi/file_upload_handler</url>

with no spaces, but the 9th line from the bottom says:

<url> http://uploader.oerc.ox.ac.uk/cpdn_cgi/file_upload_handler </url>

with a space inside each url tag?

---

Cpdn news

I hope that this is what you need?

Bill H
_____________________________________________________________________________

- <file_info>
<name>hadam3p_npdg_2000_2_006583538_1_1.zip</name>
<nbytes>26429666.000000</nbytes>
<max_nbytes>80000000.000000</max_nbytes>
<md5_cksum>84ef44a0a7849a06c98874b21dd3c96b</md5_cksum>
<generated_locally />
<status>1</status>
<upload_when_present />
<url>http://uploader.oerc.ox.ac.uk/cpdn_cgi/file_upload_handler</url>
- <persistent_file_xfer>
<num_retries>35</num_retries>
<first_request_time>1266690720.627699</first_request_time>
<next_request_time>1266911818.509391</next_request_time>
<time_so_far>8404.975822</time_so_far>
<last_bytes_xferred>26429666.000000</last_bytes_xferred>
</persistent_file_xfer>
- <signed_xml>
<name>hadam3p_npdg_2000_2_006583538_1_1.zip</name>
<generated_locally />
<upload_when_present />
<max_nbytes>80000000</max_nbytes>
<url>http://uploader.oerc.ox.ac.uk/cpdn_cgi/file_upload_handler</url>
</signed_xml>
<xml_signature>498ba8db9391b1d6b0798689ee82ecced61e04f43c0e36cfa1db3e4d5566265c 45c007b739023f6279d8dcfef7cbfb5bb00da7c8fcf81f75cd8df8bda6cda2b7 3e1c9eefdf2fb1a549405b79db22ddcade68a1e78a43f797897249f7f7ee8252 3fb5d0db59795bd4ce7d5903380d90b96345661a3ea802c55a25438a057a82cb .</xml_signature>
</file_info>
- <file_info>
<name>hadam3p_npdg_2000_2_006583538_1_2.zip</name>
<nbytes>21111339.000000</nbytes>
<max_nbytes>50000000.000000</max_nbytes>
<md5_cksum>1284b3c4e21118afca01d7908a974477</md5_cksum>
<generated_locally />
<status>1</status>
<upload_when_present />
<url>http://cpdn-upload1.comlab.ox.ac.uk/cgi-bin/file_upload_handler</url>
- <persistent_file_xfer>
<num_retries>30</num_retries>
<first_request_time>1266690720.627699</first_request_time>
<next_request_time>1266917625.931492</next_request_time>
<time_so_far>7969.119488</time_so_far>
<last_bytes_xferred>21111339.000000</last_bytes_xferred>
</persistent_file_xfer>
- <signed_xml>
<name>hadam3p_npdg_2000_2_006583538_1_2.zip</name>
<generated_locally />
<upload_when_present />
<max_nbytes>50000000</max_nbytes>
<url>http://cpdn-upload1.comlab.ox.ac.uk/cgi-bin/file_upload_handler</url>
</signed_xml>
<xml_signature>a4ce46e0b86902764275feefb343bf5b75b70dfdcf57cde3fbd01fd3b5441854 a4c522d7a0322ea254ca70a1963e1c165dbcd7268fec58eab5bf902e73300bac c31d34be91dbe89ed9330a46fac6e18da6fd7479831287e94a6fbab1e5fa7c57 22ff554680318f9a9b01b8228233874d2d368dbaa732e3bdb60c98718f45c5be .</xml_signature>
</file_info>
- <file_info>
<name>hadam3p_npdg_2000_2_006583538_1_3.zip</name>
<nbytes>217557.000000</nbytes>
<max_nbytes>50000000.000000</max_nbytes>
<md5_cksum>c9de2bc4a1e2fde268aa8ccd9e843890</md5_cksum>
<generated_locally />
<status>0</status>
<uploaded />
<upload_when_present />
<url>http://uploader.oerc.ox.ac.uk/cpdn_cgi/file_upload_handler</url>
- <signed_xml>
<name>hadam3p_npdg_2000_2_006583538_1_3.zip</name>
<generated_locally />
<upload_when_present />
<max_nbytes>50000000</max_nbytes>
<url>http://uploader.oerc.ox.ac.uk/cpdn_cgi/file_upload_handler</url>
</signed_xml>
<xml_signature>ae5726f3a4df24f905a90a917aa1064e5c7762e13faf8719c508fed1f69899cf be1233b6266a8f56f7d0748e509b54d8597d4d1818ec57c2604f8770c178cafe bd4c2bb06301c8c19d92c4d781e4e356bd874db10f57bfd981c74599fa7f65b6 185f1c92c132c12f6156618a81c2309d1d7c28b802acad57da54eddcc4945edd .</xml_signature>
</file_info>

I\'m not a programmer, but does it matter that line 9 of each section says:

<url>http://uploader.oerc.ox.ac.uk/cpdn_cgi/file_upload_handler</url>

with no spaces, but the 9th line from the bottom says:

<url> http://uploader.oerc.ox.ac.uk/cpdn_cgi/file_upload_handler </url>

with a space inside each url tag?

It\'s nothing to worry about. The BOINC core client gets the upload URL from the first one. The second (inside the <signed_xml> block) is one of the mechanisms used by the server to ensure data integrity.

---

"The ultimate test of a moral society is the kind of world that it leaves to its children." - Dietrich Bonhoeffer

Not idea about the extra space in the name. Its possible that the integity check rejects if they are not both exactly the same, but that will depend on the code involved(?)

If you like I can manually edit the XML file - but (if it is an issue) that wouldn\'t correct the initial script that inserted it in the first place although it would give an indication if this was a problem or not(?)

Any suggestions gratefully accepted!

Bill H

Thanks for the replies.
I was away yesterday with sporadic access but I\'m now taking a look at uploader.oerc.

Result: It looks like that server is experiencing memory errors, and so I will have to shut it down for further testing.