climateprediction.net (CPDN) home page
Thread 'not uploading ressuts: Disk quota exceeded'

Thread 'not uploading ressuts: Disk quota exceeded'

Message boards : Number crunching : not uploading ressuts: Disk quota exceeded
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
ProfileByron Leigh Hatch @ team Carl ...
Avatar

Send message
Joined: 17 Aug 04
Posts: 289
Credit: 44,103,664
RAC: 0
Message 57920 - Posted: 13 Mar 2018, 20:35:53 UTC

2018-03-13 13:31:32 | climateprediction.net | [error] Error reported by file upload server: can't write file wah2_nam50_p5ip_201012_13_698_011403606_1_r171553032_8.zip: Disk quota exceeded
ID: 57920 · Report as offensive     Reply Quote
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 57922 - Posted: 13 Mar 2018, 21:49:55 UTC - in response to Message 57920.  

Reported...
ID: 57922 · Report as offensive     Reply Quote
flashawk

Send message
Joined: 29 Jun 12
Posts: 31
Credit: 1,438,478
RAC: 0
Message 57923 - Posted: 13 Mar 2018, 22:00:51 UTC

Me too, 7 trickles waiting to upload
ID: 57923 · Report as offensive     Reply Quote
ProfileAlan K

Send message
Joined: 22 Feb 06
Posts: 491
Credit: 31,043,057
RAC: 14,620
Message 57925 - Posted: 14 Mar 2018, 9:20:54 UTC - in response to Message 57920.  

And another one:

14/03/2018 07:38:21 | climateprediction.net | [http] [ID#75] Info: Connected to upload2.cpdn.org (129.67.195.140) port 80 (#1023)


14/03/2018 07:39:48 | climateprediction.net | [http] [ID#75] Received header from server: <data_server_reply>
14/03/2018 07:39:48 | climateprediction.net | [http] [ID#75] Received header from server: <status>1</status>
14/03/2018 07:39:48 | climateprediction.net | [http] [ID#75] Received header from server: <message>can't write file wah2_eu25_qpwj_200512_13_710_011470239_1_r268652926_6.zip: Disk quota exceeded
14/03/2018 07:39:48 | climateprediction.net | [http] [ID#75] Received header from server: </message>
14/03/2018 07:39:48 | climateprediction.net | [http] [ID#75] Received header from server: </data_server_reply>
ID: 57925 · Report as offensive     Reply Quote
flashawk

Send message
Joined: 29 Jun 12
Posts: 31
Credit: 1,438,478
RAC: 0
Message 57926 - Posted: 14 Mar 2018, 14:50:54 UTC

I'm up to 29 trickleup's, 1.2GB of zip files waiting. Server status has everything green, my internet is working fine, other projects are uploading and downloading without issue.
ID: 57926 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 57927 - Posted: 14 Mar 2018, 15:24:23 UTC - in response to Message 57926.  

I'm up to 29 trickleup's, 1.2GB of zip files waiting. Server status has everything green, my internet is working fine


Not surprised you have that much on a fast machine with 24 processors!

The issue is that the server status page only shows the status of the servers at Oxford and not the server that it sends the zips on to. However I not that at least two different regions are affected which may mean it is an Oxford problem. as nam50 zips almost certainly go somewhere different to eu25 ones.

A disk being full wouldn't stop the server from running so it would still show green. I will follow up George's report to let Oxford know that at least two different batches are affected.
ID: 57927 · Report as offensive     Reply Quote
WB8ILI

Send message
Joined: 1 Sep 04
Posts: 161
Credit: 81,522,141
RAC: 1,164
Message 57928 - Posted: 14 Mar 2018, 15:47:00 UTC
Last modified: 14 Mar 2018, 16:19:05 UTC

I have a cam25 (batch 691) that has been trying to upload for maybe 6 weeks.

Edit: Actually 8 weeks - since 14 January.
ID: 57928 · Report as offensive     Reply Quote
Profilebcavnaugh
Avatar

Send message
Joined: 28 Jul 14
Posts: 14
Credit: 3,522,445
RAC: 0
Message 57929 - Posted: 14 Mar 2018, 16:40:53 UTC
Last modified: 14 Mar 2018, 16:42:24 UTC

Seeing this Error on my Host for a few days how:
03/14/2018 10:28:23 | climateprediction.net | [error] Error reported by file upload server: can't write file wah2_eu25_qnz1_199812_13_710_011467737_1_r1966274936_1.zip: Disk quota exceeded

I have 27 Pending and uploading over and over

Crunching@EVGA The Number One Team in the BOINC Community. Folding@EVGA The Number One Team in the Folding@Home Community.
ID: 57929 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 57930 - Posted: 14 Mar 2018, 18:29:27 UTC

Andy is now looking into this. He didn't think there was a space problem at the moment and it might be a false negative. Two moderators told him their computers didn't think it was a false negative. However it may mean the problem isn't exactly what the error message suggests.

I have a cam25 (batch 691) that has been trying to upload for maybe 6 weeks.
I think it may be time to abort tasks trying to upload from the cam25 batches. I know one moderator has done so as they stated as much in the thread on that problem.
ID: 57930 · Report as offensive     Reply Quote
flashawk

Send message
Joined: 29 Jun 12
Posts: 31
Credit: 1,438,478
RAC: 0
Message 57931 - Posted: 14 Mar 2018, 20:08:13 UTC - in response to Message 57930.  

Thanks Dave for looking into this, I'm up to 1.7GB of uploads now and WAS getting a little concerned.
ID: 57931 · Report as offensive     Reply Quote
ProfileJIM

Send message
Joined: 31 Dec 07
Posts: 1152
Credit: 22,363,583
RAC: 5,022
Message 57932 - Posted: 14 Mar 2018, 20:37:17 UTC

There may be another problem. I have a hadcm3s on one of my machines that has been trying to upload zips for 3 days now.
3/14/2018 2:19:58 PM | climateprediction.net | [error] Error reported by file upload server: can't write file hadcm3s_77bh_200012_168_514_010916142_1_r1070049319_1.zip: Disk quota exceeded
3/14/2018 2:19:58 PM | climateprediction.net | Temporarily failed upload of hadcm3s_77bh_200012_168_514_010916142_1_r1070049319_1.zip: transient upload error
3/14/2018 2:19:58 PM | climateprediction.net | Backing off 00:03:18 on upload of hadcm3s_77bh_200012_168_514_010916142_1_r1070049319_1.zip
3/14/2018 3:44:39 PM | climateprediction.net | Started upload of hadcm3s_75mt_200012_168_514_010913958_1_r1785738456_9.zip
3/14/2018 3:44:39 PM | climateprediction.net | Started upload of hadcm3s_77bh_200012_168_514_010916142_1_r1070049319_1.zip
3/14/2018 3:50:01 PM | climateprediction.net | [error] Error reported by file upload server: can't write file hadcm3s_77bh_200012_168_514_010916142_1_r1070049319_1.zip: Disk quota exceeded
3/14/2018 3:50:01 PM | climateprediction.net | Temporarily failed upload of hadcm3s_77bh_200012_168_514_010916142_1_r1070049319_1.zip: transient upload error
3/14/2018 3:50:01 PM | climateprediction.net | Backing off 00:07:55 on upload of hadcm3s_77bh_200012_168_514_010916142_1_r1070049319_1.zip
3/14/2018 3:50:12 PM | climateprediction.net | [error] Error reported by file upload server: can't write file hadcm3s_75mt_200012_168_514_010913958_1_r1785738456_9.zip: Disk quota exceeded
3/14/2018 3:50:12 PM | climateprediction.net | Temporarily failed upload of hadcm3s_75mt_200012_168_514_010913958_1_r1785738456_9.zip: transient upload error
3/14/2018 3:50:12 PM | climateprediction.net | Backing off 00:42:35 on upload of hadcm3s_75mt_200012_168_514_010913958_1_r1785738456_9.zip
Looks like someone else has a full disc. Don’t the Hadcm3s still go to Oxford.
ID: 57932 · Report as offensive     Reply Quote
mmonnin

Send message
Joined: 28 May 17
Posts: 49
Credit: 17,340,396
RAC: 7,292
Message 57933 - Posted: 14 Mar 2018, 22:23:51 UTC

8 trickles including a task is waiting to upload for me.
ID: 57933 · Report as offensive     Reply Quote
ProfileVicki

Send message
Joined: 28 Nov 15
Posts: 50
Credit: 4,099,809
RAC: 0
Message 57934 - Posted: 14 Mar 2018, 22:24:09 UTC

15/03/2018 10:58:49 a.m. | climateprediction.net | [error] Error reported by file upload server: can't write file wah2_eu25_qku7_198612_13_710_011463675_1_r487278854_8.zip: Disk quota exceeded

Hello from another person stuck in this same traffic Jam that seams to have the name Disk Quota Exceeded. any clues as to when to try again, rather than wasting multiples of 64ish megs of data per upload at a time?

Thanks Vicki.
ID: 57934 · Report as offensive     Reply Quote
MartinNZ

Send message
Joined: 22 Mar 06
Posts: 144
Credit: 24,695,428
RAC: 0
Message 57935 - Posted: 15 Mar 2018, 4:16:21 UTC - in response to Message 57934.  

Two different models failing for me, same disk full message, e.g.:

15/03/2018 16:34:17 | climateprediction.net | [error] Error reported by file upload server: can't write file wah2_sas50_qb38_201612_13_713_011505115_0_r2101745720_2.zip: Disk quota exceeded

15/03/2018 15:42:31 | climateprediction.net | [error] Error reported by file upload server: can't write file wah2_eas50_n1ey_201612_10_712_011491593_0_r956618033_restart.zip: Disk quota exceeded
ID: 57935 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 57936 - Posted: 15 Mar 2018, 9:39:35 UTC
Last modified: 15 Mar 2018, 9:42:24 UTC

any clues as to when to try again, rather than wasting multiples of 64ish megs of data per upload at a time?


I am hoping that Andy will at least have an update sometime later today. I think all the moderators active on here have sent messages along with a couple of others who are on that email list. I will post an update as soon as I see it unless someone else on the list has already done so.

Edit: I have disabled network activity and will leave it disabled for at least a few hours after the problem is resolved as most crunchers won't disable network activity and the servers will get hammered leading to some more transient HTTP errors.
ID: 57936 · Report as offensive     Reply Quote
mmonnin

Send message
Joined: 28 May 17
Posts: 49
Credit: 17,340,396
RAC: 7,292
Message 57937 - Posted: 15 Mar 2018, 10:38:41 UTC - in response to Message 57936.  

[quote]Edit: I have disabled network activity and will leave it disabled for at least a few hours after the problem is resolved as most crunchers won't disable network activity and the servers will get hammered leading to some more transient HTTP errors.


Clients should automatically go into a back-off with increasing periods between communications attempts. This reduces the mass rush of connections once servers come back online.

My log backed this one off 3 hours.

5581 climateprediction.net 3/14/2018 6:01:01 PM Backing off 03:05:03 on upload of wah2_eas50_a1kv_201612_10_711_011486556_0_r423592631_3.zip
ID: 57937 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 57938 - Posted: 15 Mar 2018, 12:03:12 UTC - in response to Message 57937.  

Clients should automatically go into a back-off with increasing periods between communications attempts. This reduces the mass rush of connections once servers come back online.


I am sure it does reduce the problems but when the problem has lasted several days, there is still a logjam and if it has been over a week I will usually wait 24 hours to avoid wasting bandwidth. (less of an issue now I am on an unlimited plan but I like to think that in a small way I am reducing congestion on the local system for my neighbours.) A new router arriving soon should also speed things up for me - a friend with the same ISP who replaced his ageing router got around six times the speed of the old one and mine is older than his was.
ID: 57938 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 57939 - Posted: 15 Mar 2018, 12:09:02 UTC

My backoffs were up to 5 or 6 hours, but are now down to 1.5 hours again. Either they recycle by themselves, or maybe if you attempt a manual upload.
ID: 57939 · Report as offensive     Reply Quote
Jacob Klein

Send message
Joined: 28 Mar 13
Posts: 16
Credit: 5,383,625
RAC: 0
Message 57940 - Posted: 15 Mar 2018, 12:09:39 UTC

I have several PCs on my local network, because of this issue, continually uploading then failing, causing a nasty local bandwidth problem. I wonder if the backoffs aren't aggressive enough. Hope the project gets this figured out promptly. Frustrated. Thanks for listening.
ID: 57940 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 57942 - Posted: 15 Mar 2018, 13:19:22 UTC

This has also been raised in the BOINC forum.

http://boinc.berkeley.edu/dev/forum_thread.php?id=12349


I see Richard has also raised a ticket for the developers suggesting a change to the BOINC server code so that a project such as CPDN which has large uploads which are loaded to a central server which then sends them on to further servers depending on where the research for that particular batch is being done will check with the final destination servers before uploading the data rather than uploading it all and then sending a message back to say that there isn't any space etc.

This would eliminate the bandwidth part of the issue.
ID: 57942 · Report as offensive     Reply Quote
1 · 2 · Next

Message boards : Number crunching : not uploading ressuts: Disk quota exceeded

©2024 cpdn.org