climateprediction.net (CPDN) home page
Thread 'Upload server is out of disk space'

Thread 'Upload server is out of disk space'

Message boards : Number crunching : Upload server is out of disk space
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · Next

AuthorMessage
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 59349 - Posted: 8 Jan 2019, 11:38:13 UTC

My uploads have started to clear. I know Sarah had been working on freeing up some disk space. Don't know whether it will be enough yet or not though.


Well, one got through, all the others have reached 100%, then failed with the transient upload error. Clearly what Sarah has done is not enough yet.

Does anyone have a few hundred terrabytes to donate that can be connected just before the end of Hilary term?
ID: 59349 · Report as offensive     Reply Quote
Iceberg

Send message
Joined: 28 Dec 17
Posts: 18
Credit: 1,097,261
RAC: 147
Message 59368 - Posted: 9 Jan 2019, 21:30:04 UTC - in response to Message 59349.  

Looks good to go now! Thanks to those who worked on getting things back on track!
ID: 59368 · Report as offensive     Reply Quote
ProfileSteve Dodd

Send message
Joined: 28 Oct 11
Posts: 15
Credit: 9,930,176
RAC: 6,998
Message 59371 - Posted: 10 Jan 2019, 0:17:53 UTC - in response to Message 59368.  

Not so fast :) Still getting upload server out of disk space. I've probably got 2 or 3 GB of data to upload still.
ID: 59371 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 59372 - Posted: 10 Jan 2019, 1:53:19 UTC

It may well be next week before the data moving is complete.

In the mean time, I'd recommend turn off Network Access.
ID: 59372 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 59374 - Posted: 10 Jan 2019, 6:57:36 UTC - in response to Message 59372.  

In the mean time, I'd recommend turn off Network Access.


And if like me you have a not too fast connection, when testing to see if things have changed, reduce the maximum number of uploads at a time to one. My laptop with uploads both from here and testing with two uploads/project and 4 total (the defaults) was taking over two hours with the ones over 120MB to tell me they were failing at the end.


Edit: default maximum number of file transfers is 8
The line in cc_config.xml is
<max_file_xfers>8</max_file_xfers>
Changing this to
<max_file_xfers>1</max_file_xfers>
means I know the answer in just over half an hour on a 124MB upload and more typically two mugs of coffee including making them (which for me means grinding the beans by hand) to describe the time in more civilised units.

Edit2: Do this in a text editor and save as is. Using Word, Open Office etc, can seriously mess things up if you don't choose the right option in, "save as."
ID: 59374 · Report as offensive     Reply Quote
Triantafyllos Malakis

Send message
Joined: 18 Nov 15
Posts: 3
Credit: 2,813,238
RAC: 1,866
Message 59375 - Posted: 10 Jan 2019, 7:10:31 UTC

I still get the following reports

10/01/2019 08:55:25 | climateprediction.net | Temporarily failed upload of wah2_safr50_n33l_201412_16_780_011711213_0_r1760893991_10.zip: transient HTTP error
10/01/2019 08:55:25 | climateprediction.net | Backing off 05:52:02 on upload of wah2_safr50_n33l_201412_16_780_011711213_0_r1760893991_10.zip
10/01/2019 08:55:25 | climateprediction.net | Started upload of wah2_safr50_n33l_201412_16_780_011711213_0_r1760893991_12.zip
10/01/2019 08:55:25 | | Project communication failed: attempting access to reference site
10/01/2019 08:55:27 | | Internet access OK - project servers may be temporarily down.

ID: 59375 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 59376 - Posted: 10 Jan 2019, 15:35:05 UTC - in response to Message 59375.  

Getting that on everything now. - better than uploads completing and then failing at 100%.
ID: 59376 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 59377 - Posted: 10 Jan 2019, 15:59:17 UTC - in response to Message 59376.  

Getting that on everything now.

Good grief. I thought I was immune, since the only thing I have left are three of the sam25s - the long ones that run for 70 days on my i7-3770. Now I have a zip stuck in upload too.

The disadvantage of being a world-wide project is that you have experts in all areas to subvert the system.
ID: 59377 · Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 59383 - Posted: 10 Jan 2019, 21:49:33 UTC - in response to Message 59374.  

Edit: default maximum number of file transfers is 8
The line in cc_config.xml is
<max_file_xfers>8</max_file_xfers>
Changing this to
<max_file_xfers>1</max_file_xfers>


There is no such file on my machine.
Where should it be?
ID: 59383 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 59384 - Posted: 10 Jan 2019, 22:04:46 UTC
Last modified: 14 Jan 2019, 5:02:43 UTC

It's in the BOINC Data directory, but only if you create it.

Doc file: here.

And yesterday Sarah was talking about taking the server offline to speed up the data moving.
ID: 59384 · Report as offensive     Reply Quote
ProfileVicki

Send message
Joined: 28 Nov 15
Posts: 50
Credit: 4,099,809
RAC: 0
Message 59397 - Posted: 11 Jan 2019, 19:17:00 UTC

i AM STILL STUCK, CONNECT FAILED or out of server space errors. crunching on knowing the data will be ready to send when there is space for it.
ID: 59397 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 59398 - Posted: 11 Jan 2019, 19:44:14 UTC - in response to Message 59397.  
Last modified: 14 Jan 2019, 5:17:47 UTC

i AM STILL STUCK, CONNECT FAILED or out of server space errors. crunching on knowing the data will be ready to send when there is space for it.



Please read my post here and STOP crunching.

edit
I've just edited this next bit to make it more accurate.


If too much data accumulates in the transfers tab, you may come up against a file limitation in the tasks. If you do, then the task will crash.
And when they crash, BOINC will proceed to the next steps in it's To Do list:

1) Send a completion message to the project, which contains the error message, and a list of zips that didn't get sent back before then.
2) Delete all of the files remaining that didn't get sent back.
ID: 59398 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 59402 - Posted: 12 Jan 2019, 7:00:58 UTC - in response to Message 59398.  

If too much data accumulates in the transfers tab, you may come up against a limitation of BOINC, where ALL of the data will just be deleted, because it's exceeded a limit.


One time when those of us with less speedy machines have an advantage! Mind you if I had two fast machines with 8 cores each I don't think my bored band could keep up even when things are working well!
ID: 59402 · Report as offensive     Reply Quote
Speedy

Send message
Joined: 20 Jul 05
Posts: 25
Credit: 414,873
RAC: 406
Message 59413 - Posted: 12 Jan 2019, 20:36:05 UTC

Something is moving in the right direction. I just received some credit for the task in progress. However none of my trickles have uploaded. Has anybody else noticed receiving credit without anything uploading?
ID: 59413 · Report as offensive     Reply Quote
Christophe Daulie

Send message
Joined: 19 Aug 15
Posts: 5
Credit: 2,899,370
RAC: 0
Message 59414 - Posted: 12 Jan 2019, 21:07:07 UTC - in response to Message 59413.  

yes
ID: 59414 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 59416 - Posted: 12 Jan 2019, 23:30:51 UTC - in response to Message 59413.  

Something is moving in the right direction. I just received some credit for the task in progress. However none of my trickles have uploaded. Has anybody else noticed receiving credit without anything uploading?


Please read my post here about the difference between zips and Trickles.
ID: 59416 · Report as offensive     Reply Quote
Speedy

Send message
Joined: 20 Jul 05
Posts: 25
Credit: 414,873
RAC: 406
Message 59426 - Posted: 13 Jan 2019, 21:29:44 UTC - in response to Message 59416.  

Something is moving in the right direction. I just received some credit for the task in progress. However none of my trickles have uploaded. Has anybody else noticed receiving credit without anything uploading?


Please read my post here about the difference between zips and Trickles.

Thank you for the interesting reading.
Hopefully my 12 zip files will start to upload in the next day or so so I can continue processing the work unit
ID: 59426 · Report as offensive     Reply Quote
rlepra

Send message
Joined: 23 Nov 05
Posts: 1
Credit: 4,133,719
RAC: 0
Message 59427 - Posted: 14 Jan 2019, 3:35:33 UTC

I have more than 80 tasks ready for upload that are not uploading. My connection can upload 300 Mb/s and all SETI and Einstein@home are uploading without problems Can anybody tell me if there are problems with the servers?
ID: 59427 · Report as offensive     Reply Quote
ProfileJIM

Send message
Joined: 31 Dec 07
Posts: 1152
Credit: 22,363,583
RAC: 5,022
Message 59428 - Posted: 14 Jan 2019, 4:36:44 UTC

The server has been out of disc space since last Thursday. Hopefully with the start pf the new weeks enough data will be moved so as to free up adequate space. I have 33 zip waiting to up load and have stopped crunching CPDN for now.
ID: 59428 · Report as offensive     Reply Quote
[SG]Felix

Send message
Joined: 4 Oct 15
Posts: 34
Credit: 9,075,151
RAC: 374
Message 59429 - Posted: 14 Jan 2019, 11:00:56 UTC

do you think the server will be back today?
ID: 59429 · Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 · Next

Message boards : Number crunching : Upload server is out of disk space

©2024 cpdn.org