Questions and Answers : Unix/Linux : out of disk space, why?
Message board moderation
Author | Message |
---|---|
Send message Joined: 28 Jan 05 Posts: 3 Credit: 2,382 RAC: 0 |
Hi, I set the boinc client to leave min .5GB free but it uses up all my disk space constantly so what can I do? |
Send message Joined: 5 Sep 04 Posts: 7629 Credit: 24,240,330 RAC: 0 |
CP is a BIG project: 700Mb per processor during the run, plus 330Mb of zipped files at the completion of a model. The project is really designed for modern computers, which usually have large HDs. How much disk space do you have free to allocate? Les |
Send message Joined: 27 Jan 05 Posts: 6 Credit: 6,054 RAC: 0 |
I have a very similar problem. I have set boinc to use at most 2GB of HD space, but it has already used 3.3 GB. While there is still enough space left now, I wonder what will happen if boinc continues ignoring this setting... (only about 4GB left, and my CP model is not even halfway through yet). Is there any way to force boinc to free up some space without interrupting its calculations? |
Send message Joined: 5 Sep 04 Posts: 7629 Credit: 24,240,330 RAC: 0 |
janhefti, It's a bit hard to be specific because you have hidden your computers. If it's used that much space, it must mean that you have a lot of old model data still in the project folder. Move completed models out of the BOINC folder to a 'save' folder. Les |
Send message Joined: 27 Jan 05 Posts: 6 Credit: 6,054 RAC: 0 |
Hi Les, thank you very much for your fast reply. Actually, most of the data is in the 'dataout' folder of my current project (over 600 files named '3ujcaa._______', e.g. '3ujcaa.ph22c10'). Are these supposed to be sent to cp.net, and could their still being in this folder signify a network problem? Or is it safe to back up the data and move it to another medium? Jan |
Send message Joined: 5 Aug 04 Posts: 1283 Credit: 15,824,334 RAC: 0 |
> thank you very much for your fast reply. Actually, most of the data is in the > 'dataout' folder of my current project (over 600 files named '3ujcaa._______', > e.g. '3ujcaa.ph22c10'). Are these supposed to be sent to cp.net, and could > their still being in this folder signify a network problem? Or is it safe to > back up the data and move it to another medium? Whatever you do don't delete them. They're the working files for the job and are required to generate the 5 result files that will be uploaded when the run completes. After the model finishes the files will be compressed (down to about 330Mb) and moved up to the '3ujc*' directory. You can then delete that directory, preferably archiving it first so that it'll be available should the project team want to analyse more than the 7Mb that gets uploaded (some people were asked to do that for the CPDN classic THC experiment). "The ultimate test of a moral society is the kind of world that it leaves to its children." - Dietrich Bonhoeffer |
Send message Joined: 27 Jan 05 Posts: 6 Credit: 6,054 RAC: 0 |
> Whatever you do don't delete them. They're the working files for the job and > are required to generate the 5 result files that will be uploaded when the run > completes. > > After the model finishes the files will be compressed (down to about 330Mb) > and moved up to the '3ujc*' directory. You can then delete that directory, > preferably archiving it first so that it'll be available should the project > team want to analyse more than the 7Mb that gets uploaded (some people were > asked to do that for the CPDN classic THC experiment). Ok, thank you, I won't. But does that mean that it is impossible to save disk space until the project is finished, and that I will have to make more space available to let boinc complete its task? I did not expect CPDN to take more than 2GB (the limit I set), but it will be around 10GB if disk usage keeps increasing at the current rate. |
Send message Joined: 17 Aug 04 Posts: 753 Credit: 9,804,700 RAC: 0 |
> does that mean that it is impossible to save disk > space until the project is finished, and that I will have to make more space > available to let boinc complete its task? I did not expect CPDN to take more > than 2GB (the limit I set), but it will be around 10GB if disk usage keeps > increasing at the current rate. I'm puzzled that you should be needing so much already. Your computers are hidden, so I cannot be sure whether you are running hyperthreaded with two models on one machine, but even if you are my experience is that you should not need much more than a gigabyte. Each model run is in its own folder, and these are readily identifiable from their number. Other folders in the projects directory are from earlier runs and as Thyme Lawn says can be removed to a storage drive elsewhere or onto CD/DVD. Even toward the end of a model, the total size of a run folder should not get much above 600MB. What is the size of the 'dataout' folder for your current model(s)? |
Send message Joined: 27 Jan 05 Posts: 6 Credit: 6,054 RAC: 0 |
> I'm puzzled that you should be needing so much already. Your computers are > hidden, so I cannot be sure whether you are running hyperthreaded with two > models on one machine, but even if you are my experience is that you should > not need much more than a gigabyte. > > Each model run is in its own folder, and these are readily identifiable from > their number. Other folders in the projects directory are from earlier runs > and as Thyme Lawn says can be removed to a storage drive elsewhere or onto > CD/DVD. Even toward the end of a model, the total size of a run folder should > not get much above 600MB. > > What is the size of the 'dataout' folder for your current model(s)? That machine is crunching only one model at a time, and its 'dataout' folder stands at 3.5GB (it is really the only folder that uses a considerable amount of space). Just in case it helps - I put a list of all files in the projects directory at http://www.janhefti.de/boinc_lsr.txt , and my boinc client is boinc_4.19_i686-pc-linux-gnu. |
Send message Joined: 5 Aug 04 Posts: 1283 Credit: 15,824,334 RAC: 0 |
> That machine is crunching only one model at a time, and its 'dataout' folder > stands at 3.5GB (it is really the only folder that uses a considerable amount > of space). > > Just in case it helps - I put a list of all files in the projects directory at > http://www.janhefti.de/boinc_lsr.txt , and my boinc client is > boinc_4.19_i686-pc-linux-gnu. I've no idea what all those *.da* files in your dataout directory are. They're certainly not generated by the Windoze version of the program, but I'm afraid you'll have to wait for someone who knows the Linux version to say why they're being created and if it's safe to delete them. "The ultimate test of a moral society is the kind of world that it leaves to its children." - Dietrich Bonhoeffer |
Send message Joined: 7 Aug 04 Posts: 2187 Credit: 64,822,615 RAC: 5,275 |
> Just in case it helps - I put a list of all files in the projects directory at > http://www.janhefti.de/boinc_lsr.txt , and my boinc client is > boinc_4.19_i686-pc-linux-gnu. > > Those *da* files are new to me. I don't have anything like that in the dataout directory of my linux PC. Very odd. |
Send message Joined: 6 Aug 04 Posts: 65 Credit: 1,605,224 RAC: 0 |
> Those *da* files are new to me. I don't have anything like that in the > dataout directory of my linux PC. Very odd. > My dataout directory doesn't contain those files either. Are you running the sulphur cycle or some other experiment? When running 4 concurrent models the maximum that CP took up was just over 3 Gb at the end. That includes new models for processing, etc. Once the 4 new ones started and I backed up the completed models it was back down to around 350 Mb. I think you need to know more from the core CP staff on this to figure out what is going on. |
Send message Joined: 27 Jan 05 Posts: 6 Credit: 6,054 RAC: 0 |
Thank you very much, I really appreciate everybody's help. In the meantime, my model completed its run for the first period, presumably sent its data to cp.net (I can access the temperature and precipitation graphs), and aborted on a client error (exit status 251). My computer then started a new model, and it is building up those *da* files again. If they were just a nuisance, it wouldn't matter much, I could just clean them up with a cron job periodically, but if the new model fails on the same error, I guess I will have to give up, as gaining credit without presenting usable results just doesn't feel right. But I will definitely try again once a new version of boinc is available. |
Send message Joined: 5 Sep 04 Posts: 7629 Credit: 24,240,330 RAC: 0 |
You could try the new 4.25 version of BOINC, available from <a href="http://boinc.berkeley.edu/download.php"> here.</a> Although it seems that they have moved to 4.26 now. The 251 error seems to be confined to Linux users. Les |
Send message Joined: 27 Jan 05 Posts: 6 Credit: 6,054 RAC: 0 |
> You could try the new 4.25 version of BOINC, available from <a> href="http://boinc.berkeley.edu/download.php"> here.</a> Although it seems > that they have moved to 4.26 now. > > The 251 error seems to be confined to Linux users. > > Les Good idea! I'll give 4.26 a try. Jan |
©2024 cpdn.org