climateprediction.net (CPDN) home page
Thread 'Why is Climate Prediction grabbing all the CPU time'

Thread 'Why is Climate Prediction grabbing all the CPU time'

Message boards : Number crunching : Why is Climate Prediction grabbing all the CPU time
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · Next

AuthorMessage
old_user62579

Send message
Joined: 10 Mar 05
Posts: 3
Credit: 18,163
RAC: 0
Message 29069 - Posted: 30 May 2007, 11:02:30 UTC

I\'m running Rosetta, Climate Prediction, and Seti. And it appears that Climate Prediction is grabbing all the CPU time. The only way that I can get time for Seti and Rosetta is to suspend Climate. How do I get around this other than constantly monitoring BOINC?

Ted
ID: 29069 · Report as offensive     Reply Quote
ProfileMikeMarsUK
Volunteer moderator
Avatar

Send message
Joined: 13 Jan 06
Posts: 1498
Credit: 15,613,038
RAC: 0
Message 29070 - Posted: 30 May 2007, 11:26:59 UTC


The Boinc manager tries to work out if the various project\'s tasks can complete within the deadlines (given the amount of \'resource share\' you\'ve assigned to each project).

If it thinks that a project has insufficient resource share, and therefore the task will miss it\'s deadline, it gives 100% of time to that particular task until it looks OK.
I'm a volunteer and my views are my own.
News and Announcements and FAQ
ID: 29070 · Report as offensive     Reply Quote
old_user62579

Send message
Joined: 10 Mar 05
Posts: 3
Credit: 18,163
RAC: 0
Message 29071 - Posted: 30 May 2007, 11:36:41 UTC

OK. But, in looking at the calculated amount of CPU time that\'s going to be needed to complete this task, 2508:02:10 by 12/11/2007 or 104 days of constant cpu use (which it won\'t get), it seems that BOINC is going to give Climate all the time. This is unfair to the other tasks. Maybe Climate needs to rethink the size of work units that they give out. Or, maybe I need to rethink running Climate.

Ted
ID: 29071 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 29072 - Posted: 30 May 2007, 12:45:58 UTC

Just rethink how it all works.
Which is that BOINC will run climate exclusively for some time, perhaps as long as a week. (It depends on your computer speed, the percentage of the day the computer is on, etc.)
Then it will realise that there isn\'t a problem after all, and go back to sharing the cpu among the projects.


Backups: Here
ID: 29072 · Report as offensive     Reply Quote
ProfileThyme Lawn
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1283
Credit: 15,824,334
RAC: 0
Message 29075 - Posted: 30 May 2007, 13:51:00 UTC - in response to Message 29071.  

OK. But, in looking at the calculated amount of CPU time that\'s going to be needed to complete this task, 2508:02:10 by 12/11/2007 or 104 days of constant cpu use (which it won\'t get), it seems that BOINC is going to give Climate all the time. This is unfair to the other tasks. Maybe Climate needs to rethink the size of work units that they give out. Or, maybe I need to rethink running Climate.

What you can\'t see in the BOINC Manager interface is the project debt. While a project is running exclusively due to deadline problems it builds up a negative long-term debt. This will prevent that project from downloading more work until the debt has been brought back down to your task switching period. After BOINC calculates that your CPDN model is going to meet its deadline (possibly not until it has finished) your other projects will run until the CPDN debt has been worked off. This ensures that your project resource shares are satisfied in the long term. Generally speaking, the more you try to manually manage the way BOINC schedules tasks the worse things will get deadline-wise.

I believe that the next major release of BOINC will show debt on the project tab.
"The ultimate test of a moral society is the kind of world that it leaves to its children." - Dietrich Bonhoeffer
ID: 29075 · Report as offensive     Reply Quote
old_user62579

Send message
Joined: 10 Mar 05
Posts: 3
Credit: 18,163
RAC: 0
Message 29092 - Posted: 31 May 2007, 21:48:48 UTC - in response to Message 29075.  

I believe that the next major release of BOINC will show debt on the project tab.


If it doesn\'t it should. It might help people understand on how it allocates time.
ID: 29092 · Report as offensive     Reply Quote
ProfileKeck_Komputers
Avatar

Send message
Joined: 5 Aug 04
Posts: 426
Credit: 2,426,069
RAC: 0
Message 29093 - Posted: 1 Jun 2007, 5:36:03 UTC - in response to Message 29092.  

I believe that the next major release of BOINC will show debt on the project tab.


If it doesn\'t it should. It might help people understand on how it allocates time.

Not in the 5.10.x versions, maybe in the 6.x.x ones. The first of the RPCs needed was added recently but the manager does not use it yet.
BOINC WIKI

BOINCing since 2002/12/8
ID: 29093 · Report as offensive     Reply Quote
EclipseHA

Send message
Joined: 28 Aug 04
Posts: 42
Credit: 1,443,857
RAC: 0
Message 29230 - Posted: 13 Jun 2007, 5:39:12 UTC

This does seem to be a \"twist\" starting with Boinc CC 5.8.x.

BTW folks, Boinc Manager is only a tool to display what\'s happening. The Core Client is what\'s doing the work.

The reason I say this, is that I don\'t run Boinc Manager - I run the CC with BoincView. BV has displayed short/long term debt for a long time!

For some time, the CC \"knew\" about CPDN, and ignored the \"due date\" for the WU. That way the CC didn\'t devote all it\'s time to CPDN if the was a chance to miss the \"unused\" deadline.

Since moving to the 5.8.16 CC, on a slower box of mine, I do notice the CC really favoring CPDN due to debt (which it shouldn\'t, as the CPDN is kind of meaningless, and logic in the code knew this). I\'ll see nothing but CPDN for days on that box, where project switching used to work quite well.

I suspect that somewhere between 5.4.x and 5.8.x this \"knowledge\" of CPDN was removed in the CC - someone probably wanted to clean up project specific conditions in the standard code (not a bad thing by itself, but not understanding why the code was there resulted in a problem with the local scheduler). Wasn\'t there a new scheduler introduced in 5.8.x of the CC?

Seems there\'s two things that could be done to make multi-project scheduling work better:

1) put back in the CPDN specific code in the CC (not the cleanest solution, but read on!)

2) Increase the deadlines for CPDN WU\'s (double them to allow for slower machines - BTW, by \"slower\", I\'m seeing the problem on a 1.8ghz machine)

The thing is, #2 won\'t take effect until a new CPDN WU is downloaded, which could be a few months. While #1 isn\'t the clean solution, it\'s something that can be done in the middle of a WU, with an update to the CC, and could be done in a much shorter time.

I use CPDN as a \"backup project\", in that if LHC, for example, has work, I want to crunch LHC as much as possible (CPDN have a 25% share, LHC has a 750% share on my machine). With the current scheduler scheme in the CC, even if LHC had work (any day now they say!), my machine wouldn\'t even request work from LHC, but would keep on doing my 10 month long CPDN WU!

BTW, boincview is a pretty good \"viewer\" for what\'s happening with the CC, and can deal with multiple clients at once. I\'ve tried Boinc Manager a few time, and it\'s really limited (things like debts, for example)

See http://boincview.amanheis.de/
ID: 29230 · Report as offensive     Reply Quote
EclipseHA

Send message
Joined: 28 Aug 04
Posts: 42
Credit: 1,443,857
RAC: 0
Message 29231 - Posted: 13 Jun 2007, 5:44:42 UTC - in response to Message 29072.  
Last modified: 13 Jun 2007, 5:47:18 UTC

Just rethink how it all works.
Which is that BOINC will run climate exclusively for some time, perhaps as long as a week. (It depends on your computer speed, the percentage of the day the computer is on, etc.)
Then it will realise that there isn\'t a problem after all, and go back to sharing the cpu among the projects.




Prior to 5.8.x, the CC knew that CPDN wasn\'t ever a problem. Like I said in another post, this was also when there was a \"new and improved\" scheduler in the CC.

Les, I do notice that your RAC is zero, and that kind of indicates that you\'re not even crunching CPDN right now!
ID: 29231 · Report as offensive     Reply Quote
EclipseHA

Send message
Joined: 28 Aug 04
Posts: 42
Credit: 1,443,857
RAC: 0
Message 29232 - Posted: 13 Jun 2007, 5:49:26 UTC - in response to Message 29093.  

I believe that the next major release of BOINC will show debt on the project tab.


If it doesn\'t it should. It might help people understand on how it allocates time.

Not in the 5.10.x versions, maybe in the 6.x.x ones. The first of the RPCs needed was added recently but the manager does not use it yet.


Boincview had displayed debt for something like a year. The API has been around for a long time, but maybe not in a way that boincmgr could use it!

ID: 29232 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 29233 - Posted: 13 Jun 2007, 6:52:01 UTC

Les, I do notice that your RAC is zero, and that kind of indicates that you\'re not even crunching CPDN right now!


Yes, I\'ve been \'elsewhere\' since late 2005.
First with the 200 year \'spinups\', (needed to \'seed\' the TCMs), then an early TCM on the BBC, then nearly a year of Seasonal Attribution models, then back to a cpdn / BBC pair, which were stopped in favour of beta testing new TCMs for a while.
The cpdn model failed, and I decided to concentrate on finishing off the BBC model to get it \'off my books\'.
About 10 days ago I finished the BBC model, (\"waits for applause; not a sausage.\" - Bluebottle, The Goon Show), and shut down for some renovations.

This is taking longer than I expected/hoped, but Real Soon Now. Then I\'m spoilt for choice again:
SAP
2 types of beta testing
TCMs on cpdn
And 2 other climate projects starting 4th quarter!

In the mean time, my RAC has gone to ruin. Haw Haw Haw. (Sorry. A bit stressed at present.)

ID: 29233 · Report as offensive     Reply Quote
ProfileMikeMarsUK
Volunteer moderator
Avatar

Send message
Joined: 13 Jan 06
Posts: 1498
Credit: 15,613,038
RAC: 0
Message 29234 - Posted: 13 Jun 2007, 7:02:23 UTC
Last modified: 13 Jun 2007, 7:05:44 UTC

Ditto with me. 1,200 RAC on CPDN-beta (slab), 500-ish RAC on CPDN-SAP, but I don\'t have enough computers to go round hence the original CPDN isn\'t getting any at the moment, nor am I able to run any CPDN-beta coupled models.

I'm a volunteer and my views are my own.
News and Announcements and FAQ
ID: 29234 · Report as offensive     Reply Quote
Profileold_user17525

Send message
Joined: 13 Sep 04
Posts: 161
Credit: 284,548
RAC: 0
Message 29235 - Posted: 13 Jun 2007, 9:32:08 UTC

same here as well, down to a miserly 8.63 now :-)
_________________________________
ID: 29235 · Report as offensive     Reply Quote
Profilemo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 29240 - Posted: 13 Jun 2007, 21:39:23 UTC

Some of the BBC members have noticed Les\'s completed model:

http://bbc.cpdn.org/forum_thread.php?id=5565&PHPSESSID=88edcf984909ae8c5e973a491e04e1d8#42616

I spent more than 8 months 24/7 on a slow machine completing a BBC model (same 180-year model as the cpdn ones). I\'m now manually sharing time among a couple of betas, a couple of SAPs and two standard cpdn runs.
Cpdn news
ID: 29240 · Report as offensive     Reply Quote
Craig W Post

Send message
Joined: 24 Nov 05
Posts: 2
Credit: 2,606,078
RAC: 0
Message 29365 - Posted: 29 Jun 2007, 14:09:53 UTC - in response to Message 29069.  

I\'m running Rosetta, Climate Prediction, and Seti. And it appears that Climate Prediction is grabbing all the CPU time. The only way that I can get time for Seti and Rosetta is to suspend Climate. How do I get around this other than constantly monitoring BOINC?

Ted

My previous posting on this problem seems to have disappeared. I have the same problem as mentioned in the original message. I have run SETI for years and more recently EINSTEIN, ROSETTA, and Climate Prediction. I have permanently suspended Climate Prediction since BOINC seems to give it all the time on my machine. If this problem is ever fixed and I find out that it has been fixed, I will then allow Climate Prediction to run on my system. This is is a bug and a disruptive one at best. I have SETI, Einstein, and ROSETTA running on my machine without any problems. It seems to me that Climate Prediction should be able to share the machine with these projects in a like manner. Climate Prediction has lost my machine permanently until this bug has been fixed.
ID: 29365 · Report as offensive     Reply Quote
ProfileMikeMarsUK
Volunteer moderator
Avatar

Send message
Joined: 13 Jan 06
Posts: 1498
Credit: 15,613,038
RAC: 0
Message 29368 - Posted: 29 Jun 2007, 17:38:10 UTC
Last modified: 29 Jun 2007, 17:43:04 UTC



Sorry about your original post, the database had to be restored from backup tapes, and some recent posts were lost.

It\'s not a bug, nor is it anything to do with the CPDN project itself, but with the Boinc manager (which is behaving \'as designed\').

Boinc is designed to look at the deadlines for individual workunits, and will bump up their priority if the deadline looks like it will be missed. It keeps track of the extra time, and after the work unit has been finished, it\'ll give the extra time back to the other projects (i.e., it won\'t start another CPDN work unit for a long time).

So in the long run, all projects get the appropriate share, but in the short term some may temporarily get more or less.

If you want to find out more, search for LTD (long term debt) on the boinc boards or in the documentation, or read the other posts in this thread.


I'm a volunteer and my views are my own.
News and Announcements and FAQ
ID: 29368 · Report as offensive     Reply Quote
Craig W Post

Send message
Joined: 24 Nov 05
Posts: 2
Credit: 2,606,078
RAC: 0
Message 29376 - Posted: 29 Jun 2007, 22:19:04 UTC - in response to Message 29368.  



Sorry about your original post, the database had to be restored from backup tapes, and some recent posts were lost.

It\'s not a bug, nor is it anything to do with the CPDN project itself, but with the Boinc manager (which is behaving \'as designed\').

Boinc is designed to look at the deadlines for individual workunits, and will bump up their priority if the deadline looks like it will be missed. It keeps track of the extra time, and after the work unit has been finished, it\'ll give the extra time back to the other projects (i.e., it won\'t start another CPDN work unit for a long time).

So in the long run, all projects get the appropriate share, but in the short term some may temporarily get more or less.

If you want to find out more, search for LTD (long term debt) on the boinc boards or in the documentation, or read the other posts in this thread.


No this is BS. Climate Prediction took all my system time for 2 months and completely kept SETI, Einstein, and ROSSETTA from running. This is BS. I don\'t care if in the \"LONG\" run everything gets a appropriate share if the \"LONG\" run is 2 months and running. SETI, EINSTEIN, and ROSSETTA share the system without any problems so there is a problem with either Climate Prediction or BOINC on how they calculate the correct amount of time required for its goals. If BOINC is working as designed then someone has screwed up Climate Predictions goals so that BOINC is misbehaving. Climate Prediciton worked just fine up to 2+ months ago. I will not allow Climate Prediction to take the system over for multiple months! Climate Prediction is now off of my machine until this is goal issue has been corrected!
ID: 29376 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 29378 - Posted: 29 Jun 2007, 22:37:29 UTC


One way of \'fixing the goal\', is to adjust the Result duration correction factor for each computer that has become biased in favour of one project.

But you\'ve saved us the work of explaining it to you.

I hope that you have better luck with the other projects.

ID: 29378 · Report as offensive     Reply Quote
EclipseHA

Send message
Joined: 28 Aug 04
Posts: 42
Credit: 1,443,857
RAC: 0
Message 29555 - Posted: 16 Jul 2007, 2:33:14 UTC

On my Linux box, CP has been the only running project for weeks now.

I\'ve zapped the long term/short term debts, reset the \"correction factor\" but still only CP!

(the resource share for CP is much lower than what I use for other projects, as CP doesn\'t have a fixed deadline....)

This is crazy!

Something has been lost in the BOINC scheduler, in that it used to have a specific \"case\" for CPDN, where deadlines didn\'t throw it into panic mode for CPDN. It could be the CPDN cruncher changed something too, as I noticed the \"CPDN\" centric behavior not long after getting a new WU.

For me, this is ONLY happening on my Linux box.. The windoz boxes are load leveling between projects just fine.

Les... Please in the future try running the project (RAC=0 I note) before you offer stock answers. Seems others are seeing this \"CPDN\" centric crunching, and all we get is \"that\'s the way BOINC works\". I\'d kind of expect a forum moderator to be running the software. Be a help and not a hinerence to others looking for a fix.

Let me know if anyone connected with CPDN wants additional information from me to help resolve this problem.
ID: 29555 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 29556 - Posted: 16 Jul 2007, 3:29:48 UTC


Azwoody

I\'ve already explained why I\'m not crunching on the main site here.
And I DON\'T crunch anything except climate models, so I wouldn\'t see the supposed problem anyway.
The new Intel chips should be out in a week or two, and I hope to set up 2 more machines; one for APS, and the other for SAP. When SAP finishes, THEN I\'ll be back here. With only a single, long, climate model on it, so RAC will still be meaningless.

In the meantime, someone else can answer these posts.

ID: 29556 · Report as offensive     Reply Quote
1 · 2 · 3 · Next

Message boards : Number crunching : Why is Climate Prediction grabbing all the CPU time

©2024 cpdn.org