climateprediction.net (CPDN) home page
Thread 'How to limit number of WUs'

Thread 'How to limit number of WUs'

Questions and Answers : Preferences : How to limit number of WUs
Message board moderation

To post messages, you must log in.

AuthorMessage
flensr

Send message
Joined: 17 Oct 18
Posts: 8
Credit: 1,808,770
RAC: 3,204
Message 62005 - Posted: 22 Jan 2020, 16:32:08 UTC

Hello,

I have a problem with managing the number of WUs I get from CPDN. I will get no work units for a month, then suddenly I get 8 (I have 7 cpu cores dedicated to CPDN. Since these take a few days to complete, this makes it so my other projects won't send WUs for several days. I set the CPDN priority to a very low number (under 2%) but this still happens. No WUs for a long time, then suddenly way too many.

Is there any way to get it to send just one or two WUs when it does send work? I've also tried changing global settings to have it only get work for half a day and accept up to one more half day of work, but it still grabs way too many WUs at a time.

Thanks for any help!
ID: 62005 · Report as offensive     Reply Quote
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 62006 - Posted: 22 Jan 2020, 18:04:05 UTC - in response to Message 62005.  

This may not be perfectly what you want, but it can restrict the number of tasks from a given project running at the same time.

https://boinc.berkeley.edu/wiki/Client_configuration#Project-level_configuration

and then set <project_max_concurrent> to the max number of cpdn jobs you would want to run at one time. It might still download the same number of tasks as it currently does, but it would be able to run tasks from other projects at the same time. The documentation is not the greatest there, and I haven't used this method so am unsure as to how many of the options mentioned really need to be in that app_config.xml file. So you may have to go elsewhere for answers to specific questions about the requirements in that file.
ID: 62006 · Report as offensive     Reply Quote
Aurum
Avatar

Send message
Joined: 15 Jul 17
Posts: 99
Credit: 18,701,746
RAC: 318
Message 64239 - Posted: 31 Jul 2021, 16:34:54 UTC - in response to Message 62006.  

CP is in dire need of some upgrades. Preferences should allow us to check boxes to opt-in to the projects we want to run. It should also have a way to designate the maximum number of WUs to send a computer at any time.
Now the way to control work flow is to Abort unwanted excess WUs. Seems like CP would want to minimize that kind of wastefulness and lighten demands on its server. An app_config.xml just helps organize which WUs to Abort.
<app_config>
<app>
<name>hadcm3s</name>
<!-- UK Met Office HadCM3 short, 3 days -->
<max_concurrent>1</max_concurrent>
</app>
<app>
<name>hadam4</name>
<!-- UK Met Office HadAM4 at N144 resolution, 4 days -->
<max_concurrent>1</max_concurrent>
</app>
<app>
<name>hadsm4</name>
<!-- UK Met Office HadSM4 at N144 resolution, 4 days -->
<max_concurrent>1</max_concurrent>
</app>
<app>
<name>hadam4h</name>
<!-- UK Met Office HadAM4 at N216 resolution, 14 days -->
<max_concurrent>1</max_concurrent>
</app>
<project_max_concurrent>3</project_max_concurrent>
</app_config>
ID: 64239 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4541
Credit: 19,039,635
RAC: 18,944
Message 64240 - Posted: 31 Jul 2021, 18:49:31 UTC - in response to Message 64239.  

Moderators have asked the project about allowing choice as to which types of task they receive and been told that they discussed this and decided against it. Given the length of work units and associated deadlines, I don't see the problem with using app_config to limit the number of tasks from CPDN or any other project running at one time but then I don't often run other projects except when there is a lack of work here. Not really sure why you need to abort tasks.

When the OpenIFS tasks make it to the main site may be the best chance of revisiting the option of choosing which types of tasks to download.
ID: 64240 · Report as offensive     Reply Quote
Weaver

Send message
Joined: 8 Mar 24
Posts: 1
Credit: 111,168
RAC: 488
Message 70739 - Posted: 4 Apr 2024, 20:51:55 UTC - in response to Message 64239.  

Is there any way to get it to send just one or two WUs when it does send work?

It should also have a way to designate the maximum number of WUs to send a computer at any time.

It appears this is now possible to configure in the web interface by visiting Project -> Preferences and setting "Max # jobs [for this project]" to the desired number.
ID: 70739 · Report as offensive     Reply Quote

Questions and Answers : Preferences : How to limit number of WUs

©2024 cpdn.org