climateprediction.net (CPDN) home page
Thread 'Tasks in progress'

Thread 'Tasks in progress'

Message boards : Cafe CPDN : Tasks in progress
Message board moderation

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
ProfileJIM

Send message
Joined: 31 Dec 07
Posts: 1152
Credit: 22,363,583
RAC: 5,022
Message 53214 - Posted: 8 Jan 2016, 4:51:12 UTC
Last modified: 8 Jan 2016, 4:53:20 UTC

It has been mentioned above that it is good to keep the work buffer set short. This might have been true back in the days when there was always work available, but, no more. I find that it is better to set the work buffer at the 10 day maximum. This allows me to store enough tasks when they are availed to see me through the increasingly frequent work droughts. (Like the one we�re in now.)

I like it best when I have at least one task �waiting to start� for each running task. Two tasks is better. And just so you know it doesn�t take me a year to return the results. Not even 3 months. Most downloaded tasks are finished in 6 weeks or less. I don�t think that is too long to wait.

If the climate scientists are in such a big hurry they will just have to raise the money, buy time on supercomputers, and get it all done fast. If they are going to relay on volunteers using their home computers they are going to have to except the fact that most of us are not using 3.5 GHz and 4 GHz speed demons. Most of the work is going to be done on middling in 2 GHz to 2.5 GHz machines. Some even on 1.6 GHz electronic snails. That is the nature of the beast in volunteer distributed computing.
ID: 53214 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 53215 - Posted: 8 Jan 2016, 17:35:39 UTC - in response to Message 53214.  

It has been mentioned above that it is good to keep the work buffer set short. This might have been true back in the days when there was always work available, but, no more. I find that it is better to set the work buffer at the 10 day maximum. This allows me to store enough tasks when they are availed to see me through the increasingly frequent work droughts. (Like the one we�re in now.)

I have never considered it practical to have CPDN as my main project on any machine, due to the sporadic nature of the work. The question is how to accommodate that fact. Trying to buffer enough is one way, but that deprives the scientists of fast returns for no good purpose that I can see. My machines are set up mainly for other projects, and I prefer a short buffer for a variety of reasons. For example, on WCG by returning results in a timely manner you gain the status of a reliable machine and are sent work units that must be completed promptly. And on GPUGrid you get extra bonus points if you return work within a 24-hour time limit. Whether that is of any interest is of course up to you, but for purposes of just providing work in case of outages, I prefer to set up secondary projects rather than use very large buffers. It helps all concerned.
ID: 53215 · Report as offensive     Reply Quote
ProfileJIM

Send message
Joined: 31 Dec 07
Posts: 1152
Credit: 22,363,583
RAC: 5,022
Message 53216 - Posted: 8 Jan 2016, 21:20:36 UTC - in response to Message 53215.  
Last modified: 8 Jan 2016, 21:24:46 UTC

For all practical purposes CPDN is my only project. I only run other projects when there is no climate work. When I started running this project over 10 years ago there was always work.

Speaking of which, it would be nice if someone would throw about 20,000 new tasks in the hopper. I am about 3 days away from having empty cores.
ID: 53216 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 53217 - Posted: 8 Jan 2016, 21:39:14 UTC - in response to Message 53216.  

From 17-10 years ago, the work was just basic research being done in-house at Oxford by pre and post grads.
These days it's being done by professionals external to Oxford, who have their own agenda.

ID: 53217 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 53218 - Posted: 8 Jan 2016, 22:09:31 UTC - in response to Message 53216.  

Speaking of which, it would be nice if someone would throw about 20,000 new tasks in the hopper. I am about 3 days away from having empty cores.


4 empty cores here despite having put wine on machine to give me the option of getting Windows tasks
ID: 53218 · Report as offensive     Reply Quote
ProfileAlan K

Send message
Joined: 22 Feb 06
Posts: 491
Credit: 31,033,074
RAC: 14,759
Message 53225 - Posted: 9 Jan 2016, 19:05:58 UTC - in response to Message 53216.  
Last modified: 9 Jan 2016, 19:06:19 UTC

Largely the same for me running CPDN as a main project with Milky Way at Home as second all the time. Also run some WCG but these seem to be very short models so tend to get a lot at one go and BOINC gives these priority even though it is set to the lowest. Tend to set NNT on this when CPDN has work.
ID: 53225 · Report as offensive     Reply Quote
ProfileJIM

Send message
Joined: 31 Dec 07
Posts: 1152
Credit: 22,363,583
RAC: 5,022
Message 53226 - Posted: 10 Jan 2016, 14:51:24 UTC - in response to Message 53225.  

Largely the same for me running CPDN as a main project with Milky Way at Home as second all the time. Also run some WCG but these seem to be very short models so tend to get a lot at one go and BOINC gives these priority even though it is set to the lowest. Tend to set NNT on this when CPDN has work.


Yes, the problem is the very short deadlines on WCG tasks and the long one for CPDN. Boinc days to itself �I�ve got lots of time to get them done, but, the WCG tasks need to be done right away� and gives them priority as long these tasks are present. Changing the shares doesn�t seem to have any effect. On my machines WCG is set at 100 while CPDN is set at 600. No effect.


ID: 53226 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 53227 - Posted: 10 Jan 2016, 15:54:14 UTC - in response to Message 53226.  

I once proposed on the BOINC forum that they do away with the scheduler entirely (silly idea, I know) and just allow you to set priorities for each CPU core. Then, if Project A was out of work, it would fall back to Project B, etc. The idea was to allow you to control your work, rather than having your preferences disappear into the Black Hole of the scheduler, where they would never be seen again.
ID: 53227 · Report as offensive     Reply Quote
Kevin

Send message
Joined: 5 Jul 09
Posts: 63
Credit: 6,091,274
RAC: 0
Message 53228 - Posted: 10 Jan 2016, 20:20:55 UTC

Just a thought.

Is there anything in the pipeline about ghosts?

The WU's that appear in your tasks list but never appear on your machine.

with such a long deadlines there must be a few in limbo somewhere.


Kevin
ID: 53228 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 53229 - Posted: 10 Jan 2016, 20:51:14 UTC - in response to Message 53228.  

I haven't heard of anything. Could be a bit hard to tell the difference at the project side between them and slow crunchers / stock piled data sets.
I think they are probably just written off as "Oh well", and more of the same created if necessary.

ID: 53229 · Report as offensive     Reply Quote
Kevin

Send message
Joined: 5 Jul 09
Posts: 63
Credit: 6,091,274
RAC: 0
Message 53230 - Posted: 10 Jan 2016, 21:33:21 UTC

Thought so.

A server side cure does put extra loading on the servers, client side most probably don't even notice them anyway.

Just for those that are curious - when you have no work on a machine and a ghost or two listed online a simple detach and then re-attach will make them available for others.


Kevin
ID: 53230 · Report as offensive     Reply Quote
Previous · 1 · 2

Message boards : Cafe CPDN : Tasks in progress

©2024 cpdn.org