climateprediction.net (CPDN) home page
Thread 'Tasks in progress'

Thread 'Tasks in progress'

Message boards : Cafe CPDN : Tasks in progress
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 53191 - Posted: 5 Jan 2016, 6:00:19 UTC

At the time that the project ran out of new work units just before Christmas, the number in progress was just over 92,000. It's now just under 82,000.
So the stock piles are being eaten up, which is a good thing for researchers who want their data.

ID: 53191 · Report as offensive     Reply Quote
bernard_ivo

Send message
Joined: 18 Jul 13
Posts: 438
Credit: 25,620,508
RAC: 4,981
Message 53192 - Posted: 5 Jan 2016, 13:10:41 UTC - in response to Message 53191.  

Hi Les,
I was wondering why the return pace is so slow? I was expecting after 10-12 days after the last batch was sent most computers to finish with all CPDN work, hence reduce the number in progress fast. But I guess most crunchers allocate some time to CPDN tasks and they will end much later than the longest model run at 24/7.
ID: 53192 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,004,017
RAC: 21,574
Message 53193 - Posted: 5 Jan 2016, 14:16:17 UTC - in response to Message 53192.  

Many have slow computers - it needs a new hd at the moment but I have completed tasks on a very slow netbook. Also many do not crunch 24/7. I have one quad core that does but my desktop machine is switched off when not in use. The two tasks it is running are currently 1/3 of the way through. Tasks were sent on 4th December. Sadly my computer that is running 24/7 is out of work at the moment. I spotted a task on service status the other day and did a project update but got a permanent http error on it. So I am now waiting to see whether windows tasks to run under WINE or Linux tasks arrive first.
ID: 53193 · Report as offensive     Reply Quote
ProfileJIM

Send message
Joined: 31 Dec 07
Posts: 1152
Credit: 22,363,583
RAC: 5,022
Message 53194 - Posted: 5 Jan 2016, 14:39:13 UTC
Last modified: 5 Jan 2016, 14:40:48 UTC

Dave is right. We don�t all have 3.5 GHz or 4 GHz desktop machines, sll laptops. I run CPDN on 3 machines, one with a 2.66 GHz I5 processor with 8 GB�s of RAM, one with a 2.5 GHz I5 processor with 4GB�s of RAM and the third an old machine with a 2.2 GHz Core2duo processor with 4 GB�s of RAM. These take quite a bit longer to finish a task than one of faster desktop machines.
ID: 53194 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,004,017
RAC: 21,574
Message 53195 - Posted: 5 Jan 2016, 14:59:32 UTC - in response to Message 53194.  

We don�t all have 3.5 GHz or 4 GHz desktop machines,


The netbook is 12.GHz with 2G of ram.
ID: 53195 · Report as offensive     Reply Quote
bernard_ivo

Send message
Joined: 18 Jul 13
Posts: 438
Credit: 25,620,508
RAC: 4,981
Message 53196 - Posted: 5 Jan 2016, 15:28:21 UTC - in response to Message 53193.  

So my guess was almost correct. Not running 24/7 and shared project resources might be the leading reason though as most of my machines are Core2Duo's and some even run at 1.6GHz, but they run 24/7 exclusively for CPDN. I got recently a machine with i5 2520M, so nowhere near 3-4 GHz. A Devil's Canyon is still a dream.

Happy new year fellow crunchers
ID: 53196 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 53197 - Posted: 5 Jan 2016, 19:44:28 UTC

Hi Bernard

This thread expanded a bit while I was sleeping. :)

My guess for the long return times is that a lot of people still treat the BOINC deadline as the time when the project expects to get results back, and just take their time.
Plus I've seen a few computers with lots more tasks "in progress" than the number of cpu cores. (Hence my use of the term "stock piles".)

ID: 53197 · Report as offensive     Reply Quote
bernard_ivo

Send message
Joined: 18 Jul 13
Posts: 438
Credit: 25,620,508
RAC: 4,981
Message 53198 - Posted: 5 Jan 2016, 19:56:55 UTC - in response to Message 53197.  

Hi Les,
in fact I also have 3 tasks in progress that has been aborted or otherwise cancelled and no longer crunched, but will die out in 2016 and one in 2023. As for more tasks than CPU cores I guess this is a BOINC setting (or project specific) that allows users to stock pile few tasks not to get starved, isn't it?
ID: 53198 · Report as offensive     Reply Quote
noderaser
Avatar

Send message
Joined: 17 Nov 04
Posts: 16
Credit: 1,938,125
RAC: 2,007
Message 53199 - Posted: 6 Jan 2016, 6:28:40 UTC - in response to Message 53198.  

in fact I also have 3 tasks in progress that has been aborted or otherwise cancelled and no longer crunched, but will die out in 2016 and one in 2023.

Is that a typo? I thought the deadlines for the longest tasks were one year out...?

I must admit, when I can get CPDN units on my less-often-used computers, I eat up a few units there because they're not on frequently enough to get tasks back to projects that have 1-2 week deadlines. But, a year is usually more than enough :P Hope I'm not causing too much trouble there.
Click Here to see My Detailed BOINC Stats
ID: 53199 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 53200 - Posted: 6 Jan 2016, 7:06:56 UTC - in response to Message 53199.  

I think that the very long deadline was an attempt to get around a BOINC problem.
I don't think that they should actually run.

1 year is not the "results wanted by" date. It's a BOINC deadline that has to be worked around, for people running cpdn stuff very slowly.

At present, if a result that's of particular interest hasn't been returned in a timely fashion, a new task can be created, same data set, different name. Then the original task will be ignored.

But I've been thinking about this ...


ID: 53200 · Report as offensive     Reply Quote
bernard_ivo

Send message
Joined: 18 Jul 13
Posts: 438
Credit: 25,620,508
RAC: 4,981
Message 53202 - Posted: 6 Jan 2016, 8:50:02 UTC - in response to Message 53199.  

Is that a typo? I thought the deadlines for the longest tasks were one year out...? .


Nope, here is the task until 2023. The other two are as Les said within a year.

So noderaser how do you manage to get more tasks than CPU cores? Where can one set such a preference?
ID: 53202 · Report as offensive     Reply Quote
Kevin

Send message
Joined: 5 Jul 09
Posts: 63
Credit: 6,091,274
RAC: 0
Message 53203 - Posted: 6 Jan 2016, 8:57:39 UTC - in response to Message 53200.  

I think that the very long deadline was an attempt to get around a BOINC problem.
I don't think that they should actually run.




A longer deadline is handy, I only run CPND as a secondary project on my CPU, as it is only a secondary project I normally try to only allocate 1 or 2 cores to CPND, unfortunately sometimes I have to pause these for several days at a time.

Sometimes I find that I get clobbered when work appears here, nothing for a long while then crash more work than I need. So sometimes some work units may not be processed immediately, they may be subject to some delay.

I am not asking for a full year but 3 to 4 months would a comfortable deadline.




Kevin
ID: 53203 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 53204 - Posted: 6 Jan 2016, 9:28:21 UTC - in response to Message 53203.  

I am not asking for a full year but 3 to 4 months would a comfortable deadline.


Kevin

That's my thinking too.
The 1 year was from 9-10 years ago, when the models took 3-4 months to run. Now they take 1-3 weeks. With at least one batch that ran for 1 day. But that may have been beta.

Perhaps in 2 stages - first to 6 months for a while, to get peoples attention.
Then to 3-4 months, as you said. Although some of the models are getting bigger.
It can always be relaxed a bit if this tend continues.

ID: 53204 · Report as offensive     Reply Quote
ProfileAlan K

Send message
Joined: 22 Feb 06
Posts: 491
Credit: 30,975,898
RAC: 14,500
Message 53206 - Posted: 6 Jan 2016, 11:24:51 UTC - in response to Message 53204.  

It's almost a shame that there is no easy way to diferentiate between "short" and "long" models so that the short ones can be processed by the slower computers leaving the faster machines to cope with the longer - and possibly more demanding - longer models. Mearly a thought.
ID: 53206 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,004,017
RAC: 21,574
Message 53207 - Posted: 6 Jan 2016, 12:20:15 UTC

It's almost a shame that there is no easy way to diferentiate between "short" and "long" models so that the short ones can be processed by the slower computers leaving the faster machines to cope with the longer - and possibly more demanding - longer models.


Agreed, though, at the moment I will take any tasks I can get long or short!
ID: 53207 · Report as offensive     Reply Quote
ProfileJIM

Send message
Joined: 31 Dec 07
Posts: 1152
Credit: 22,363,583
RAC: 5,022
Message 53209 - Posted: 6 Jan 2016, 16:48:49 UTC

One problem is that when Boinc tries to run short deadline projects along side CPDN Boinc has the tendency to pause all CPDN work and run the other projects first to avoid missing there deadlines. This means that CPDN tasks sit and wait despite the fact that CPDN has a 600 share and other projects have a 100 share.

ID: 53209 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 53210 - Posted: 6 Jan 2016, 17:39:41 UTC - in response to Message 53209.  
Last modified: 6 Jan 2016, 17:46:18 UTC

It helps to keep a short buffer. If you load up on projects, BOINC has no choice but to try and get them done on time. It has no other way to prioritize them.

Also, I set "Switch between tasks" to 1440 minutes (24 hours), and it should perhaps be longer. How much good it does may depend on the projects running, and it won't extend the due dates in any case.
ID: 53210 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 53211 - Posted: 6 Jan 2016, 21:35:26 UTC - in response to Message 53209.  

Jim

That IS the reason for the year long deadline, but there's too many people now that are taking advantage of it.
If I ordered an item for delivery in the next few days, I wouldn't be happy if it took a year to arrive. And now that our "customers" are professional climate physicists, I feel that they should get a better deal. Even if it means losing a lot of slow crunchers, with slow (relative), computers. :(

And thanks everyone for all of the feed back. I didn't expect it, but it's welcome. :(


ID: 53211 · Report as offensive     Reply Quote
Kevin

Send message
Joined: 5 Jul 09
Posts: 63
Credit: 6,091,274
RAC: 0
Message 53212 - Posted: 7 Jan 2016, 5:32:04 UTC - in response to Message 53210.  

It helps to keep a short buffer. If you load up on projects, BOINC has no choice but to try and get them done on time. It has no other way to prioritize them.


Sometimes buffer length settings are dictated by another project and when you are not expecting to get anything from CPND and in a single request you get 4 tasks (_1 or_2's) as happened on the 4th of this month to me, yes they will all be processed, but it may take a month to do so.

It might be an idea to ask if buffer settings per project could be set in Boinc, but that would have to be requested elsewere. This would also be very useful when getting work from Einstein which has short deadline work. I know that we can set NNT per project but sometimes I do not remember untill it is too late.

One thing that has got to be taken into account, not all people that are doing CPND work are doing it as their main project, some of us have other projects, mine are SETI and Einstein.


Kevin
ID: 53212 · Report as offensive     Reply Quote
noderaser
Avatar

Send message
Joined: 17 Nov 04
Posts: 16
Credit: 1,938,125
RAC: 2,007
Message 53213 - Posted: 7 Jan 2016, 7:12:59 UTC - in response to Message 53202.  

So noderaser how do you manage to get more tasks than CPU cores? Where can one set such a preference?

I don't, I just try to get on task per core on those machines which are used less often. Unfortunately, that can be pretty hard due to the nature of the project and the low flow of tasks.

I know the long deadlines must be rough for the researchers as well, it's certainly a long waiting game for results. I think a deadline of even six months would work for my purposes, if that ended up being the new deadline.
Click Here to see My Detailed BOINC Stats
ID: 53213 · Report as offensive     Reply Quote
1 · 2 · Next

Message boards : Cafe CPDN : Tasks in progress

©2024 cpdn.org