climateprediction.net (CPDN) home page
Thread 'Weather At Home 2 (wah2) (region independent) v8.29 - very short deadline & mismatch for total time calculation'

Thread 'Weather At Home 2 (wah2) (region independent) v8.29 - very short deadline & mismatch for total time calculation'

Message boards : Number crunching : Weather At Home 2 (wah2) (region independent) v8.29 - very short deadline & mismatch for total time calculation
Message board moderation

To post messages, you must log in.

AuthorMessage
ProfileFriedrich S.

Send message
Joined: 22 Jan 05
Posts: 45
Credit: 4,617,986
RAC: 1,077
Message 70862 - Posted: 24 Apr 2024, 16:25:37 UTC

Hello,

I used to run CP for a long time. The WU always had very generous deadlines, usually way beyond what my machines would need. But having more time never is a problem.
However, the recent batches of Windows WU have a really tight deadline and need much more time that shown.

I am not sure how BOINC calculates the total time for each WU, but when I use the values from the BOINC client (progress and CPU time used so far), Excel shows a total CPU-time requirement of more than 45 days, compared to the 16-17 days the BOINC client claims. E.g.
Progress: 8.915%
CPU elapsed: 4d 01:28:00
Remaining (acc. to BOINC client): 12d 17:48:15
Remaining (acc. to Excel): 41d 11:49:18

I don't see such a mismatch with other projects.

I pretty much would have to run my computer 24/7 to make the deadline.
Is that the new policy/requirement? If this is the case, I would have to quit CP after so many years because I don't want to waste WU that I cannot finish in time.
I run BOINC and CP on my home computer which usually runs just between 4-8 hours per day. That always was sufficient to finish CP WU in time, but not with these recent WU.

Any feedback from the CP project team would be helpful,
- whether it is reasonable to run these WU on my computer or not.
- why there is such a mismatch between the time requirement as calculated by the BOINC client vs. Excel.

Thank your very much!

Friedrich
I love CPDN!
--
ID: 70862 · Report as offensive     Reply Quote
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 70863 - Posted: 24 Apr 2024, 17:15:03 UTC - in response to Message 70862.  

It looks like you are running 10 tasks at a time on your i5-10400 which has 6 cores, 12 threads. These models are more compute intensive than a lot of other boinc projects. Generally, we recommend running no more tasks than the number of cores on our processor. The Hyperthreading does not help total throughput of work with these models. If you look at the sec/TS on your trickle listing, your i5-10400 is only getting 12+ sec/TS. If you were to run only half the number of tasks at a time (5), it would be well below 5 sec/TS, possibly below 3 sec/TS so considerably increasing throughput.

As for boinc time estimates, that really doesn't work well with this project when hyperthreads are used.

Back in the very olden days at cpdn, using hyperthreading on a processor could help a little in total throughput. The more advanced models over the last several years do not benefit from it at all.
ID: 70863 · Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 70864 - Posted: 24 Apr 2024, 17:18:30 UTC - in response to Message 70862.  
Last modified: 24 Apr 2024, 17:19:14 UTC

However, the recent batches of Windows WU have a really tight deadline and need much more time that shown.


I do not have this problem even though I have a pipsqueak computer running Windows(10).
Computer 1512658
CPU type 	GenuineIntel
11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz [Family 6 Model 140 Stepping 1]
Number of processors 	8
Operating System 	Microsoft Windows 10
Core x64 Edition, (10.00.19045.00)
BOINC version 	7.24.1
Memory 	15.64 GB
Cache 	256 KB

My machine does run 24/7 running CPDN, Denis,WCG,Rosetta, and Einstein. I allow 7 cores to run Boinc tasks. It has been running two CPDN 1015 tasks at the moment. It originally predicted it would take about 17 1/2 days to complete each one, but they are running faster, so now it says it is has run 9 days and is 75% done with a little over 4 days to go. The deadline is June 24. I expect them to be done by the end of April with almost two months to spare.
ID: 70864 · Report as offensive     Reply Quote
ProfileFriedrich S.

Send message
Joined: 22 Jan 05
Posts: 45
Credit: 4,617,986
RAC: 1,077
Message 70865 - Posted: 24 Apr 2024, 17:39:19 UTC - in response to Message 70863.  

If you were to run only half the number of tasks at a time (5), it would be well below 5 sec/TS, possibly below 3 sec/TS so considerably increasing throughput.

Thank you very much for your insights!
I paused half of the tasks and limited the number of CP tasks for the future.

Have a wonderful day!

Friedrich
I love CPDN!
--
ID: 70865 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1049
Credit: 16,578,380
RAC: 15,009
Message 70866 - Posted: 24 Apr 2024, 17:43:05 UTC - in response to Message 70863.  
Last modified: 24 Apr 2024, 17:44:01 UTC

The original task deadlines used to be a year, which had not been changed since the days of the very long climate models. They are no longer used, instead weather@home is a much shorter forecast.
Unfortunately we saw alot of task hogging with those long deadlines so they have recently been shortened. I believe it's set to 70 days a task but with a 20(?) day grace period. What that means is a client has 70 days to start the task otherwise it gets bounced to someone else. Once started the deadline is then 70+20 days from the date the host computer received the task.
I'm not near a computer so I can't check those numbers but they should be mostly correct.
As noted earlier, don't overload the CPU, stick to one CPDN task per core, not per thread. There are only one set of floating point units per core.
---
CPDN Visiting Scientist
ID: 70866 · Report as offensive     Reply Quote
ProfileFriedrich S.

Send message
Joined: 22 Jan 05
Posts: 45
Credit: 4,617,986
RAC: 1,077
Message 70867 - Posted: 24 Apr 2024, 17:49:52 UTC - in response to Message 70866.  

Once started the deadline is then 70+20 days from the date the host computer received the task.

Thank you very much for your insights!
I will see to it that I do finish the tasks within that time frame.

Have a wonderful day!

Friedrich
I love CPDN!
--
ID: 70867 · Report as offensive     Reply Quote
noderaser
Avatar

Send message
Joined: 17 Nov 04
Posts: 16
Credit: 1,939,803
RAC: 823
Message 70906 - Posted: 6 May 2024, 17:47:17 UTC - in response to Message 70866.  

I noticed this behavior on one of my machines; the deadline reported in BOINC manager has already passed, but the one listed on the tasks page still shows into the future. Is this a onetime "bonus", or does it keep giving you additional time if you are still (somewhat) actively working on the task? I used to run my machines 24/7, but don't (on purpose) anymore.
Click Here to see My Detailed BOINC Stats
ID: 70906 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1049
Credit: 16,578,380
RAC: 15,009
Message 70909 - Posted: 6 May 2024, 22:59:50 UTC - in response to Message 70906.  

It's a grace period. It was originally implemented to deal with upload servers that develop problems preventing workunits from being uploaded.

The server will not keep adding to the grace period. If the task is not completed by the end of the deadline+grace time the task will be reassigned to another host and the current host gets credit for work done.
---
CPDN Visiting Scientist
ID: 70909 · Report as offensive     Reply Quote

Message boards : Number crunching : Weather At Home 2 (wah2) (region independent) v8.29 - very short deadline & mismatch for total time calculation

©2024 cpdn.org