climateprediction.net (CPDN) home page
Thread 'BOINC - Time sharing between projects'

Thread 'BOINC - Time sharing between projects'

Message boards : Number crunching : BOINC - Time sharing between projects
Message board moderation

To post messages, you must log in.

AuthorMessage
old_user3861

Send message
Joined: 31 Aug 04
Posts: 31
Credit: 1,768,065
RAC: 0
Message 3692 - Posted: 10 Sep 2004, 8:51:47 UTC
Last modified: 10 Sep 2004, 8:53:55 UTC

I've been logging the output from the Linux CLI BOINC client to text files to get a clear view of what 's going on. To simplify matters, I adjusted the resource sharing between SETI and CPDN to 50/50 (actually 100/100)....and updated the preferences on all systems.

(Note that a SETI WU takes 2:50 to process to completion on the AMD Athlon XP 2800+ system examined. )

Linux client: 4.05

Despite the processing being set to 50/50 SETI seems to get most (2:1) of the time...and I see why.

When I first start the client, SETI may begin and run for one hour - if a WU does not finish.

If SETI finishes a WU within the hour it is alotted for processing, it will start anotehr WU....and go for one hour before being pre-empted.

The CPDN model then processes for exactly one hour.

The SETI WU will resume, and EVER AFTER will process for TWO hours for each ONE CPDN processes.

Bear in mind that if the SETI WU completes within that TWO hours, it will reset the clock form that point for TWO hours.

Which explains why CPDN has been making snail-like progress despite the projects being set at 50/50 and the -update_prefs have been reset.

Summary: Ratio set 50/50 (100 / 100)

1. The 50 / 50 will be maintained for at least the first cycle of pre-emption (subject to a SETI WU completing and resetting the clock in it's time).

2. All subsequent cycles will 2:1 in favour of SETI (subject to a SETI WU completing and resetting the the clock on its time.)

Possible Solution: To allow for effect of the clock resets on completed / started SETI WUs, a CPDN project supporter may wish to set preferences to SETI 50 / CPDN 100 and see if this more closely approximates 50/50 on the system concerend. This won't entirely address the problem of the SETI project receiving 2;1 of time after the first cyucle of pre-emption. It may be necessary to try SETI 33 / CPDN 67 in order to approach 50/50 in reality.

The slower the system, the less likely this problem is to be observed as SETI will be less frequently complete a work unit in a processing period, thus not resetting the 'share' clock.
<a href="http://www.boinc.dk/index.php?page=user_statistics&amp;project=cpdn&amp;userid=3861"><img border="0" height="68" src="http://3861.cpdn.sig.boinc.dk?142"></a>
ID: 3692 · Report as offensive     Reply Quote
ProfileKeck_Komputers
Avatar

Send message
Joined: 5 Aug 04
Posts: 426
Credit: 2,426,069
RAC: 0
Message 3700 - Posted: 10 Sep 2004, 9:23:33 UTC

Good observations. In the short term adjusting resource shares should help with this, however this kind of big will be fixed eventually. So long term it may be better to leave the settings as they are.
<br>John Keck -- BOINCing since 2002/12/08 -- <a href="http://www.boinc.dk/index.php?page=user_statistics&amp;project=cpdn&amp;userid=191"><img border="0" height="80" src="http://191.cpdn.sig.boinc.dk?188"></a>
ID: 3700 · Report as offensive     Reply Quote
ProfileThyme Lawn
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1283
Credit: 15,824,334
RAC: 0
Message 3705 - Posted: 10 Sep 2004, 9:39:34 UTC

Great observation, Steve.

I guess this means that if a project processes jobs in under an hour it could potentially never get pre-empted ...

<a href="http://www.teampicard.net"><img src="http://www.teampicard.net/templates/fisubice/images/phpbb2_logo.jpg"></a><a href="http://climateapps2.oucs.ox.ac.uk/cpdnboinc/team_display.php?teamid=3">Join us here</a>
ID: 3705 · Report as offensive     Reply Quote
Profileold_user156
Avatar

Send message
Joined: 5 Aug 04
Posts: 186
Credit: 1,612,182
RAC: 0
Message 3717 - Posted: 10 Sep 2004, 11:47:31 UTC
Last modified: 10 Sep 2004, 11:48:25 UTC

This behaviour appears to have altered with BOINC v4.08 <i>Alpha</i> - even though my resource share was 99:1 CPDN:SETI, 'Alison' was processing mainly SETI in a similar fashion to Steve's machine - I upgraded to BOINC v4.08 and she hasn't processed any SETI since then, even though she has a SETI unit 'Paused' and hanging at 99.2% completed.

('Alison' <i>should</i> process SETI work_units just within their deadline at those settings.)

<a href="http://www.nmvs.dsl.pipex.com/"><img src="http://boinc.mundayweb.com/cpdn/stats.php?userID=6&amp;team=off&amp;trans=off"></a>
ID: 3717 · Report as offensive     Reply Quote
John McLeod VII
Avatar

Send message
Joined: 5 Aug 04
Posts: 172
Credit: 4,023,611
RAC: 0
Message 3727 - Posted: 10 Sep 2004, 13:03:07 UTC

What I and a couple of other people had noticed was that under 4.05, the resource sharing seemed to be affected by the number of WUs on the system. If you had one CPDN WU and 6 S@H WUs on your system and the resource share was 1:1 the actual share was 6:1 in favor of S@H. This seems to be fixed in 4.08. However, if a project runs out of work -- even for an instant, the debt is reset to 0, even if the debt was negative.
<a href="http://www.boinc.dk/index.php?page=user_statistics&amp;project=cpdn&amp;userid=13"><img border="0" height="80" src="http://13.cpdn.sig.boinc.dk?188"></a>
ID: 3727 · Report as offensive     Reply Quote
old_user3861

Send message
Joined: 31 Aug 04
Posts: 31
Credit: 1,768,065
RAC: 0
Message 3730 - Posted: 10 Sep 2004, 13:16:27 UTC - in response to Message 3727.  

&gt; What I and a couple of other people had noticed was that under 4.05, the
&gt; resource sharing seemed to be affected by the number of WUs on the system. If
&gt; you had one CPDN WU and 6 S@H WUs on your system and the resource share was
&gt; 1:1 the actual share was 6:1 in favor of S@H. This seems to be fixed in 4.08.
&gt; However, if a project runs out of work -- even for an instant, the debt is
&gt; reset to 0, even if the debt was negative.

I have not seen that. I run a 3 days cache and that means roughly 20 SETI WUs waiting on 2 of my systems. The number is lower on a slower system and higher on the faster system. I'll check the other system logs and see if the time varies as you suggest.
<a href="http://www.boinc.dk/index.php?page=user_statistics&amp;project=cpdn&amp;userid=3861"><img border="0" height="68" src="http://3861.cpdn.sig.boinc.dk?142"></a>
ID: 3730 · Report as offensive     Reply Quote
old_user3861

Send message
Joined: 31 Aug 04
Posts: 31
Credit: 1,768,065
RAC: 0
Message 3735 - Posted: 10 Sep 2004, 13:35:29 UTC
Last modified: 10 Sep 2004, 13:38:16 UTC

Well...I set SETI to 50 and CPDN to 100 and refreshed the preferences - and eye-balled the relevant account file to ensure they had been read.

But the system still spends a minimum of 2 hours on SETI and exactly one hour on CPDN - despite being supposed to do 1 hour on SETI and two on CPDN.

So it appears the preferences have no affect whatever.

It's a minimum of 2:1 in favour of SETI....and often more like 3:1 if SETI finishes a unit and resets the clock at THAT point for two more hours.

I seem to have no say in the matter. The ratio is close to 3:1 in favour of SETI due to the wu-completion timer reset issue.

<a href="http://www.boinc.dk/index.php?page=user_statistics&amp;project=cpdn&amp;userid=3861"><img border="0" height="68" src="http://3861.cpdn.sig.boinc.dk?142"></a>
ID: 3735 · Report as offensive     Reply Quote
Profileold_user33

Send message
Joined: 5 Aug 04
Posts: 10
Credit: 54,088
RAC: 0
Message 3843 - Posted: 11 Sep 2004, 7:56:14 UTC

I am sharing CPDN and S@H in the ratio 10:1 on one of omy PCs, but find that almost all the time, Boinc 4.05 is switching between the projects hourly (giving a 50:50 ratio), with S@H very occassionally getting a 2 hour slot.

Looking at the client_state.xml file, I see the CPDN currently has a "debt" of 0 and S@H a "debt" of 799 (S@H has just preempted CPDN). I admit this is a single observation. I have not yet done a longer term study of whether/if this changes.

Once again, as I often find myself thinking that Boinc's behaviour seems bizarre. Does anyone actually know for sure (not just guessing!) what the debt value actually means, how it is calculated, what it's units are, what Boinc's behaviour is for any given values?

Could it be that "debt" is just another of these overly complex mathematical functions like the notion of Recent Average Credit. Are these "bugs" that JM7 refers to just bugs, or have the Boinc Dev team just got bogged down in such complexities that there are too many circumstances when their logic fails. When I worked as a programmer, I was always taught to use the KISS principle - Keep It Simple Stupid. It would seem very simple to me to have a 10:1 ratio that simply gives 10 hrs to the first project and then 1 hr on the other, and then round the loop again?

Sorry if I sound a bit cynical.
ID: 3843 · Report as offensive     Reply Quote
Profileold_user156
Avatar

Send message
Joined: 5 Aug 04
Posts: 186
Credit: 1,612,182
RAC: 0
Message 3904 - Posted: 11 Sep 2004, 14:17:40 UTC

With BOINC v4.08, which is currently in Alpha test, project time sharing seems to work in accordance with my project preferences - I currently have them set 90:10 CPDN:SETI &amp; that's the sort of proportion they're crunching at...


<a href="http://www.nmvs.dsl.pipex.com/"><img src="http://boinc.mundayweb.com/cpdn/stats.php?userID=6&amp;team=off&amp;trans=off"></a>
ID: 3904 · Report as offensive     Reply Quote

Message boards : Number crunching : BOINC - Time sharing between projects

©2024 cpdn.org