I\'ve installed the beta BOINC client this afternoon and am running BOINC on one CPU as a Windows service. I\'ve also left my legacy CPDN setup running.

The hardware here is a dual Xeon 2.66GHz machine (Windows XP Professional SP1) with Hyperthreading on - so I have the potential to run four CPDNs at once using BOINC.


I care somewhat about my machine ranking - and appreciate that BOINC statistics are as yet uncertain. I\'m about 35% into Phase 1 of legacy CPDN at the moment.

For now, I think I\'ll leave things set as they are (if I increase BOINC to multiple CPUs, legacy CPDN throughput will likely drop as I only have two actual CPUs).

Obviously if I decide to discontinue legacy CPDN, I\'ll uncheck the option somewhere in my account that means my CPDN client will finish after the end of the run.


I guess what I\'m after is some ideas as to how the BOINC migration will be made in the end - for which I guess the answer, right now, is \"we haven\'t decided\". Presumably, though, there\'ll come a point after the BOINC public launch where legacy CPDN will be turned off - once machines finish a run, they won\'t be offered another.

A hyper-threading processor counts as 2 processors. If you set your preferences to use a maximum of 3 processors you should be able to run 3 BOINC clients and your legacy CPDN at the same time.

> A hyper-threading processor counts as 2 processors. If you set your
> preferences to use a maximum of 3 processors you should be able to run 3 BOINC
> clients and your legacy CPDN at the same time.

True - though remember that an HT processor only has one set of execution resources. Add to that the lack of use of explicit processor affinity (apparently) by BOINC and by legacy CPDN, and I fear any more than one BOINC model executing has the potential to hurt legacy CPDN throughout.

> True - though remember that an HT processor only has one set of execution
> resources. Add to that the lack of use of explicit processor affinity
> (apparently) by BOINC and by legacy CPDN, and I fear any more than one BOINC
> model executing has the potential to hurt legacy CPDN throughout.

I did some HT testing during the alpha trial. Running the BOINC client didn't seem to affect the performance of my legacy CPDN.

<a href="http://climateapps2.oucs.ox.ac.uk/cpdnboinc/team_display.php?teamid=3"><img src="http://www.teampicard.net/templates/fisubice/images/phpbb2_logo.jpg"></a>

actually, the "classic CPDN" (Windows client 2.2.29/3.0.1) will be around a long time (years) as that is what the Open University course and other teaching materials is based on etc. Future models and development will be under the BOINC platform.

Hyperthreading seems to be doing OK, we have a new laptop (the one I used for our recent "Open Day") and it can run two models simultaneously fairly well (a hyperthreaded Pentium 2.8GHz; getting 3.9sec per timestep).

&gt; Hyperthreading seems to be doing OK

Yes, I'm also running CPDN(BOINC) on a P4 3 Ghz HT; it looks fine (running 29 hours now). It crunches actually two models simultaneously

&gt; hyperthreaded Pentium 2.8GHz; getting 3.9sec per timestep).

On my "trickles-Page", it says average 2.89 Is this 2.89 sec per timestep ? If so, why is my 3 GHz machine so much faster than your 2.8GHz ? (Okay, it has 1 GB memory, but does this make such a big difference ?)

&gt; On my "trickles-Page", it says average 2.89 Is this 2.89 sec per timestep ? If
&gt; so, why is my 3 GHz machine so much faster than your 2.8GHz ? (Okay, it has 1
&gt; GB memory, but does this make such a big difference ?)

well this is a laptop 2.8GHz, so probably not as fast as a desktop 3GHz overall when you factor in memory speed, hard drive speed etc. Memory speed is a major factor in running this model (a huge amount of memory I/O).

&gt; well this is a laptop 2.8GHz, so probably not as fast as a desktop 3GHz
&gt; overall when you factor in memory speed, hard drive speed etc. Memory speed
&gt; is a major factor in running this model (a huge amount of memory I/O).

Sorry, yeah, laptops have become so fast, that with "normal user usage" there seems to be no big difference to normal workstations, but with DC the differences come back to the eyes ...

The following may be of interest to this thread ...

I have, for some time, been running CPDN on three dual processor machines - two are Tyan Tiger MPXs running MP2800s and the third is a Supermicro board running P4 Xeons at 3.2 GHz. In order to maximise instances of CPDN, I have, up to now, run two instances of CPDN classic per machine - one native and one inside a VMWare shell running Windows 2000.

I have now maximised the use of my P4 Xeon by adding CPDN BOINC beta instances on the other two "processors" not previously used. This machine is giving me the following s/ts for the various instances of CPDN (as measured by CPFarmView for the instances of CPDN classic):
- CPDN classic (native) 5.26 s/ts
- CPDN classic (VMWare) 5.45 s/ts
- CPDN BOINC 1 3.82 s/ts
- CPDN BOINC 2 3.78 s/ts

It would appear that, in some fashion, the BOINC CPDN client is more efficient than the CPDN classic!

Unless someone has a better reason?

Cheers
Dave

&gt; It would appear that, in some fashion, the BOINC CPDN client is more efficient
&gt; than the CPDN classic!
&gt;
&gt; Unless someone has a better reason?

Much of the speed improvement is down to the amount of disk access - the BOINC client writes to disk less frequently than the classic one.

<a href="http://climateapps2.oucs.ox.ac.uk/cpdnboinc/team_display.php?teamid=3"><img src="http://www.teampicard.net/templates/fisubice/images/phpbb2_logo.jpg"></a>

&gt; &gt; It would appear that, in some fashion, the BOINC CPDN client is more
&gt; efficient
&gt; &gt; than the CPDN classic!
&gt; &gt;
&gt; &gt; Unless someone has a better reason?
&gt;
&gt; Much of the speed improvement is down to the amount of disk access - the BOINC
&gt; client writes to disk less frequently than the classic one.
&gt;
Also the BOINC version uses CPU time and the classic uses wall time. So the improvement seems greater than it is.
<br>John Keck
BOINCing since 2002/12/08

&gt; I did some HT testing during the alpha trial. Running the BOINC client didn't
&gt; seem to affect the performance of my legacy CPDN.

Running BOINC affects legacy CPDN here (just as a recap, the machine is dual Xeon 2.66GHz, Hyperthreading on, Windows XP Professional SP1). The following discussion is inevitably Windows-centric.

You can take a look at my legacy CPDN trickles at http://cpdn.comlab.ox.ac.uk/user/trklm.php?mid=12393


In the first period where the seconds per trickle jumps, I was running as many as 3 BOINC CPDNs. I dropped back to 1 BOINC CPDN subsequently and the time period per trickle dropped, but there's still around a 40% speed penalty over running without BOINC.

I'm somewhat unclear why this is the case, what I can do about it, and what I should do about it.


The machine does have some real work to be getting on with as well as crunching climateprediction.net models. Being the biggest machine by far on my network, it is my backup server, it runs all my network and UPS monitoring and it's my mail server. It doesn't run a server version of Windows because I don't really need a Windows server on my relatively small network so didn't want to pay for the server OS, also Dell doesn't support server versions of Windows on this hardware - not that I think I would have a problem with drivers if I did install Windows Server 2003 - it's a relatively standard Intel E7505 chipset motherboard, and both the motherboard SCSI and Gigabit Ethernet chips (components not necessarily on an E7505 motherboard) have Windows Server 2003 drivers, as does my graphics card (nVidia Quadro FX500).

Clearly, then, my processor resources aren't idle all the time that the machine isn't in foreground use - there's probably more like 1.5 or 1.6 processors spare (plus Hyperthreading on those). With about 57 hours of uptime since reboot, the various other tasks the machine has run have taken around 300 minutes of CPU time - whatever that is in real execution time bearing in mind Hyperthreading is on.


I do wonder whether Hyperthreading is part of the reason for this result. Two processes executing on one processor using Hyperthreading will not complete in the same time as one, barring the very occasional case. The gain from Hyperthreading can, if my understanding is correct, often be no better than a few percent. That said, my understanding is that the task scheduler in Windows XP (and Windows Server 2003 - but not Windows 2000) is intelligent enough to use spare physical processors before logical processors.


I have to say that this result is in line with what I predicted - though apparently Thyme Lawn's results are different (maybe his machines are genuinely idle when they're not in use, with no background tasks).


Whatever other people's results, it looks like I'm not going to be able to have the best of both worlds - if I run any BOINC, then my throughput on classic CPDN will drop unless a version of classic CPDN is made available that has "Below Normal" priority as an option instead of "Low".

I guess, too, at some point if I want maximum BOINC throughput, I'll have to cut away from legacy CPDN to all BOINC - at which point I may as well run four BOINC instances to use every last shred of spare processor resources in the machine (my disk resources are fast - the main disks in the machine are a pair of Seagate Cheetah 15K.3 SCSI drives, one of the fastest HDs you can get).


At the moment, I think I have the worst of both worlds - degraded legacy CPDN performance and sub-optimal BOINC throughput!


I do wonder whether support for explicit processor affinity in BOINC (and, for that matter, in legacy CPDN) would help - if it's possible to add such support to a program. If you don't set processor affinity, then, at least in Windows XP, my impression is that the process can 'spin' between processors. On a single processor Hyperthreading machine, there's no performance hit spinning between the two virtual processors; they share one cache. On a machine with more than one physical processor, there is a performance hit, as each processor can only access its own cache.

There's people here who are far more experienced with BOINC and probably more experienced with IA-32 architecture than I am - I raise this simply for what it's worth based on my less than complete understanding of both subjects.


Meanwhile, I'd appreciate advice on what I should do - I suppose the optimal way ahead is to mark my legacy CPDN account to finish at the end of this run, ratchet up to 3 BOINC CPDNs, accept that my place in the legacy CPDN machine league table will fall consistently from this point (sob! - I've already dropped one place), but that my place in the BOINC machine league table will be much better.

Then, when my legacy CPDN run finishes, I should switch to 4 BOINC CPDNs.


I don't suppose there's any plan to link the legacy CPDN stats with the BOINC ones, is there?


Any thoughts are welcome,




David

Clarifying what I wrote:

&gt; If you don't set processor affinity, then, at least in Windows
&gt; XP, my impression is that the process can 'spin' between processors.

My impression is that processes not only can 'spin' between processors, but that they usually *do* spin.



David

&gt; Running BOINC affects legacy CPDN here (just as a recap, the machine is dual
&gt; Xeon 2.66GHz, Hyperthreading on, Windows XP Professional SP1).
&gt;
&gt; I have to say that this result is in line with what I predicted - though
&gt; apparently Thyme Lawn's results are different (maybe his machines are
&gt; genuinely idle when they're not in use, with no background tasks).

David,

I've got 2 single processor P4s (2.6GHz and 3.0GHz). There are no background tasks on them and when they get used for real work both of the virtual processors are 100% utilised. Cache and memory are large enough to avoid any swapping.

In the alpha trial I found running 2 models with HT enabled gave 13% improved performance over a single model with HT disabled.

I presume your other tasks are running at a higher priority than the CPDN ones, so every time they require &gt;12.5% of the available processing power it's possible that a CPDN job could be bumped to another processor (this assumes the worst case scenario of the other tasks all being assigned to the same processor). If you're lucky the CPDN job will be bumped to the other virtual processor on the same physical one. Sods law dictates this won't be the case :(

One thing that will *really* slow things down is running one or more visualisation. That can eat up lots of processing power.

&gt; I do wonder whether support for explicit processor affinity in BOINC (and, for
&gt; that matter, in legacy CPDN) would help - if it's possible to add such support
&gt; to a program. If you don't set processor affinity, then, at least in Windows
&gt; XP, my impression is that the process can 'spin' between processors. On a
&gt; single processor Hyperthreading machine, there's no performance hit spinning
&gt; between the two virtual processors; they share one cache. On a machine with
&gt; more than one physical processor, there is a performance hit, as each
&gt; processor can only access its own cache.

Being able to set processor affinity would definitely help, but you'd have to live with at least one of the CPDN jobs having reduced performance because of your background tasks. The SETI@Home command line client has an option to run on a fixed CPU, so it's certainly possible.

&gt; Meanwhile, I'd appreciate advice on what I should do - I suppose the optimal
&gt; way ahead is to mark my legacy CPDN account to finish at the end of this run,
&gt; ratchet up to 3 BOINC CPDNs, accept that my place in the legacy CPDN machine
&gt; league table will fall consistently from this point (sob! - I've already
&gt; dropped one place), but that my place in the BOINC machine league table will
&gt; be much better.

My position in CPDN classic started plummeting when I started alpha testing. I was in the top 150 but last time I checked I'd dropped about 100 places :(

But I guess that means there's lots of people active in the project ;-)

&gt; Then, when my legacy CPDN run finishes, I should switch to 4 BOINC CPDNs.

I'd one box finish a THC experiment the day after the first batch of BOINC invites went out. I'd no hesitation in switching it over so I could run 2 faster experiments.

&gt; I don't suppose there's any plan to link the legacy CPDN stats with the BOINC
&gt; ones, is there?

No, but the legacy experiments will continue to run for a long time yet (the planned OU course is built around that rather then BOINC).

<a href="http://climateapps2.oucs.ox.ac.uk/cpdnboinc/team_display.php?teamid=3"><img src="http://www.teampicard.net/templates/fisubice/images/phpbb2_logo.jpg"></a>

&gt; In the alpha trial I found running 2 models with HT enabled gave 13% improved
&gt; performance over a single model with HT disabled.

That sort of gain would be in line with my expectations for HT.


&gt; I presume your other tasks are running at a higher priority than the CPDN
&gt; ones, so every time they require &gt;12.5% of the available processing power
&gt; it's possible that a CPDN job could be bumped to another processor (this
&gt; assumes the worst case scenario of the other tasks all being assigned to the
&gt; same processor). If you're lucky the CPDN job will be bumped to the other
&gt; virtual processor on the same physical one. Sods law dictates this won't be
&gt; the case :(

I'm curious as to the derivation of the 12.5% figure - is this based on the architecture of the Windows task scheduler?


My experience is that on a machine that's really doing nothing more than idling (though there are various background tasks idle that will wake up periodically) that Windows apparently rolls climateprediction.net (I'm talking about the legacy client here, not BOINC) between the processors on my system. This happens even if you turn Hyperthreading off, so you just have two physical processors.


If I'm correct about this behaviour it is, of course, nonsensical, as it wipes out all the caching gains (my Xeons have no L3 cache, but they do have a 512K L2 cache). I don't see the point of putting any L3 cache processors in this box; they're extremely expensive even now. 3.06GHz 533MHz FSB 1MB L3 cache Xeons are currently around 480 pounds each! The same processor without the L3 cache is about 200 pounds - it's not worth spending 400 pounds for a 15% processor performance gain. I will keep an eye on prices - if I find a couple of faster processors being remaindered when the 533MHz stuff is becoming obsolete, it may be worth spending a bit of money then - though I suspect my workstation will be very long in the tooth at that point.

Rather than buying new processors for this box, it would arguably better to buy a new E7525 based Dell Precision 670 with 800MHz FSB Xeons in a largely minimal configuration (apart from, of course, memory - and a graphics card as the E7525 board is PCI Express, not AGP), and move my SCSI hard disks and other high-end peripherals to the new box.


In fact, I'm not going to do any hardware changes. I'm quite happy with the performance of this E7505 based Dell Precision 650 workstation, and will be sitting still for some time. The computers I've got are likely to serve my needs for several years to come, and by the time I'm in the market for a new computer, I expect various significant changes will have happened (64-bit may well be mainstream, for example).


&gt; One thing that will *really* slow things down is running one or more
&gt; visualisation. That can eat up lots of processing power.

I'm not using any visualisations - I'm running BOINC as a service (that way, it doesn't matter whether a user is logged on or not). BOINC climateprediction.net as a Windows service doesn't currently offer visualisations.


&gt; &gt; I do wonder whether support for explicit processor affinity in BOINC
&gt; (and, for
&gt; &gt; that matter, in legacy CPDN) would help - if it's possible to add such
&gt; support
&gt; &gt; to a program. If you don't set processor affinity, then, at least in
&gt; Windows
&gt; &gt; XP, my impression is that the process can 'spin' between processors. On
&gt; a
&gt; &gt; single processor Hyperthreading machine, there's no performance hit
&gt; spinning
&gt; &gt; between the two virtual processors; they share one cache. On a machine
&gt; with
&gt; &gt; more than one physical processor, there is a performance hit, as each
&gt; &gt; processor can only access its own cache.
&gt;
&gt; Being able to set processor affinity would definitely help, but you'd have to
&gt; live with at least one of the CPDN jobs having reduced performance because of
&gt; your background tasks. The SETI@Home command line client has an option to run
&gt; on a fixed CPU, so it's certainly possible.

I wonder if the BOINC folk could investigate this - I believe, but am not 100% sure (I don't have the time to check MSDN) that Windows processes can set their own processor affinity on startup.

Of course, running BOINC as a service, I could sort this myself manually - if I run the BOINC service as something I can log in as, then when I reboot I can log in and set the processor affinity of my BOINC climateprediction.net processes. I may experiment with this when I've finished my legacy CPDN run and am running four BOINC models.


&gt; &gt; Meanwhile, I'd appreciate advice on what I should do - I suppose the
&gt; optimal
&gt; &gt; way ahead is to mark my legacy CPDN account to finish at the end of this
&gt; run,
&gt; &gt; ratchet up to 3 BOINC CPDNs, accept that my place in the legacy CPDN
&gt; machine
&gt; &gt; league table will fall consistently from this point (sob! - I've already
&gt; &gt; dropped one place), but that my place in the BOINC machine league table
&gt; will
&gt; &gt; be much better.
&gt;
&gt; My position in CPDN classic started plummeting when I started alpha testing.
&gt; I was in the top 150 but last time I checked I'd dropped about 100 places :(
&gt;
&gt; But I guess that means there's lots of people active in the project ;-)

There's no way I can run BOINC and keep my machine leader board place - I think the decision is made to run my legacy CPDN run to the end, and stop there, then switch to four BOINC runs. It may not get me that spectacular a leader board place on either system, but it's about the science.


&gt; &gt; Then, when my legacy CPDN run finishes, I should switch to 4 BOINC
&gt; CPDNs.
&gt;
&gt; I'd one box finish a THC experiment the day after the first batch of BOINC
&gt; invites went out. I'd no hesitation in switching it over so I could run 2
&gt; faster experiments.

If I want to finish legacy CPDN completely, presumably I remove the tick next to "account active" in my account page - that is the right way to stop my machine downloading a new run when it finishes this one, isn't it?


&gt; &gt; I don't suppose there's any plan to link the legacy CPDN stats with the
&gt; BOINC
&gt; &gt; ones, is there?
&gt;
&gt; No, but the legacy experiments will continue to run for a long time yet (the
&gt; planned OU course is built around that rather then BOINC).

I understand - basically the two will continue in parallel, though BOINC will be the way ahead, in terms of new experiments and optimised use of current and future hardware.



David

&gt; &gt; Being able to set processor affinity would definitely help, but you'd
&gt; have to
&gt; &gt; live with at least one of the CPDN jobs having reduced performance
&gt; because of
&gt; &gt; your background tasks. The SETI@Home command line client has an option
&gt; to run
&gt; &gt; on a fixed CPU, so it's certainly possible.
&gt;
&gt; I wonder if the BOINC folk could investigate this - I believe, but am not 100%
&gt; sure (I don't have the time to check MSDN) that Windows processes can set
&gt; their own processor affinity on startup.
&gt;
The BOINC development team has had a discussion about this. It seems that processor affinity will help dual CPU systems, and hurt HT systems. I wonder if it would be possible to get the number of real CPUs and set the processor affinity to real CPUs rather than virtual CPUs?

jm7

&gt; I'm curious as to the derivation of the 12.5% figure - is this based on the
&gt; architecture of the Windows task scheduler?

Each of your virtual processors has 25% of the available processing power. When all your background tasks are idling the CPDN jobs can effectively have a dedicated virtual processor each. When your background tasks wake up what happens is going to depend on how much processing power they need. If they need &gt;12.5% on a single VP it's possible that Windows might decide to swap things around because the CPDN job on that VP could get more processing power by sharing another CPDN job's VP.

&gt; My experience is that on a machine that's really doing nothing more than
&gt; idling (though there are various background tasks idle that will wake up
&gt; periodically) that Windows apparently rolls climateprediction.net (I'm talking
&gt; about the legacy client here, not BOINC) between the processors on my system.
&gt; This happens even if you turn Hyperthreading off, so you just have two
&gt; physical processors.

If the CPDN job is on CPU0 and a background task wakes up and runs on CPU0 your CPDN job will probably switch to CPU1. And it might switch back when the background task goes back to sleep. That's Windows for you :(

&gt; I wonder if the BOINC folk could investigate this - I believe, but am not 100%
&gt; sure (I don't have the time to check MSDN) that Windows processes can set
&gt; their own processor affinity on startup.

It must be possible because Fire Daemon has the ability to bind a service to a particular CPU.

&gt; There's no way I can run BOINC and keep my machine leader board place - I
&gt; think the decision is made to run my legacy CPDN run to the end, and stop
&gt; there, then switch to four BOINC runs. It may not get me that spectacular a
&gt; leader board place on either system, but it's about the science.

Indeed it is

&gt; If I want to finish legacy CPDN completely, presumably I remove the tick next
&gt; to "account active" in my account page - that is the right way to stop my
&gt; machine downloading a new run when it finishes this one, isn't it?

Yes, that's what I did.



<a href="http://climateapps2.oucs.ox.ac.uk/cpdnboinc/team_display.php?teamid=3"><img src="http://www.teampicard.net/templates/fisubice/images/phpbb2_logo.jpg"></a>