Quick question, out of interest. How many cores on your machines to you make available to CPDN/BOINC?

I ask because we considering what to create for OpenIFS; thinking 2 & 4 core versions but maybe people on this thread have some thoughts?

Thanks.

I'm probably the wrong person to answer this since I use Windows, but all of them. I think most people are the same, although some make half available if they're using the machine for other purposes. I have three 4-core machines, one 6-core machine, and three 24-core machines. One of the 4-cores only has 2 CPU cores for Boinc as it runs 5 GPUs. The 6-core is 4 cores for Boinc for the same reason. One of the 24 cores is 18 cores for Boinc, as it's my main machine for gaming, watching TV, using the internet, and monitoring security cameras. The rest have the entire CPU for Boinc.

I don't know how flexible your programming is, but LHC@Home for example releases ATLAS tasks up to 8 cores (I can't remember their reasoning for the limit), but if someone only has 4 available, you get a 4 core version. Their program works by starting a number of single-core workers, so might be different to what you're going to do. Milkyway@Home does a similar thing with their n-body tasks. Might be worth asking on their forums to see if you can chat with their programmers.

Thanks, useful. The scaling for OpenIFS drops off beyond 8 threads so we are unlikely to go any further.

I have several Intel machines but since the physical cores support 2 hyper-threads, I tend to limit the BOINC cpus to just the number of physical cores.

Thanks, useful. The scaling for OpenIFS drops off beyond 8 threads so we are unlikely to go any further.

I have several Intel machines but since the physical cores support 2 hyper-threads, I tend to limit the BOINC CPUs to just the number of physical cores.

This seems to depend on the project. Some programs do better using the HT threads aswell, some don't. I think the major factor is the amount of cache on the CPU. I run several projects and overall I find it best to use the HT threads.

Over at Primegrid you can set how ever many cores you want to use to multithread, from none to as many as you think you need or can spare. A Seventeen or Bust (SoB) work unit takes 400 to whatever hours on a single core but this drops to less than 100 hours using 6 cores.

I have a ryzen 3900X with 12 cores and 12 threads but limit Primegrid to just 6 cores for multithreading due to the large number of other projects that I run at the same time.

The n-body work units from Milkyway use 16 cores and I can't seem to change that but as they only run for a few minutes to an hour or so it is not a problem, still have 8 other cores.

At Yafu@home you can set from 1 to 32 cores due to their work unit types.

Conan

---

Quick question, out of interest. How many cores on your machines to you make available to CPDN/BOINC?

I ask because we considering what to create for OpenIFS; thinking 2 & 4 core versions but maybe people on this thread have some thoughts?

My main machine claims to have 16 cores, but half of them are hyperthreaded, so I set Boinc up to use only 8 cores. I have 65 GBytes of RAM.

When it runs Milkyway@home, it sometimes runs four cores in parallel on a single work unit. NBody.

It is like this

Computer 1511241
CPU type 	GenuineIntel
Intel(R) Xeon(R) W-2245 CPU @ 3.90GHz [Family 6 Model 85 Stepping 7]
Number of processors 	16
Operating System 	Linux Red Hat Enterprise Linux
Red Hat Enterprise Linux 8.6 (Ootpa) [4.18.0-372.19.1.el8_6.x86_64|libc 2.28 (GNU libc)]
BOINC version 	7.16.11
Memory 	62.28 GB <---<<<
Cache 	16896 KB <---<<<
Swap space 	15.62 GB
Total disk space 	488.04 GB
Free Disk Space 	482.76 GB
Measured floating point speed 	6.58 billion ops/sec
Measured integer speed 	30.58 billion ops/sec
Average upload rate 	409.01 KB/sec
Average download rate 	84563.86 KB/sec
Average turnaround time 	2.47 days

---

I have an AMD 12C/24T w/ 64GB of RAM and an Intel 4C/8T w/ 16 GB of RAM running BOINC 24/7 using all cores/threads (also doing GPUs work). Making the app available in multiple versions of up to 8 threads sounds like a good idea.

Conan,
That's strange that you're having difficulty changing MilkyWay N-Body core usage. The only way to do it is via app_config.xml file. An example one to use 4 cores is here: https://milkyway.cs.rpi.edu/milkyway/forum_thread.php?id=4891&postid=73503
Create it using a basic text editor and save it in the MilkyWay project directory and run "Read config files" option via GUI or command line or shut down and restart BOINC for it to take effect.

Quick question, out of interest. How many cores on your machines to you make available to CPDN/BOINC?

Currently, if running the higher resolution N216 tasks, performance starts dropping after using 5 out of 8 real cores because they hammer the chip's cache so much. If running N144 tasks I go up to 8 real cores. Assuming, I double my RAM to 64GB, I will probably go to 8 real cores with the OpenIFS tasks. Of course, I will hopefully have the luxury of being able to see what works best in testing. How much the memory (RAM not cache) gets hit will depend a lot on whether all 8 processes peak at the same time or whether they end up all at different times. If the latter,I could probably get away with just 32GB of RAM rather than upping to 64. With 32, I will probably start by seeing what happens with 4 cores.

@ AndreyOR,

The app_config.xml file in that example is insufficient. It should read:

<app_config>
   <app_version>
      <app_name>milkyway_nbody</app_name>
      <plan_class>mt</plan_class>
      <avg_ncpus>4</avg_ncpus>
      <cmdline>--nthreads 4</cmdline>
   </app_version>
</app_config>

For efficient use of the CPU, you need both commands.

<avg_ncpus> controls BOINC, and tells it how to schedule other work around the multi-threaded task.

--nthreads on the command line is passed to the multi-threaded app itself, and is intended to limit its own core usage.

@Glenn Carver, @Jean-David Beyer,

If a multi-threaded application is deployed via a BOINC server, the automatic (default) behaviour is for the server to configure each task to use every one of the cores reported by the requesting client - 16, in Jean-David's example. The assumption is that the application will understand the --nthreads directive, and configure itself accordingly. If the proposed IFS application uses a different MT calling convention, it will require a bespoke modification of the CPDN server code.

If a multi-threaded application is deployed via a BOINC server, the automatic (default) behaviour is for the server to configure each task to use every one of the cores reported by the requesting client - 16, in Jean-David's example.

I am not sure if I understand you.

My machine has 16 cores (8 real, 8 hyperthreaded. Boinc client is set up to use 8 of these. As far as MilkWay is concerned,

[/var/lib/boinc/projects/milkyway.cs.rpi.edu_milkyway]$ cat app_config.xml 
<app_config>
    <project_max_concurrent>4</project_max_concurrent>
<app_version>
      <app_name>milkyway_nbody</app_name>
      <plan_class>mt</plan_class>
      <avg_ncpus>4</avg_ncpus>
   </app_version>
</app_config>

So I run a maximum of four Milky Way tasks at a time because of max_concurrent.
And those multi-processor tasks run 4 cores each because of
<app_name>milkyway_nbody</app_name>
<plan_class>mt</plan_class>
<avg_ncpus>4</avg_ncpus>
Now sometimes the Boinc Clent schedules two of those multi-core processes so they consume all 8 of the processors. In that case the client turns off any other boinc processes until at least one of those multi-process tasks completes.

---

The acid test is to look at the stderr output from one of your tasks. For example, task 421516802is from your 16-core computer:

<stderr_txt>
<search_application> milkyway_nbody 1.76 Linux x86_64 double  OpenMP, Crlibm </search_application>
Using OpenMP 4 max threads on a system with 16 processors

'OpenMP' is the tool BOINC is designed around and - as you say - in your case the tlread count is limited to 4. My understanding is that under OpenMP, that will have been set by an '--nthreads=4' directive on the command line. If you don't have it in your app_config.xml, it's possible that they have - at long last - configured their server in the way I suggested CPDN may have to do.

I participated in some of the very early testing of nbody at Milkyway, and I found it very frustrating dealing with the management team of the day. I gather things have improved, but I'm not currently active there. For some background, read my message 58339 of 19 May 2013. I don't run many high core-count computers myself, and I mainly run Windows computers. If I could, I'd run something like the Windows utility 'Process Explorer' to see exactly how many threads the calling application spawned, under various conditions. Something like that should be possible under Linux too.

Quick question, out of interest. How many cores on your machines to you make available to CPDN/BOINC?

I have two Ryzen 3600's (12 virtual cores each) and two Ryzen 3900X (24 cores each), each with 64 GB of memory, three on Ubuntu 20.04.4 and one on Win10 (but easily switched).
I have even more Ryzen 5000 series, but usually reserve them for Folding. But if you need them to warn of hurricanes, I can do them too.

For Milkyway I don't always set the numbers the same in app_config. Since the program uses only 1 core for a bit to set things up, then most of the cores allocated for the rest of it, you can tell the scheduler it uses an average of 6 cores, but tell the program to use up to 8. That way a 24 core CPU is maxed out all the time running four programs at up to 8 cores each.

Richard,

It seems like the --nthreads is redundant, at least for N-Body, as avg_ncpus seems to work without problems as I've never used --nthreads and have run N-Body with a variety of CPU count configurations on both Linux and Windows. I did see that --nthreads is a valid parameter for N-Body though. Could it be, like you hypothesized in another post, that MilkyWay may have made server side changes to make --nthreads unnecessary?

In another project, LHC ATLAS native, I found that --nthreads is necessary, at least under WSL2 Ubuntu (it's not quite the same as regular Ubuntu).

For CPDN, from mass user perspective who just want plug-and-play, it'd be helpful if OpenIFS be set up so that all such configurations be made on the website preferences page. So, an option of how many CPUs to use per task and an option of maximum number of tasks to have in progress.

Nthreads is a completely different thing. It tells the app how many threads to use as a maximum. Avg_cpus tells the scheduler how many it uses on average.

Web-side settings are pointless, most of us have more than one machine and fine tune each accordingly.

Web preferences would help the casual user to get going quicker and without problems and thus make them more likely to stay, in this regard I think they'd be very useful. Dedicated users can still use the various local configurations, such as app_config, app_info, and other options to override the web preferences or do other customized set ups.

I don't believe nthreads and avg_ncpus are as you describe them. nthreads tells the app exactly how many cores to use, no more no less, it is not a BOINC configuration parameter but a configuration parameter of the app itself. avg_ncpus is a BOINC configuration parameter and is used by BOINC to schedule tasks and it seems in some apps can also control how many cpus are used for a multicore apps without the need for nthreads.

I don't believe nthreads and avg_ncpus are as you describe them. nthreads tells the app exactly how many cores to use, no more no less, it is not a BOINC configuration parameter but a configuration parameter of the app itself. avg_ncpus is a BOINC configuration parameter and is used by BOINC to schedule tasks and it seems in some apps can also control how many cpus are used for a multicore apps without the need for nthreads.

You've stated almost exactly what I said about them. Nthreads tells the app how many threads to use. It will use up to that amount, a lot of multi threads apps need some single thread time to start up. Avg_ncpus is just number for the scheduler, so it knows how many to start.

Edit: I'll double check this as some claim it's incorrect. Windows version might have some differences, I ran tests on Linux.

Richard, Jean-David,

I did some N-Body testing on Ubuntu 20.04 and this is what I found:

* --nthreads x is an incorrect syntax and is ignored, it's --nthreads=x
* --nthreads is unnecessary as avg_ncpus can control the number of cores the app uses and it's reflected in the stderr output "Using OpenMP ..."
* --nthreads always overrides avg_ncpus as to how many cores the app will use and it's reflected in the stderr output "Using OpenMP ..."
* If --nthreads is higher than avg_ncpus, you'll be limited as to how many tasks are ran at the same time as avg_ncpus controls that
* If avg_ncpus is higher than --nthreads then you'll be forcing more tasks per core at the same time, like making a GPU run more than 1 task at a time with the gpu_usage flag

On that last points, just like with the GPU tasks, one needs to test to see if it's worth doing, i.e.is the throughput higher, does error rate go up, etc.

So very wrong.

I use "--nthreads 12" and it makes the app use up to 12 threads.

For the last time, avg_ncpus is a command to the boinc scheduler, not the project app. For example you can tell the scheduler to run 4 of 6 thread nbody on your 24 thread CPU. But you can tell the app using nthreads to use up to 10 threads each, so they can use each other's threads when one isn't very busy.

Your last two points are inside out. Please proof read before making a fool of yourself.

I checked things again and here are some corrections:

* both --nthreads x and --nthreads=x work. Not sure where I went wrong. Looking up usage help for the app though, it says to use --nthreads=INT
* the last 2 points are somewhat inaccurate. Probably a more general way to say it is that depending on the values of --nthreads and avg_ncpus you can have situations where in some cases not all cores are used and in other cases where multiple tasks per core are forced, like making a GPU run more than 1 task at a time with the gpu_usage flag. The latter cases should be tested to see if it's worthwhile, i.e.is the throughput higher, does error rate go up, etc. I'd assume not though unless testing shows otherwise.