climateprediction.net (CPDN) home page
Thread 'New work Discussion'

Thread 'New work Discussion'

Message boards : Number crunching : New work Discussion
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 79 · 80 · 81 · 82 · 83 · 84 · 85 . . . 91 · Next

AuthorMessage
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65537 - Posted: 10 Jun 2022, 5:13:26 UTC

Previous practice was to upload the zips to a server located in the region under investigation - NZ, in this case. The kick might be better directed there...
Thanks for the hint Andy. zips now uploading.
ID: 65537 · Report as offensive
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 65538 - Posted: 10 Jun 2022, 11:49:12 UTC - in response to Message 65535.  

I now have three N216 work units running. IIRC, they trickle every 1/8 of the time through.


It appears I do not remember correctly. These work units give 4 or 5 trickles, each about 25% apart. So about one every two days on my machine.
ID: 65538 · Report as offensive
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 65539 - Posted: 10 Jun 2022, 13:30:03 UTC - in response to Message 65538.  

These work units give 4 or 5 trickles, each about 25% apart.

I am just at 25% and see only one trickle too.

Running four at a time, they are taking over 13 1/2 days on a Ryzen 3900X, but that is with all cores loaded.
When I ran them on a Ryzen 3600 with only six cores (50%) in use, they would take a little under 10 days. So I think the ratio is right for virtual cores.
ID: 65539 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65540 - Posted: 10 Jun 2022, 14:23:08 UTC - in response to Message 65539.  
Last modified: 10 Jun 2022, 14:23:49 UTC

These work units give 4 or 5 trickles, each about 25% apart.


The _5_ in the task name just before the batch number is the clue here. It means they are five month tasks which will send trickles at each 20% or shortly after except for 5.zip which for some reason is produced a bit before the task reaches 100%.
ID: 65540 · Report as offensive
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 65541 - Posted: 10 Jun 2022, 17:42:35 UTC - in response to Message 65539.  

Running four at a time, they are taking over 13 1/2 days on a Ryzen 3900X, but that is with all cores loaded.
When I ran them on a Ryzen 3600 with only six cores (50%) in use, they would take a little under 10 days. So I think the ratio is right for virtual cores.


On my machine, they take about 8 days. using 8 of the 16 cores of my machine. I limit my machine to a maximum of 4 cores for ClimatePrediction work units, and at the moment the other 4 cores are doing MilkyWay.

My machine:
Computer 1511241

CPU type 	GenuineIntel
Intel(R) Xeon(R) W-2245 CPU @ 3.90GHz [Family 6 Model 85 Stepping 7]
Number of processors 	16
Operating System 	Linux Red Hat Enterprise Linux
Red Hat Enterprise Linux 8.6 (Ootpa) [4.18.0-372.9.1.el8.x86_64|libc 2.28 (GNU libc)]
BOINC version 	7.16.11
Memory 	 62.4 GB
Cache 	16896 KB  <---<<<


Cache miss ratio -- not too bad, but not wonderful either
# perf stat -aB -e cache-references,cache-misses
 Performance counter stats for 'system wide':

    36,976,268,857      cache-references                                            
    20,197,925,561      cache-misses              #   54.624 % of all cache refs    

      60.143708333 seconds time elapsed

ID: 65541 · Report as offensive
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65542 - Posted: 10 Jun 2022, 18:17:22 UTC

If you're on Intel/Linux, https://github.com/opcm/pcm will get you much more detailed stats on cache behavior (per core), global memory bandwidth, instructions retired, etc.
ID: 65542 · Report as offensive
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 65543 - Posted: 10 Jun 2022, 20:18:57 UTC - in response to Message 65542.  

If you're on Intel/Linux, https://github.com/opcm/pcm will get you much more detailed stats on cache behavior (per core), global memory bandwidth, instructions retired, etc.


Those are for RHEL7 and I am running Red Hat Enterprise Linux release 8.6 (Ootpa)
ID: 65543 · Report as offensive
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 262
Credit: 34,915,412
RAC: 16,463
Message 65544 - Posted: 10 Jun 2022, 22:23:49 UTC

Oh, I just build it from source. As long as it's got MSR access (modprobe msr), it should work fine.
ID: 65544 · Report as offensive
bibi

Send message
Joined: 22 Dec 08
Posts: 7
Credit: 21,960,423
RAC: 24,787
Message 65546 - Posted: 13 Jun 2022, 9:36:23 UTC

Don't get tasks

this PC https://www.cpdn.org/show_host_detail.php?hostid=1521336 on VirtualBox got six tasks N216, but

this PC https://www.cpdn.org/show_host_detail.php?hostid=1532298 on WSL2 won't get tasks
sidock@home is running, so there is no basically problem.

Any idea?
ID: 65546 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,039,635
RAC: 18,944
Message 65547 - Posted: 13 Jun 2022, 11:33:42 UTC - in response to Message 65546.  

Don't get tasks
this PC https://www.cpdn.org/show_host_detail.php?hostid=1521336 on VirtualBox got six tasks N216, but

this PC https://www.cpdn.org/show_host_detail.php?hostid=1532298 on WSL2 won't get tasks
sidock@home is running, so there is no basically problem.
Any idea?

Nothing obvious looking at the link for the machine. One thing to remember is, not to keep clicking the update button because once an attempt has been made to get work, there is a back off time of one hour before another attempt can be made.

Could you post from the event log what shows up after you request work? That might give some clues.
ID: 65547 · Report as offensive
bibi

Send message
Joined: 22 Dec 08
Posts: 7
Credit: 21,960,423
RAC: 24,787
Message 65548 - Posted: 13 Jun 2022, 15:10:10 UTC - in response to Message 65547.  

Hi Dave,

I know of the back time. From the Event log:
13.06.2022 15:21:06 | climateprediction.net | Scheduler request completed: got 0 new tasks
13.06.2022 15:21:06 | climateprediction.net | No tasks sent
13.06.2022 15:21:06 | climateprediction.net | Project requested delay of 3636 seconds
13.06.2022 16:21:46 | climateprediction.net | Sending scheduler request: To fetch work.
13.06.2022 16:21:46 | climateprediction.net | Requesting new tasks for CPU
13.06.2022 16:21:51 | climateprediction.net | Scheduler request completed: got 0 new tasks
13.06.2022 16:21:51 | climateprediction.net | No tasks sent
13.06.2022 16:21:51 | climateprediction.net | Project requested delay of 3636 seconds
ID: 65548 · Report as offensive
AndreyOR

Send message
Joined: 12 Apr 21
Posts: 317
Credit: 14,899,790
RAC: 18,157
Message 65550 - Posted: 13 Jun 2022, 18:46:53 UTC - in response to Message 65548.  

Double check your basic "Computing preferences" settings, like work cache, CPU usage, and others. Try turning on sched_op_debug and work_fetch_debug in "Event Log options" before requesting work again. Check the Event Log afterwards to see if any new clues show up. I use WSL2 also, although 20.04, and have no problems getting work.
ID: 65550 · Report as offensive
ProfileAlan K

Send message
Joined: 22 Feb 06
Posts: 491
Credit: 31,051,362
RAC: 14,678
Message 65553 - Posted: 13 Jun 2022, 22:31:24 UTC - in response to Message 65546.  

Don't get tasks

this PC https://www.cpdn.org/show_host_detail.php?hostid=1521336 on VirtualBox got six tasks N216, but

this PC https://www.cpdn.org/show_host_detail.php?hostid=1532298 on WSL2 won't get tasks
sidock@home is running, so there is no basically problem.

Any idea?


Try suspending any other running projects and increase number of days of work to 10.
ID: 65553 · Report as offensive
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 65554 - Posted: 13 Jun 2022, 23:52:44 UTC - in response to Message 65553.  

Try suspending any other running projects and increase number of days of work to 10.


I have my days work set to 0,25 minimum and 2 days additional. I have no difficulty getting CPDN work units, when they are available for my machine and OS. For CPDN, I do not turn off other BOINC processes just to get work units.

ID: 1511241
ID: 65554 · Report as offensive
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 65555 - Posted: 14 Jun 2022, 2:27:38 UTC - in response to Message 65548.  

bibi

What we need to see is from near the start of the Event log.
Below is from mine, but I was fiddling a bit to get it set up, so it's an approximation of what yours will have:


Tue 07 Jun 2022 21:40:29 AEST | | Processor: 8 GenuineIntel Intel(R) Core(TM) i7-3770K CPU @ 3.50GHz [Family 6 Model 58 Stepping 9]
Tue 07 Jun 2022 21:40:29 AEST | | Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb pti tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts
Tue 07 Jun 2022 21:40:29 AEST | | OS: Linux: 4.15.0-142-generic
Tue 07 Jun 2022 21:40:29 AEST | | Memory: 31.30 GB physical, 15.89 GB virtual
Tue 07 Jun 2022 21:40:29 AEST | | Disk: 900.64 GB total, 741.77 GB free
Tue 07 Jun 2022 21:40:29 AEST | | Local time is UTC +10 hours
Tue 07 Jun 2022 21:40:29 AEST | | Config: GUI RPCs allowed from:
Tue 07 Jun 2022 21:40:29 AEST | climateprediction.net | URL https://climateprediction.net/; Computer ID 1485228; resource share 100
Tue 07 Jun 2022 21:40:29 AEST | cpdnboinc_dev | URL https://dev.cpdn.org/; Computer ID 30; resource share 100
Tue 07 Jun 2022 21:40:30 AEST | cpdnboinc_dev | General prefs: from cpdnboinc_dev (last modified 01-Feb-2021 22:04:07)
Tue 07 Jun 2022 21:40:30 AEST | cpdnboinc_dev | Computer location: home
Tue 07 Jun 2022 21:40:30 AEST | | General prefs: using separate prefs for home
Tue 07 Jun 2022 21:40:30 AEST | | Reading preferences override file
Tue 07 Jun 2022 21:40:30 AEST | | Preferences:
Tue 07 Jun 2022 21:40:30 AEST | | max memory usage when active: 32055.89MB
Tue 07 Jun 2022 21:40:30 AEST | | max memory usage when idle: 32055.89MB
Tue 07 Jun 2022 21:40:30 AEST | | max disk usage: 50.00GB
Tue 07 Jun 2022 21:40:30 AEST | | max CPUs used: 2
Tue 07 Jun 2022 21:40:30 AEST | | (to change preferences, visit a project web site or select Preferences in the Manager)
Tue 07 Jun 2022 21:40:31 AEST | | gui_rpc_auth.cfg is empty - no GUI RPC password protection
Tue 07 Jun 2022 21:40:35 AEST | | Running CPU benchmarks
Tue 07 Jun 2022 21:40:35 AEST | | Suspending computation - CPU benchmarks in progress
Tue 07 Jun 2022 21:40:35 AEST | | Suspending network activity - user request
Tue 07 Jun 2022 21:41:07 AEST | | Benchmark results:
Tue 07 Jun 2022 21:41:07 AEST | | Number of CPUs: 2
Tue 07 Jun 2022 21:41:07 AEST | | 4886 floating point MIPS (Whetstone) per CPU
Tue 07 Jun 2022 21:41:07 AEST | | 16298 integer MIPS (Dhrystone) per CPU
Tue 07 Jun 2022 21:41:17 AEST | | Resuming network activity
Tue 07 Jun 2022 21:41:17 AEST | climateprediction.net | Sending scheduler request: To fetch work.
Tue 07 Jun 2022 21:41:17 AEST | climateprediction.net | Requesting new tasks for CPU
Tue 07 Jun 2022 21:43:17 AEST | | Project communication failed: attempting access to reference site
Tue 07 Jun 2022 21:43:17 AEST | climateprediction.net | Scheduler request failed: Timeout was reached
Tue 07 Jun 2022 21:43:19 AEST | | Internet access OK - project servers may be temporarily down.
ID: 65555 · Report as offensive
bibi

Send message
Joined: 22 Dec 08
Posts: 7
Credit: 21,960,423
RAC: 24,787
Message 65558 - Posted: 14 Jun 2022, 13:49:54 UTC - in response to Message 65555.  

from log
start of VM:
14.06.2022 15:44:25 | | Starting BOINC client version 7.18.1 for x86_64-pc-linux-gnu
14.06.2022 15:44:25 | | This a development version of BOINC and may not function properly
14.06.2022 15:44:25 | | log flags: file_xfer, sched_ops, task, sched_op_debug, work_fetch_debug
14.06.2022 15:44:25 | | Libraries: libcurl/7.81.0 OpenSSL/3.0.2 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.11
14.06.2022 15:44:25 | | Data directory: /var/lib/boinc-client
14.06.2022 15:44:25 | | No usable GPUs found
14.06.2022 15:44:25 | | libc: version 2.35
14.06.2022 15:44:25 | | Host name: Chris-Pc
14.06.2022 15:44:25 | | Processor: 6 GenuineIntel Intel(R) Core(TM) i7-4770S CPU @ 3.10GHz [Family 6 Model 60 Stepping 3]
14.06.2022 15:44:25 | | Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single pti ssbd ibrs ibpb stibp fsgsbase bmi1 avx2 smep bmi2 erms invpcid xsaveopt flush_l1d arch_capabilities
14.06.2022 15:44:25 | | OS: Linux Ubuntu: Ubuntu 22.04 LTS [5.10.102.1-microsoft-standard-WSL2|libc 2.35]
14.06.2022 15:44:25 | | Memory: 15.63 GB physical, 16.00 GB virtual
14.06.2022 15:44:25 | | Disk: 250.98 GB total, 236.01 GB free
14.06.2022 15:44:25 | | Local time is UTC +2 hours
14.06.2022 15:44:25 | climateprediction.net | Found app_config.xml
14.06.2022 15:44:25 | SiDock@home | Found app_config.xml
14.06.2022 15:44:25 | climateprediction.net | Max 6 concurrent jobs
14.06.2022 15:44:25 | SiDock@home | Max 4 concurrent jobs
14.06.2022 15:44:25 | SiDock@home | General prefs: from SiDock@home (last modified 27-May-2022 21:33:43)
14.06.2022 15:44:25 | SiDock@home | Host location: none
14.06.2022 15:44:25 | SiDock@home | General prefs: using your defaults
14.06.2022 15:44:25 | | Reading preferences override file
14.06.2022 15:44:25 | | Preferences:
14.06.2022 15:44:25 | | max memory usage when active: 14401.33 MB
14.06.2022 15:44:25 | | max memory usage when idle: 14401.33 MB
14.06.2022 15:44:25 | | max disk usage: 30.00 GB
14.06.2022 15:44:25 | | suspend work if non-BOINC CPU load exceeds 50%
14.06.2022 15:44:25 | | (to change preferences, visit a project web site or select Preferences in the Manager)
14.06.2022 15:44:25 | | [work_fetch] Request work fetch: Prefs update
14.06.2022 15:44:25 | | [work_fetch] Request work fetch: Startup
14.06.2022 15:44:25 | | Setting up project and slot directories
14.06.2022 15:44:25 | | Checking active tasks
14.06.2022 15:44:25 | | Using account manager BOINCstatsBAM!
14.06.2022 15:44:25 | climateprediction.net | URL https://climateprediction.net/; Computer ID 1532298; resource share 200
14.06.2022 15:44:25 | SiDock@home | URL https://www.sidock.si/sidock/; Computer ID 37142; resource share 100


one fetch cycle:
14.06.2022 15:26:58 | | [work_fetch] Request work fetch: Backoff ended for climateprediction.net
14.06.2022 15:27:02 | | choose_project(): 1655213222.774189
14.06.2022 15:27:02 | | [work_fetch] ------- start work fetch state -------
14.06.2022 15:27:02 | | [work_fetch] target work buffer: 180.00 + 172800.00 sec
14.06.2022 15:27:02 | | [work_fetch] --- project states ---
14.06.2022 15:27:02 | climateprediction.net | [work_fetch] REC 0.000 prio -0.000 can request work
14.06.2022 15:27:02 | | [work_fetch] --- state for CPU ---
14.06.2022 15:27:02 | | [work_fetch] shortfall 1019559.53 nidle 5.00 saturated 0.00 busy 0.00
14.06.2022 15:27:02 | climateprediction.net | [work_fetch] share 1.000
14.06.2022 15:27:02 | | [work_fetch] ------- end work fetch state -------
14.06.2022 15:27:02 | climateprediction.net | choose_project: scanning
14.06.2022 15:27:02 | climateprediction.net | can fetch CPU
14.06.2022 15:27:02 | climateprediction.net | CPU needs work - buffer low
14.06.2022 15:27:02 | climateprediction.net | checking CPU
14.06.2022 15:27:02 | climateprediction.net | [work_fetch] using MC shortfall 1019559.532852 instead of shortfall 1019559.532852
14.06.2022 15:27:02 | climateprediction.net | [work_fetch] set_request() for CPU: ninst 6 nused_total 0.00 nidle_now 5.00 fetch share 1.00 req_inst 0.00 req_secs 1019559.53
14.06.2022 15:27:02 | climateprediction.net | CPU set_request: 1019559.532852
14.06.2022 15:27:02 | climateprediction.net | [sched_op] Starting scheduler request
14.06.2022 15:27:02 | climateprediction.net | [work_fetch] request: CPU (1019559.53 sec, 0.00 inst)
14.06.2022 15:27:02 | climateprediction.net | Sending scheduler request: To fetch work.
14.06.2022 15:27:02 | climateprediction.net | Requesting new tasks for CPU
14.06.2022 15:27:02 | climateprediction.net | [sched_op] CPU work request: 1019559.53 seconds; 0.00 devices
14.06.2022 15:27:04 | climateprediction.net | Scheduler request completed: got 0 new tasks
14.06.2022 15:27:04 | climateprediction.net | [sched_op] Server version 713
14.06.2022 15:27:04 | climateprediction.net | No tasks sent
14.06.2022 15:27:04 | climateprediction.net | Project requested delay of 3636 seconds
14.06.2022 15:27:04 | climateprediction.net | [work_fetch] backing off CPU 1397 sec
14.06.2022 15:27:04 | climateprediction.net | [sched_op] Deferring communication for 01:00:36
14.06.2022 15:27:04 | climateprediction.net | [sched_op] Reason: requested by project
14.06.2022 15:27:04 | | [work_fetch] Request work fetch: RPC complete

I can not see any problem, why this VM got no CPDN tasks. Now there are 737 N216 in the queue. Ok, sidock is running.
ID: 65558 · Report as offensive
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 1061
Credit: 36,722,381
RAC: 7,664
Message 65559 - Posted: 14 Jun 2022, 14:12:12 UTC - in response to Message 65558.  

14.06.2022 15:27:02 | climateprediction.net | Sending scheduler request: To fetch work.
14.06.2022 15:27:02 | climateprediction.net | Requesting new tasks for CPU
14.06.2022 15:27:02 | climateprediction.net | [sched_op] CPU work request: 1019559.53 seconds; 0.00 devices
14.06.2022 15:27:04 | climateprediction.net | Scheduler request completed: got 0 new tasks
14.06.2022 15:27:04 | climateprediction.net | [sched_op] Server version 713
14.06.2022 15:27:04 | climateprediction.net | No tasks sent
14.06.2022 15:27:04 | climateprediction.net | Project requested delay of 3636 seconds
Well, that's a work request, all right - looks normal.

Have you checked your account on this website, to ensure that your preferences include N216 for the venue that host is using?
ID: 65559 · Report as offensive
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 65560 - Posted: 14 Jun 2022, 15:23:17 UTC - in response to Message 65559.  

Have you checked your account on this website, to ensure that your preferences include N216 for the venue that host is using?


(I am not having any problems.)

But I do not think it is possible in preferences to select the tasks the machine uses anymore. It was possible years ago.
Where would I do this if I wanted to? (I see no need to do this at this time.)
ID: 65560 · Report as offensive
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 1061
Credit: 36,722,381
RAC: 7,664
Message 65561 - Posted: 14 Jun 2022, 15:47:02 UTC - in response to Message 65560.  

Sorry, I've been away from this project for a while, and hadn't kept up to date with recent changes. It would normally be on https://www.cpdn.org/prefs.php?subset=project, but I see it's been taken away.
ID: 65561 · Report as offensive
Bryn Mawr

Send message
Joined: 28 Jul 19
Posts: 150
Credit: 12,830,559
RAC: 228
Message 65562 - Posted: 14 Jun 2022, 18:24:47 UTC - in response to Message 65561.  

Sorry, I've been away from this project for a while, and hadn't kept up to date with recent changes. It would normally be on https://www.cpdn.org/prefs.php?subset=project, but I see it's been taken away.


The setting that caused me grief of a similar nature was no_alt_platform in the cc_config.xml file. With this set on the system worked fine with all other projects but would not download any CPDN WUs.
ID: 65562 · Report as offensive
Previous · 1 . . . 79 · 80 · 81 · 82 · 83 · 84 · 85 . . . 91 · Next

Message boards : Number crunching : New work Discussion

©2024 cpdn.org