climateprediction.net (CPDN) home page
Thread 'New work Discussion'

Thread 'New work Discussion'

Message boards : Number crunching : New work Discussion
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 26 · 27 · 28 · 29 · 30 · 31 · 32 . . . 91 · Next

AuthorMessage
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,005,674
RAC: 21,647
Message 59892 - Posted: 25 Mar 2019, 9:28:25 UTC

Edit:Six completed now, the new ones since yesterday are two xp and one win7 so still no 10s.


One out of ten completed is Win 10 but given the preponderance of Win10 computers compared to 7's the evidence is stacking up.
ID: 59892 · Report as offensive
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 59908 - Posted: 30 Mar 2019, 14:30:43 UTC - in response to Message 59822.  

I would certainly say that the minimum memory should be 2GB/core these days.



Well, I have four cores and 8 GBytes of RAM. Another 8 GBytes of RAM are on order and should arrive soon. Four 2GByte modules installed and four 2 GByte modules on order.


The new RAM arrived.
These numbers are not typical, because I just booted the machine.
$ free
total used free shared buffers cached
Mem: 16254616 6313052 9941564 35264 156712 3429440
-/+ buffers/cache: 2726900 13527716
Swap: 4095996 0 4095996


I have not noticed the need for it with CPDN, but some WCG tasks were using over 8% of the RAM (each) and there are other tasks in there too. It used to be my machine ran with about 10% of RAM used for processes, 5% for (output) buffers and 80% for (input) cache. The cache stores recently read stuff and often some disk read-ahead. It seems as though nothing has swapped yet. ;-)
ID: 59908 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,005,674
RAC: 21,647
Message 59909 - Posted: 30 Mar 2019, 14:46:12 UTC - in response to Message 59908.  

My figures with 2 tasks running on a dual core machine, only other things running are browser (just one tab open) and email. I have 4GB of ram on this machine but some is used by video. Uptime is a couple of weeks.

total used free shared buff/cache available
Mem: 4029908 3076436 129192 370284 824280 353820
Swap: 5859324 1277884 4581440
ID: 59909 · Report as offensive
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 59910 - Posted: 31 Mar 2019, 16:47:13 UTC - in response to Message 59909.  

My figures with 2 tasks running on a dual core machine, only other things running are browser (just one tab open) and email. I have 4GB of ram on this machine but some is used by video. Uptime is a couple of weeks.

total used free shared buff/cache available
Mem: 4029908 3076436 129192 370284 824280 353820
Swap: 5859324 1277884 4581440


After running for a bit more than a day, my memory usage is now:

$ free
             total       used       free     shared    buffers     cached
Mem:      16254616   10984588    5270028      53092     707676    6477604
-/+ buffers/cache:    3799308   12455308
Swap:      4095996       6152    4089844


It has used a little bit of the swap space.

Early this morning, about 15 GBytes of RAM were being used. This was almost certainly because a fairly complete backup runs every morning around 3AM when I am asleep. The space used as buffers must be written out before it can be re-used. The space used as (input) cache does not have to be written out ever, so can be re-used with no IO at all. And if input is requested from a disk drive, but is already in the cache, likewise, no IO is required, just resetting some memory-mapping registers.
ID: 59910 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,005,674
RAC: 21,647
Message 59911 - Posted: 2 Apr 2019, 6:54:35 UTC

My Win7 64-bit machine is still going strong on three 797's and two 798's after 4 to 9 days, with no failures on either. I don't think that is a coincidence.


#797 is now showing 43 tasks completed of these just 10 are with Windows10. I checked on a similar number of machines to get an estimate of how many are running Win10 and of those I looked at it was a little over half so Win10 having a problem with these tasks is looking significant statistically. It would be interesting to know what if anything is different about the win10 machines that are succeeding v those that are crashing these tasks. I don't know how many if any of these win10 machines are really Linux ones in disguise. My most advanced task doing this is just over 40% complete.
ID: 59911 · Report as offensive
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 59912 - Posted: 2 Apr 2019, 13:49:32 UTC - in response to Message 59911.  

I don't know how many if any of these win10 machines are really Linux ones in disguise. My most advanced task doing this is just over 40% complete.


One way to possibly determine Windows vs. Linux with Wine is the benchmark scores. The Floating Point scores in particular are way low in Wine compared to actual performance and capability of the CPU. So a Haswell 46xx might have a floating point score of 1100 where it should be in the 4000s (or something like that) in Windows. I'll modify the floating point score in client_state so that task estimated times are better, but I can't imagine there are many people doing that.
ID: 59912 · Report as offensive
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 59913 - Posted: 2 Apr 2019, 13:54:17 UTC - in response to Message 59911.  

#797 is now showing 43 tasks completed of these just 10 are with Windows10.

I just completed a 798 on Win7, and another one will complete in five hours (with no failures). I didn't get any on my Win10 machines, but they are probably the same as the 797s.
ID: 59913 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,005,674
RAC: 21,647
Message 59914 - Posted: 2 Apr 2019, 15:24:15 UTC - in response to Message 59912.  

So a Haswell 46xx might have a floating point score of 1100 where it should be in the 4000s (or something like that)


Thanks George,
I will have to compile a table based on Linux machines so I know the ball park figure to expect and if out by a factor of >3 for instance assume an imposter.

(Some of my recent tasks, the time is out by a factor of more than ten! I know I don't have the fastest machines but estimates of 186 days when worked out using percentage completed against time taken makes it under 15 days is pythonesque (and I am not referring to the programming language!))
ID: 59914 · Report as offensive
ProfileAlan K

Send message
Joined: 22 Feb 06
Posts: 491
Credit: 30,975,898
RAC: 14,500
Message 59916 - Posted: 2 Apr 2019, 21:49:36 UTC - in response to Message 59914.  

Would you be able to get a guess on the OS from the sec/ts data? One of the 797's that I got and failed was averaging 5.95sec/ts and got to the 2nd trickle on my faster i5 under Win10 before failing.
ID: 59916 · Report as offensive
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 59917 - Posted: 2 Apr 2019, 23:36:32 UTC - in response to Message 59916.  

Would you be able to get a guess on the OS from the sec/ts data? One of the 797's that I got and failed was averaging 5.95sec/ts and got to the 2nd trickle on my faster i5 under Win10 before failing.

There is negligible difference in speed between a PC running cpdn natively in Windows vs. the same PC running cpdn in Wine under Linux. At least that's what I've seen. It's just the benchmark that's screwed up for some reason.
ID: 59917 · Report as offensive
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 59918 - Posted: 3 Apr 2019, 1:53:57 UTC

Haswell
i7-4770K CPU @ 3.50GHz : 4373.42 million ops/sec
i7-4770K CPU @ 3.50GHz :  366.07 million ops/sec

Ivy Bridge
i7-3770K CPU @ 3.50GHz : 4046.54 million ops/sec
i7-3770K CPU @ 3.50GHz :  342.09 million ops/sec
ID: 59918 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,005,674
RAC: 21,647
Message 59968 - Posted: 12 Apr 2019, 6:42:27 UTC

Batch 807 7680 tasks WAH2 tasks for South America region 50Km squares. Looking at 2005 drought.
ID: 59968 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,005,674
RAC: 21,647
Message 59969 - Posted: 12 Apr 2019, 9:44:20 UTC - in response to Message 59968.  

#808 8580 Tasks same project.
ID: 59969 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,005,674
RAC: 21,647
Message 59970 - Posted: 12 Apr 2019, 10:56:28 UTC
Last modified: 12 Apr 2019, 10:57:05 UTC

and like buses, #809 brings us another 8190 tasks, again part of the same work.
ID: 59970 · Report as offensive
nairb

Send message
Joined: 3 Sep 04
Posts: 105
Credit: 5,646,090
RAC: 102,785
Message 59974 - Posted: 14 Apr 2019, 21:44:48 UTC

Boooo another "Signal 11 received: Segment violation"

on a "wah2_safr50_a0vl_201612_24_790_011752393_0"

I thought these were ok.
ID: 59974 · Report as offensive
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 59975 - Posted: 14 Apr 2019, 21:59:32 UTC

It's partly the model, and partly the computer.
ID: 59975 · Report as offensive
ProfileastroWX
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1496
Credit: 95,522,203
RAC: 0
Message 59976 - Posted: 15 Apr 2019, 4:11:43 UTC

Four of my boxes are upchucking on downloads of SAM50 tasks of different flavors.

One wonders when those 'scientists' will get their act together ...
"We have met the enemy and he is us." -- Pogo
Greetings from coastal Washington state, the scenic US Pacific Northwest.
ID: 59976 · Report as offensive
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,005,674
RAC: 21,647
Message 60008 - Posted: 25 Apr 2019, 7:03:10 UTC

Batches 810 and 811 bring over 11,000 new SAM50 tasks into the hoppers.
ID: 60008 · Report as offensive
nairb

Send message
Joined: 3 Sep 04
Posts: 105
Credit: 5,646,090
RAC: 102,785
Message 60010 - Posted: 25 Apr 2019, 12:00:33 UTC - in response to Message 59974.  

Boooo another "Signal 11 received: Segment violation"

on a "wah2_safr50_a0vl_201612_24_790_011752393_0"

I thought these were ok.


Another safra/790 with Segment violation. Pleased I have run out of these now..

Are the sam50's OK?
ID: 60010 · Report as offensive
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 60011 - Posted: 25 Apr 2019, 13:55:13 UTC - in response to Message 60010.  

Are the sam50's OK?

I have 4 sam50 errors and 19 completed. That is better than the safr50's, though I have not counted them all.
ID: 60011 · Report as offensive
Previous · 1 . . . 26 · 27 · 28 · 29 · 30 · 31 · 32 . . . 91 · Next

Message boards : Number crunching : New work Discussion

©2024 cpdn.org