climateprediction.net (CPDN) home page
Thread 'Op Ed Piece in the NY Times Regarding Climate Model Computing'

Thread 'Op Ed Piece in the NY Times Regarding Climate Model Computing'

Message boards : Cafe CPDN : Op Ed Piece in the NY Times Regarding Climate Model Computing
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
rbpeake

Send message
Joined: 27 Feb 08
Posts: 41
Credit: 1,402,356
RAC: 0
Message 60273 - Posted: 12 Jun 2019, 13:13:59 UTC

Interesting piece from the NY Times regarding computer models for climate change research.
https://www.nytimes.com/2019/06/12/opinion/climate-change-supercomputers.html?action=click&module=Well&pgtype=Homepage&section=Opinion
Regards,
Bob P.
ID: 60273 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,005,674
RAC: 21,647
Message 60274 - Posted: 12 Jun 2019, 15:05:41 UTC - in response to Message 60273.  

Unfortunately access to the article linked to in the NY Times piece isn't free to accessbut my understanding of the climate models that we run here is that they use sequential computing with each calculation depending on the result from the previous one. In that scenario, the increased computing power of a supercomputer may not give as much benefit as is suggested.

What is not in doubt is that switching from 25Km squares (the highest resolution of current model types) to 1Km squares would make tasks even on the fastest PC's take several months.
ID: 60274 · Report as offensive     Reply Quote
rbpeake

Send message
Joined: 27 Feb 08
Posts: 41
Credit: 1,402,356
RAC: 0
Message 60275 - Posted: 12 Jun 2019, 20:29:05 UTC - in response to Message 60274.  

Unfortunately access to the article linked to in the NY Times piece isn't free to access but my understanding of the climate models that we run here is that they use sequential computing with each calculation depending on the result from the previous one. In that scenario, the increased computing power of a supercomputer may not give as much benefit as is suggested.

What is not in doubt is that switching from 25Km squares (the highest resolution of current model types) to 1Km squares would make tasks even on the fastest PC's take several months.

This link may be better (hopefully!).
https://www.nytimes.com/2019/06/12/opinion/climate-change-supercomputers.html
Regards,
Bob P.
ID: 60275 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60280 - Posted: 12 Jun 2019, 21:17:50 UTC

I think that it's GPUs in parallel that aren't of much use.
Supercomputers are what the UK Met Office use.

And the new models that are being tested on and off here, are large hi-res models.
Which use LOTS of memory and disk space.
ID: 60280 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 60281 - Posted: 12 Jun 2019, 21:23:29 UTC - in response to Message 60280.  

Which use LOTS of memory and disk space.

They aren't using any at the moment. I am waiting (with lots of each).
ID: 60281 · Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 60282 - Posted: 13 Jun 2019, 0:28:53 UTC - in response to Message 60281.  

I am waiting too. The last few that completed successfully were 22 Nov 2018, 21:08:08 UTC. There were a few after that, but all of those failed except this one: 14 Jan 2019, 20:06:36 UTC.
ID: 60282 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60285 - Posted: 13 Jun 2019, 2:29:42 UTC

No one waiting has nearly enough memory at present.

If you set your computer to 1 or 2 tasks max, they'll cope, but then there's the slow speed of a lot of the laptops.

I don't want to talk too much about it at present; requirements will probably change before release. And that may well be for the worse.
ID: 60285 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,005,674
RAC: 21,647
Message 60288 - Posted: 13 Jun 2019, 8:28:57 UTC

This link may be better (hopefully!).
https://www.nytimes.com/2019/06/12/opinion/climate-change-supercomputers.html


Sadly not, I can read all of the NY Times piece but not the pear reviewed article it links to.
ID: 60288 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 60291 - Posted: 13 Jun 2019, 11:43:57 UTC - in response to Message 60285.  
Last modified: 13 Jun 2019, 11:53:09 UTC

No one waiting has nearly enough memory at present.

That is useful to know. I had set it to download on machines with "only" 16 GB of memory, but now I will reserve it for the Ivy Bridge with 32 GB.
One or two cores will do on that machine. I don't have anything better to do with it.
(But I hope it does not need more than 183 GB of disk space. I am a bit limited on that machine.)
ID: 60291 · Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 60294 - Posted: 13 Jun 2019, 12:40:01 UTC - in response to Message 60291.  

I have a spare slot where I could put one of these in:

Seagate IronWolf 3TB NAS Hard Drive – Internal, 3TB Capacity, 3.5” Form Factor, 5900 RPM Spindle Speed, SATA 6GB/s, 64MB Buffer - ST3000VN007

Would it be fast enough? Price seems to be a trifle less than $100.

I have 16GBytes of RAM and it is mostly not doing anything:

$ free
total used free shared buffers cached
Mem: 16254616 13177616 3077000 33252 1010940 8103068
-/+ buffers/cache: 4063608 12191008
Swap: 4095996 95020 4000976

Eight 2 GByte modules. To go any larger, I would need to replace those with larger modules.
ID: 60294 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,005,674
RAC: 21,647
Message 60295 - Posted: 13 Jun 2019, 12:58:14 UTC

requirements will probably change before release. And that may well be for the worse.


OK time to order that new cpu and motherboard. Will go for 32GB memory initially but with space to add more when I can afford to increase it enough to get more cores running.
ID: 60295 · Report as offensive     Reply Quote
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 60298 - Posted: 13 Jun 2019, 13:42:21 UTC - in response to Message 60295.  

requirements will probably change before release. And that may well be for the worse.


OK time to order that new cpu and motherboard. Will go for 32GB memory initially but with space to add more when I can afford to increase it enough to get more cores running.


What CPU are you going for Dave?
ID: 60298 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60302 - Posted: 13 Jun 2019, 22:46:11 UTC

I think that it should be more a matter of planning what to do/buy, rather than jumping in now.

Who knows what hardware may show up just before these big models are finally released.
ID: 60302 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,005,674
RAC: 21,647
Message 60303 - Posted: 14 Jun 2019, 7:42:57 UTC - in response to Message 60298.  
Last modified: 14 Jun 2019, 7:49:42 UTC

requirements will probably change before release. And that may well be for the worse.


OK time to order that new cpu and motherboard. Will go for 32GB memory initially but with space to add more when I can afford to increase it enough to get more cores running.


What CPU are you going for Dave?


Was thinking of AMD Ryzen 3 2200G - 3.5 GHz 4 cores 4 threads 2 MB cache. I can't afford the latest and greatest sadly. I guess I should also be looking at Tom's Hardware etc. and researching bang per buck.

Who knows what hardware may show up just before these big models are finally released.


True and while I may not be able to afford the new hardware it may affect what does come within my budget.

No great rush as desktop still has over a month of main site work to go on it once a couple of testing tasks are completed.
ID: 60303 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 60304 - Posted: 14 Jun 2019, 8:32:51 UTC
Last modified: 14 Jun 2019, 8:53:25 UTC

My feeling at the moment is that the CPU does not matter much. Newer will always be better, but it is really a question of getting enough memory. Lots and lots (whatever that is) will apparently be required. On the other hand, it has to be within the bounds of what consumer PCs can do, or CPDN won't get much crunching done.

So that is how I am arranging my machines now; I have even ordered more memory in order to do 64 GB in one machine. I can try it out on both Intel (Coffee Lake) and Ryzen CPUs, but that probably won't make much difference in the end. And SSDs are cheap too, if it needs more. We just need the stuff to test now.

By the way, the Intel motherboards can now take 128 GB of memory if someone ones to try it. I will bow out at that point.
ID: 60304 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,005,674
RAC: 21,647
Message 60306 - Posted: 14 Jun 2019, 9:59:51 UTC - in response to Message 60304.  

By the way, the Intel motherboards can now take 128 GB of memory if someone ones to try it. I will bow out at that point.


Me too. Also, I see that some of the cheaper MB/CPU bundles have motherboards that only take up to 32GB ram. Seems a bit mad when the pairing includes an 8core cpu.
ID: 60306 · Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1120
Credit: 17,202,915
RAC: 2,154
Message 60309 - Posted: 14 Jun 2019, 11:20:35 UTC - in response to Message 60304.  

By the way, the Intel motherboards can now take 128 GB of memory if someone ones to try it. I will bow out at that point.


My mother board will take up to 512 GBytes of RAM (if someone wants to pay me to put it in). There are 16 RAM slots, so I would need 32 Gig modules. Currently I have eight 2 GByte modules. And to use the other 8 slots, I would need to add another processor chip (my motherboard will take two) -- if someone wants to pay me to do that. I do not expect anyone would want to do that. ;-) Might as well do your own machine.

I see 16 GByte modules are available, but I do not see any 32 GByte ones.
ID: 60309 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4538
Credit: 19,005,674
RAC: 21,647
Message 60310 - Posted: 14 Jun 2019, 13:20:11 UTC

I see 16 GByte modules are available, but I do not see any 32 GByte ones.


They are out there https://www.scan.co.uk/products/32gb-crucial-server-memory-module-ddr3-pc3-10600-(1333)-240-pins-ecc-registered-cas-9-135v

But at £305 a throw I don't expect to be ordering any soon!
ID: 60310 · Report as offensive     Reply Quote
Alex Plantema

Send message
Joined: 3 Sep 04
Posts: 126
Credit: 26,610,380
RAC: 3,377
Message 60312 - Posted: 14 Jun 2019, 20:31:25 UTC - in response to Message 60306.  

I see that some of the cheaper MB/CPU bundles have motherboards that only take up to 32GB ram. Seems a bit mad when the pairing includes an 8core cpu.

Each task uses at most 600 MB of ram currently.
ID: 60312 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60313 - Posted: 14 Jun 2019, 21:46:40 UTC

Yes, so it'll come as a bit of a shock to people.
Consider this info as a pre warning.

More will be posted when they get closer to release, which may not be for months.

And there'l still be the "smaller" models, that everyone is used to.
ID: 60313 · Report as offensive     Reply Quote
1 · 2 · Next

Message boards : Cafe CPDN : Op Ed Piece in the NY Times Regarding Climate Model Computing

©2024 cpdn.org