Message boards : Cafe CPDN : Op Ed Piece in the NY Times Regarding Climate Model Computing
Message board moderation
Author | Message |
---|---|
Send message Joined: 27 Feb 08 Posts: 41 Credit: 1,402,356 RAC: 0 |
Interesting piece from the NY Times regarding computer models for climate change research. https://www.nytimes.com/2019/06/12/opinion/climate-change-supercomputers.html?action=click&module=Well&pgtype=Homepage§ion=Opinion Regards, Bob P. |
Send message Joined: 15 May 09 Posts: 4538 Credit: 19,004,017 RAC: 21,574 |
Unfortunately access to the article linked to in the NY Times piece isn't free to accessbut my understanding of the climate models that we run here is that they use sequential computing with each calculation depending on the result from the previous one. In that scenario, the increased computing power of a supercomputer may not give as much benefit as is suggested. What is not in doubt is that switching from 25Km squares (the highest resolution of current model types) to 1Km squares would make tasks even on the fastest PC's take several months. |
Send message Joined: 27 Feb 08 Posts: 41 Credit: 1,402,356 RAC: 0 |
Unfortunately access to the article linked to in the NY Times piece isn't free to access but my understanding of the climate models that we run here is that they use sequential computing with each calculation depending on the result from the previous one. In that scenario, the increased computing power of a supercomputer may not give as much benefit as is suggested. This link may be better (hopefully!). https://www.nytimes.com/2019/06/12/opinion/climate-change-supercomputers.html Regards, Bob P. |
Send message Joined: 5 Sep 04 Posts: 7629 Credit: 24,240,330 RAC: 0 |
I think that it's GPUs in parallel that aren't of much use. Supercomputers are what the UK Met Office use. And the new models that are being tested on and off here, are large hi-res models. Which use LOTS of memory and disk space. |
Send message Joined: 15 Jan 06 Posts: 637 Credit: 26,751,529 RAC: 653 |
Which use LOTS of memory and disk space. They aren't using any at the moment. I am waiting (with lots of each). |
Send message Joined: 5 Aug 04 Posts: 1120 Credit: 17,202,915 RAC: 2,154 |
I am waiting too. The last few that completed successfully were 22 Nov 2018, 21:08:08 UTC. There were a few after that, but all of those failed except this one: 14 Jan 2019, 20:06:36 UTC. |
Send message Joined: 5 Sep 04 Posts: 7629 Credit: 24,240,330 RAC: 0 |
No one waiting has nearly enough memory at present. If you set your computer to 1 or 2 tasks max, they'll cope, but then there's the slow speed of a lot of the laptops. I don't want to talk too much about it at present; requirements will probably change before release. And that may well be for the worse. |
Send message Joined: 15 May 09 Posts: 4538 Credit: 19,004,017 RAC: 21,574 |
This link may be better (hopefully!). Sadly not, I can read all of the NY Times piece but not the pear reviewed article it links to. |
Send message Joined: 15 Jan 06 Posts: 637 Credit: 26,751,529 RAC: 653 |
No one waiting has nearly enough memory at present. That is useful to know. I had set it to download on machines with "only" 16 GB of memory, but now I will reserve it for the Ivy Bridge with 32 GB. One or two cores will do on that machine. I don't have anything better to do with it. (But I hope it does not need more than 183 GB of disk space. I am a bit limited on that machine.) |
Send message Joined: 5 Aug 04 Posts: 1120 Credit: 17,202,915 RAC: 2,154 |
I have a spare slot where I could put one of these in: Seagate IronWolf 3TB NAS Hard Drive – Internal, 3TB Capacity, 3.5” Form Factor, 5900 RPM Spindle Speed, SATA 6GB/s, 64MB Buffer - ST3000VN007 Would it be fast enough? Price seems to be a trifle less than $100. I have 16GBytes of RAM and it is mostly not doing anything: $ free total used free shared buffers cached Mem: 16254616 13177616 3077000 33252 1010940 8103068 -/+ buffers/cache: 4063608 12191008 Swap: 4095996 95020 4000976 Eight 2 GByte modules. To go any larger, I would need to replace those with larger modules. |
Send message Joined: 15 May 09 Posts: 4538 Credit: 19,004,017 RAC: 21,574 |
requirements will probably change before release. And that may well be for the worse. OK time to order that new cpu and motherboard. Will go for 32GB memory initially but with space to add more when I can afford to increase it enough to get more cores running. |
Send message Joined: 7 Aug 04 Posts: 2187 Credit: 64,822,615 RAC: 5,275 |
requirements will probably change before release. And that may well be for the worse. What CPU are you going for Dave? |
Send message Joined: 5 Sep 04 Posts: 7629 Credit: 24,240,330 RAC: 0 |
I think that it should be more a matter of planning what to do/buy, rather than jumping in now. Who knows what hardware may show up just before these big models are finally released. |
Send message Joined: 15 May 09 Posts: 4538 Credit: 19,004,017 RAC: 21,574 |
requirements will probably change before release. And that may well be for the worse. Was thinking of AMD Ryzen 3 2200G - 3.5 GHz 4 cores 4 threads 2 MB cache. I can't afford the latest and greatest sadly. I guess I should also be looking at Tom's Hardware etc. and researching bang per buck. Who knows what hardware may show up just before these big models are finally released. True and while I may not be able to afford the new hardware it may affect what does come within my budget. No great rush as desktop still has over a month of main site work to go on it once a couple of testing tasks are completed. |
Send message Joined: 15 Jan 06 Posts: 637 Credit: 26,751,529 RAC: 653 |
My feeling at the moment is that the CPU does not matter much. Newer will always be better, but it is really a question of getting enough memory. Lots and lots (whatever that is) will apparently be required. On the other hand, it has to be within the bounds of what consumer PCs can do, or CPDN won't get much crunching done. So that is how I am arranging my machines now; I have even ordered more memory in order to do 64 GB in one machine. I can try it out on both Intel (Coffee Lake) and Ryzen CPUs, but that probably won't make much difference in the end. And SSDs are cheap too, if it needs more. We just need the stuff to test now. By the way, the Intel motherboards can now take 128 GB of memory if someone ones to try it. I will bow out at that point. |
Send message Joined: 15 May 09 Posts: 4538 Credit: 19,004,017 RAC: 21,574 |
By the way, the Intel motherboards can now take 128 GB of memory if someone ones to try it. I will bow out at that point. Me too. Also, I see that some of the cheaper MB/CPU bundles have motherboards that only take up to 32GB ram. Seems a bit mad when the pairing includes an 8core cpu. |
Send message Joined: 5 Aug 04 Posts: 1120 Credit: 17,202,915 RAC: 2,154 |
By the way, the Intel motherboards can now take 128 GB of memory if someone ones to try it. I will bow out at that point. My mother board will take up to 512 GBytes of RAM (if someone wants to pay me to put it in). There are 16 RAM slots, so I would need 32 Gig modules. Currently I have eight 2 GByte modules. And to use the other 8 slots, I would need to add another processor chip (my motherboard will take two) -- if someone wants to pay me to do that. I do not expect anyone would want to do that. ;-) Might as well do your own machine. I see 16 GByte modules are available, but I do not see any 32 GByte ones. |
Send message Joined: 15 May 09 Posts: 4538 Credit: 19,004,017 RAC: 21,574 |
I see 16 GByte modules are available, but I do not see any 32 GByte ones. They are out there https://www.scan.co.uk/products/32gb-crucial-server-memory-module-ddr3-pc3-10600-(1333)-240-pins-ecc-registered-cas-9-135v But at £305 a throw I don't expect to be ordering any soon! |
Send message Joined: 3 Sep 04 Posts: 126 Credit: 26,610,380 RAC: 3,377 |
I see that some of the cheaper MB/CPU bundles have motherboards that only take up to 32GB ram. Seems a bit mad when the pairing includes an 8core cpu. Each task uses at most 600 MB of ram currently. |
Send message Joined: 5 Sep 04 Posts: 7629 Credit: 24,240,330 RAC: 0 |
Yes, so it'll come as a bit of a shock to people. Consider this info as a pre warning. More will be posted when they get closer to release, which may not be for months. And there'l still be the "smaller" models, that everyone is used to. |
©2024 cpdn.org