 |
|
Message boards :
Number crunching :
OpenIFS Discussion
Message board moderation
Previous · 1 . . . 21 · 22 · 23 · 24 · 25 · 26 · 27 . . . 32 · Next
| Author | Message |
|---|---|
 Dave JacksonSend message Joined: 15 May 09 Posts: 4535 Credit: 18,984,965 RAC: 21,892 |
Not a problem, it was just an observation. |
|
Send message Joined: 4 Oct 15 Posts: 34 Credit: 9,075,151 RAC: 374 |
I have a small Problem with those BL WUs: Had no Problems with PS WUs. Here a Link to one of my WUs: https://www.cpdn.org/result.php?resultid=22311544 Mo 13 Feb 2023 23:47:02 CET | climateprediction.net | Computation for task oifs_43r3_bl_a26b_2016092300_15_991_12209606_0 finished Mo 13 Feb 2023 23:47:06 CET | climateprediction.net | Output file oifs_43r3_bl_a26b_2016092300_15_991_12209606_0_r1758202807_7.zip for task oifs_43r3_bl_a26b_2016092300_15_991_12209606_0 absent Mo 13 Feb 2023 23:47:06 CET | climateprediction.net | Output file oifs_43r3_bl_a26b_2016092300_15_991_12209606_0_r1758202807_8.zip for task oifs_43r3_bl_a26b_2016092300_15_991_12209606_0 absent Mo 13 Feb 2023 23:47:06 CET | climateprediction.net | Output file oifs_43r3_bl_a26b_2016092300_15_991_12209606_0_r1758202807_9.zip for task oifs_43r3_bl_a26b_2016092300_15_991_12209606_0 absent Mo 13 Feb 2023 23:47:06 CET | climateprediction.net | Output file oifs_43r3_bl_a26b_2016092300_15_991_12209606_0_r1758202807_10.zip for task oifs_43r3_bl_a26b_2016092300_15_991_12209606_0 absent Mo 13 Feb 2023 23:47:06 CET | climateprediction.net | Output file oifs_43r3_bl_a26b_2016092300_15_991_12209606_0_r1758202807_11.zip for task oifs_43r3_bl_a26b_2016092300_15_991_12209606_0 absent Mo 13 Feb 2023 23:47:06 CET | climateprediction.net | Output file oifs_43r3_bl_a26b_2016092300_15_991_12209606_0_r1758202807_12.zip for task oifs_43r3_bl_a26b_2016092300_15_991_12209606_0 absent Mo 13 Feb 2023 23:47:06 CET | climateprediction.net | Output file oifs_43r3_bl_a26b_2016092300_15_991_12209606_0_r1758202807_13.zip for task oifs_43r3_bl_a26b_2016092300_15_991_12209606_0 absent Mo 13 Feb 2023 23:47:06 CET | climateprediction.net | Output file oifs_43r3_bl_a26b_2016092300_15_991_12209606_0_r1758202807_14.zip for task oifs_43r3_bl_a26b_2016092300_15_991_12209606_0 absent Mo 13 Feb 2023 23:47:06 CET | climateprediction.net | Starting task oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0 Mo 13 Feb 2023 23:53:18 CET | climateprediction.net | Computation for task oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0 finished Mo 13 Feb 2023 23:53:18 CET | climateprediction.net | Output file oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0_r1856183219_0.zip for task oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0 absent Mo 13 Feb 2023 23:53:18 CET | climateprediction.net | Output file oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0_r1856183219_1.zip for task oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0 absent Mo 13 Feb 2023 23:53:18 CET | climateprediction.net | Output file oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0_r1856183219_2.zip for task oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0 absent Mo 13 Feb 2023 23:53:18 CET | climateprediction.net | Output file oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0_r1856183219_3.zip for task oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0 absent Mo 13 Feb 2023 23:53:18 CET | climateprediction.net | Output file oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0_r1856183219_4.zip for task oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0 absent Mo 13 Feb 2023 23:53:18 CET | climateprediction.net | Output file oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0_r1856183219_5.zip for task oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0 absent Mo 13 Feb 2023 23:53:18 CET | climateprediction.net | Output file oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0_r1856183219_6.zip for task oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0 absent Mo 13 Feb 2023 23:53:18 CET | climateprediction.net | Output file oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0_r1856183219_7.zip for task oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0 absent Mo 13 Feb 2023 23:53:18 CET | climateprediction.net | Output file oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0_r1856183219_8.zip for task oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0 absent Mo 13 Feb 2023 23:53:18 CET | climateprediction.net | Output file oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0_r1856183219_9.zip for task oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0 absent Mo 13 Feb 2023 23:53:18 CET | climateprediction.net | Output file oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0_r1856183219_10.zip for task oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0 absent Mo 13 Feb 2023 23:53:18 CET | climateprediction.net | Output file oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0_r1856183219_11.zip for task oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0 absent Mo 13 Feb 2023 23:53:18 CET | climateprediction.net | Output file oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0_r1856183219_12.zip for task oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0 absent Mo 13 Feb 2023 23:53:18 CET | climateprediction.net | Output file oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0_r1856183219_13.zip for task oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0 absent Mo 13 Feb 2023 23:53:18 CET | climateprediction.net | Output file oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0_r1856183219_14.zip for task oifs_43r3_bl_a2nz_2016092300_15_991_12210242_0 absent Mo 13 Feb 2023 23:53:18 CET | climateprediction.net | Starting task oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0 Mo 13 Feb 2023 23:53:22 CET | climateprediction.net | Computation for task oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0 finished Mo 13 Feb 2023 23:53:22 CET | climateprediction.net | Output file oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0_r31801520_0.zip for task oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0 absent Mo 13 Feb 2023 23:53:22 CET | climateprediction.net | Output file oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0_r31801520_1.zip for task oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0 absent Mo 13 Feb 2023 23:53:22 CET | climateprediction.net | Output file oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0_r31801520_2.zip for task oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0 absent Mo 13 Feb 2023 23:53:22 CET | climateprediction.net | Output file oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0_r31801520_3.zip for task oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0 absent Mo 13 Feb 2023 23:53:22 CET | climateprediction.net | Output file oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0_r31801520_4.zip for task oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0 absent Mo 13 Feb 2023 23:53:22 CET | climateprediction.net | Output file oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0_r31801520_5.zip for task oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0 absent Mo 13 Feb 2023 23:53:22 CET | climateprediction.net | Output file oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0_r31801520_6.zip for task oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0 absent Mo 13 Feb 2023 23:53:22 CET | climateprediction.net | Output file oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0_r31801520_7.zip for task oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0 absent Mo 13 Feb 2023 23:53:22 CET | climateprediction.net | Output file oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0_r31801520_8.zip for task oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0 absent Mo 13 Feb 2023 23:53:22 CET | climateprediction.net | Output file oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0_r31801520_9.zip for task oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0 absent Mo 13 Feb 2023 23:53:22 CET | climateprediction.net | Output file oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0_r31801520_10.zip for task oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0 absent Mo 13 Feb 2023 23:53:22 CET | climateprediction.net | Output file oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0_r31801520_11.zip for task oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0 absent Mo 13 Feb 2023 23:53:22 CET | climateprediction.net | Output file oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0_r31801520_12.zip for task oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0 absent Mo 13 Feb 2023 23:53:22 CET | climateprediction.net | Output file oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0_r31801520_13.zip for task oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0 absent Mo 13 Feb 2023 23:53:22 CET | climateprediction.net | Output file oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0_r31801520_14.zip for task oifs_43r3_bl_a2nv_2016092300_15_991_12210238_0 absent Mo 13 Feb 2023 23:53:22 CET | climateprediction.net | Starting task oifs_43r3_bl_a31n_2016092300_15_991_12210734_0 Mo 13 Feb 2023 23:53:26 CET | climateprediction.net | Computation for task oifs_43r3_bl_a31n_2016092300_15_991_12210734_0 finished Mo 13 Feb 2023 23:53:26 CET | climateprediction.net | Output file oifs_43r3_bl_a31n_2016092300_15_991_12210734_0_r1142312962_0.zip for task oifs_43r3_bl_a31n_2016092300_15_991_12210734_0 absent Mo 13 Feb 2023 23:53:26 CET | climateprediction.net | Output file oifs_43r3_bl_a31n_2016092300_15_991_12210734_0_r1142312962_1.zip for task oifs_43r3_bl_a31n_2016092300_15_991_12210734_0 absent Mo 13 Feb 2023 23:53:26 CET | climateprediction.net | Output file oifs_43r3_bl_a31n_2016092300_15_991_12210734_0_r1142312962_2.zip for task oifs_43r3_bl_a31n_2016092300_15_991_12210734_0 absent Mo 13 Feb 2023 23:53:26 CET | climateprediction.net | Output file oifs_43r3_bl_a31n_2016092300_15_991_12210734_0_r1142312962_3.zip for task oifs_43r3_bl_a31n_2016092300_15_991_12210734_0 absent Mo 13 Feb 2023 23:53:26 CET | climateprediction.net | Output file oifs_43r3_bl_a31n_2016092300_15_991_12210734_0_r1142312962_4.zip for task oifs_43r3_bl_a31n_2016092300_15_991_12210734_0 absent Mo 13 Feb 2023 23:53:26 CET | climateprediction.net | Output file oifs_43r3_bl_a31n_2016092300_15_991_12210734_0_r1142312962_5.zip for task oifs_43r3_bl_a31n_2016092300_15_991_12210734_0 absent Mo 13 Feb 2023 23:53:26 CET | climateprediction.net | Output file oifs_43r3_bl_a31n_2016092300_15_991_12210734_0_r1142312962_6.zip for task oifs_43r3_bl_a31n_2016092300_15_991_12210734_0 absent Mo 13 Feb 2023 23:53:26 CET | climateprediction.net | Output file oifs_43r3_bl_a31n_2016092300_15_991_12210734_0_r1142312962_7.zip for task oifs_43r3_bl_a31n_2016092300_15_991_12210734_0 absent Mo 13 Feb 2023 23:53:26 CET | climateprediction.net | Output file oifs_43r3_bl_a31n_2016092300_15_991_12210734_0_r1142312962_8.zip for task oifs_43r3_bl_a31n_2016092300_15_991_12210734_0 absent Mo 13 Feb 2023 23:53:26 CET | climateprediction.net | Output file oifs_43r3_bl_a31n_2016092300_15_991_12210734_0_r1142312962_9.zip for task oifs_43r3_bl_a31n_2016092300_15_991_12210734_0 absent Mo 13 Feb 2023 23:53:26 CET | climateprediction.net | Output file oifs_43r3_bl_a31n_2016092300_15_991_12210734_0_r1142312962_10.zip for task oifs_43r3_bl_a31n_2016092300_15_991_12210734_0 absent Mo 13 Feb 2023 23:53:26 CET | climateprediction.net | Output file oifs_43r3_bl_a31n_2016092300_15_991_12210734_0_r1142312962_11.zip for task oifs_43r3_bl_a31n_2016092300_15_991_12210734_0 absent Mo 13 Feb 2023 23:53:26 CET | climateprediction.net | Output file oifs_43r3_bl_a31n_2016092300_15_991_12210734_0_r1142312962_12.zip for task oifs_43r3_bl_a31n_2016092300_15_991_12210734_0 absent Mo 13 Feb 2023 23:53:26 CET | climateprediction.net | Output file oifs_43r3_bl_a31n_2016092300_15_991_12210734_0_r1142312962_13.zip for task oifs_43r3_bl_a31n_2016092300_15_991_12210734_0 absent Mo 13 Feb 2023 23:53:26 CET | climateprediction.net | Output file oifs_43r3_bl_a31n_2016092300_15_991_12210734_0_r1142312962_14.zip for task oifs_43r3_bl_a31n_2016092300_15_991_12210734_0 absent Mo 13 Feb 2023 23:53:26 CET | climateprediction.net | Starting task oifs_43r3_bl_a316_2016092300_15_991_12210717_0 Mo 13 Feb 2023 23:53:30 CET | climateprediction.net | Computation for task oifs_43r3_bl_a316_2016092300_15_991_12210717_0 finished Mo 13 Feb 2023 23:53:30 CET | climateprediction.net | Output file oifs_43r3_bl_a316_2016092300_15_991_12210717_0_r815290613_0.zip for task oifs_43r3_bl_a316_2016092300_15_991_12210717_0 absent Mo 13 Feb 2023 23:53:30 CET | climateprediction.net | Output file oifs_43r3_bl_a316_2016092300_15_991_12210717_0_r815290613_1.zip for task oifs_43r3_bl_a316_2016092300_15_991_12210717_0 absent Mo 13 Feb 2023 23:53:30 CET | climateprediction.net | Output file oifs_43r3_bl_a316_2016092300_15_991_12210717_0_r815290613_2.zip for task oifs_43r3_bl_a316_2016092300_15_991_12210717_0 absent Mo 13 Feb 2023 23:53:30 CET | climateprediction.net | Output file oifs_43r3_bl_a316_2016092300_15_991_12210717_0_r815290613_3.zip for task oifs_43r3_bl_a316_2016092300_15_991_12210717_0 absent Mo 13 Feb 2023 23:53:30 CET | climateprediction.net | Output file oifs_43r3_bl_a316_2016092300_15_991_12210717_0_r815290613_4.zip for task oifs_43r3_bl_a316_2016092300_15_991_12210717_0 absent Mo 13 Feb 2023 23:53:30 CET | climateprediction.net | Output file oifs_43r3_bl_a316_2016092300_15_991_12210717_0_r815290613_5.zip for task oifs_43r3_bl_a316_2016092300_15_991_12210717_0 absent Mo 13 Feb 2023 23:53:30 CET | climateprediction.net | Output file oifs_43r3_bl_a316_2016092300_15_991_12210717_0_r815290613_6.zip for task oifs_43r3_bl_a316_2016092300_15_991_12210717_0 absent Mo 13 Feb 2023 23:53:30 CET | climateprediction.net | Output file oifs_43r3_bl_a316_2016092300_15_991_12210717_0_r815290613_7.zip for task oifs_43r3_bl_a316_2016092300_15_991_12210717_0 absent Mo 13 Feb 2023 23:53:30 CET | climateprediction.net | Output file oifs_43r3_bl_a316_2016092300_15_991_12210717_0_r815290613_8.zip for task oifs_43r3_bl_a316_2016092300_15_991_12210717_0 absent Mo 13 Feb 2023 23:53:30 CET | climateprediction.net | Output file oifs_43r3_bl_a316_2016092300_15_991_12210717_0_r815290613_9.zip for task oifs_43r3_bl_a316_2016092300_15_991_12210717_0 absent Mo 13 Feb 2023 23:53:30 CET | climateprediction.net | Output file oifs_43r3_bl_a316_2016092300_15_991_12210717_0_r815290613_10.zip for task oifs_43r3_bl_a316_2016092300_15_991_12210717_0 absent Mo 13 Feb 2023 23:53:30 CET | climateprediction.net | Output file oifs_43r3_bl_a316_2016092300_15_991_12210717_0_r815290613_11.zip for task oifs_43r3_bl_a316_2016092300_15_991_12210717_0 absent Mo 13 Feb 2023 23:53:30 CET | climateprediction.net | Output file oifs_43r3_bl_a316_2016092300_15_991_12210717_0_r815290613_12.zip for task oifs_43r3_bl_a316_2016092300_15_991_12210717_0 absent Mo 13 Feb 2023 23:53:30 CET | climateprediction.net | Output file oifs_43r3_bl_a316_2016092300_15_991_12210717_0_r815290613_13.zip for task oifs_43r3_bl_a316_2016092300_15_991_12210717_0 absent Mo 13 Feb 2023 23:53:30 CET | climateprediction.net | Output file oifs_43r3_bl_a316_2016092300_15_991_12210717_0_r815290613_14.zip for task oifs_43r3_bl_a316_2016092300_15_991_12210717_0 absent Mo 13 Feb 2023 23:53:30 CET | climateprediction.net | Starting task oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0 Mo 13 Feb 2023 23:53:34 CET | climateprediction.net | Computation for task oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0 finished Mo 13 Feb 2023 23:53:34 CET | climateprediction.net | Output file oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0_r921471673_0.zip for task oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0 absent Mo 13 Feb 2023 23:53:34 CET | climateprediction.net | Output file oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0_r921471673_1.zip for task oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0 absent Mo 13 Feb 2023 23:53:34 CET | climateprediction.net | Output file oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0_r921471673_2.zip for task oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0 absent Mo 13 Feb 2023 23:53:34 CET | climateprediction.net | Output file oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0_r921471673_3.zip for task oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0 absent Mo 13 Feb 2023 23:53:34 CET | climateprediction.net | Output file oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0_r921471673_4.zip for task oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0 absent Mo 13 Feb 2023 23:53:34 CET | climateprediction.net | Output file oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0_r921471673_5.zip for task oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0 absent Mo 13 Feb 2023 23:53:34 CET | climateprediction.net | Output file oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0_r921471673_6.zip for task oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0 absent Mo 13 Feb 2023 23:53:34 CET | climateprediction.net | Output file oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0_r921471673_7.zip for task oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0 absent Mo 13 Feb 2023 23:53:34 CET | climateprediction.net | Output file oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0_r921471673_8.zip for task oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0 absent Mo 13 Feb 2023 23:53:34 CET | climateprediction.net | Output file oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0_r921471673_9.zip for task oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0 absent Mo 13 Feb 2023 23:53:34 CET | climateprediction.net | Output file oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0_r921471673_10.zip for task oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0 absent Mo 13 Feb 2023 23:53:34 CET | climateprediction.net | Output file oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0_r921471673_11.zip for task oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0 absent Mo 13 Feb 2023 23:53:34 CET | climateprediction.net | Output file oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0_r921471673_12.zip for task oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0 absent Mo 13 Feb 2023 23:53:34 CET | climateprediction.net | Output file oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0_r921471673_13.zip for task oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0 absent Mo 13 Feb 2023 23:53:34 CET | climateprediction.net | Output file oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0_r921471673_14.zip for task oifs_43r3_bl_a3hk_2016092300_15_991_12211307_0 absent Mo 13 Feb 2023 23:53:34 CET | climateprediction.net | Starting task oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0 Mo 13 Feb 2023 23:53:39 CET | climateprediction.net | Computation for task oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0 finished Mo 13 Feb 2023 23:53:39 CET | climateprediction.net | Output file oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0_r1613172252_0.zip for task oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0 absent Mo 13 Feb 2023 23:53:39 CET | climateprediction.net | Output file oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0_r1613172252_1.zip for task oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0 absent Mo 13 Feb 2023 23:53:39 CET | climateprediction.net | Output file oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0_r1613172252_2.zip for task oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0 absent Mo 13 Feb 2023 23:53:39 CET | climateprediction.net | Output file oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0_r1613172252_3.zip for task oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0 absent Mo 13 Feb 2023 23:53:39 CET | climateprediction.net | Output file oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0_r1613172252_4.zip for task oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0 absent Mo 13 Feb 2023 23:53:39 CET | climateprediction.net | Output file oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0_r1613172252_5.zip for task oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0 absent Mo 13 Feb 2023 23:53:39 CET | climateprediction.net | Output file oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0_r1613172252_6.zip for task oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0 absent Mo 13 Feb 2023 23:53:39 CET | climateprediction.net | Output file oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0_r1613172252_7.zip for task oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0 absent Mo 13 Feb 2023 23:53:39 CET | climateprediction.net | Output file oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0_r1613172252_8.zip for task oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0 absent Mo 13 Feb 2023 23:53:39 CET | climateprediction.net | Output file oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0_r1613172252_9.zip for task oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0 absent Mo 13 Feb 2023 23:53:39 CET | climateprediction.net | Output file oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0_r1613172252_10.zip for task oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0 absent Mo 13 Feb 2023 23:53:39 CET | climateprediction.net | Output file oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0_r1613172252_11.zip for task oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0 absent Mo 13 Feb 2023 23:53:39 CET | climateprediction.net | Output file oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0_r1613172252_12.zip for task oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0 absent Mo 13 Feb 2023 23:53:39 CET | climateprediction.net | Output file oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0_r1613172252_13.zip for task oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0 absent Mo 13 Feb 2023 23:53:39 CET | climateprediction.net | Output file oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0_r1613172252_14.zip for task oifs_43r3_bl_a3i0_2016092300_15_991_12211323_0 absent Mo 13 Feb 2023 23:53:39 CET | climateprediction.net | Starting task oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0 Mo 13 Feb 2023 23:53:43 CET | climateprediction.net | Computation for task oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0 finished Mo 13 Feb 2023 23:53:43 CET | climateprediction.net | Output file oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0_r1513650664_0.zip for task oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0 absent Mo 13 Feb 2023 23:53:43 CET | climateprediction.net | Output file oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0_r1513650664_1.zip for task oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0 absent Mo 13 Feb 2023 23:53:43 CET | climateprediction.net | Output file oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0_r1513650664_2.zip for task oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0 absent Mo 13 Feb 2023 23:53:43 CET | climateprediction.net | Output file oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0_r1513650664_3.zip for task oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0 absent Mo 13 Feb 2023 23:53:43 CET | climateprediction.net | Output file oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0_r1513650664_4.zip for task oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0 absent Mo 13 Feb 2023 23:53:43 CET | climateprediction.net | Output file oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0_r1513650664_5.zip for task oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0 absent Mo 13 Feb 2023 23:53:43 CET | climateprediction.net | Output file oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0_r1513650664_6.zip for task oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0 absent Mo 13 Feb 2023 23:53:43 CET | climateprediction.net | Output file oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0_r1513650664_7.zip for task oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0 absent Mo 13 Feb 2023 23:53:43 CET | climateprediction.net | Output file oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0_r1513650664_8.zip for task oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0 absent Mo 13 Feb 2023 23:53:43 CET | climateprediction.net | Output file oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0_r1513650664_9.zip for task oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0 absent Mo 13 Feb 2023 23:53:43 CET | climateprediction.net | Output file oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0_r1513650664_10.zip for task oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0 absent Mo 13 Feb 2023 23:53:43 CET | climateprediction.net | Output file oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0_r1513650664_11.zip for task oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0 absent Mo 13 Feb 2023 23:53:43 CET | climateprediction.net | Output file oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0_r1513650664_12.zip for task oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0 absent Mo 13 Feb 2023 23:53:43 CET | climateprediction.net | Output file oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0_r1513650664_13.zip for task oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0 absent Mo 13 Feb 2023 23:53:43 CET | climateprediction.net | Output file oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0_r1513650664_14.zip for task oifs_43r3_bl_a3hx_2016092300_15_991_12211320_0 absent Update: And this one did work at the beginning: https://www.cpdn.org/result.php?resultid=22309804 |
|
Dave Jackson Send message Joined: 15 May 09 Posts: 4535 Credit: 18,984,965 RAC: 21,892 |
Here a Link to one of my WUs: Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12211307.zip /var/lib/boinc-client/slots/1/159l_2/o3chem_l91/O3CHEM06_V2.9 bad CRC 4c885ac3 (should be 54ab7da4) ..Unzipping the climate data file failed Is where the problem is. None of your tasks have completed their second attempt yet so I am holding off on posting any guesses as to the reason. It looks as if you have wisely decided not to try your second Linux box which is much lower on memory with these but it may be worth letting it try just one at a time to see what happens. Edit: Just had a reply from Glen about a failed task on the testing branch. It may be worth doing a project reset. It may be a corrupt file for the model type downloaded. Edit2: Looking at the results in so far as of about 0100UTC when it was updated, 4 out of the 106 fails were file read/copy/unzip fails. About 70 were other but I haven't yet tried looking to see if there is anything linking enough of them to give them their own category. |
|
Send message Joined: 12 Apr 21 Posts: 317 Credit: 14,797,862 RAC: 19,598 |
Had a failure on a newer Ryzen PC with the following code, but there's nothing in the error log, at least that I could pick up on, indicating a problem. Task: https://www.cpdn.org/result.php?resultid=22312560 process exited with code 9 (0x9, -247) |
|
Send message Joined: 29 Oct 17 Posts: 1048 Credit: 16,419,240 RAC: 16,637 |
Errors Had a failure on a newer Ryzen PC with the following code, but there's nothing in the error log, at least that I could pick up on, indicating a problem. Task: https://www.cpdn.org/result.php?resultid=22312560That's one of those annoying tasks where the model ran just fine and then the task either goes wrong in the boinc client (in the boinc_finish function) or maybe as it exits. I know it has something to do with file locks or files not being cleared but exactly what files (whether task or boinc related) we don't (yet) know. Bad CRC[SG] Felix, as Dave has already said, this is a file corruption. Either something went wrong in downloading the file, or when it was copied into the slot dir. There's nothing the task code can do about this. My advice if it keeps happening would be to reset the project but this will Abort your current tasks so only do it if things keep going wrong. Model crashes. There will also be tasks that fail in the model because this experiment is designed to push the limits of what the model can do. If you have a failed task with a very long code traceback in the output, that'll be why. Credit is based on trickles, so credit will be given for work done. I had a look at the batch stats, over 1000 tasks completed already and the error stats show the memory corruption issue is much reduced (less than 5%); the model crashes by comparison are ~10%. |
|
Dave Jackson Send message Joined: 15 May 09 Posts: 4535 Credit: 18,984,965 RAC: 21,892 |
I have posted a question in the BOINC fora about this. I really feel BOINC should be doing CRC checks on all downloads and if the check doesn't match, re-downloading the file. I am awaiting Richard who has responded doing some digging before deciding whether to put in a feature request to git-hub. |
|
Send message Joined: 1 Jan 07 Posts: 1061 Credit: 36,695,025 RAC: 10,194 |
Richard who has responded doing some digging before deciding whether to put in a feature request to git-hub.It may take me a while. My suspicion is - following similar errors at other projects - that sometimes BOINC introduces errors if a long download of a large file is interrupted, and tries to restart from an intermediate "OK so far" point. I think BOINC loses count of how much has been downloaded so far, and how much remains to be downloaded. But that usually results in a byte count mismatch. This case is different: the file in question is a zipped climate data file (clim_data_12211307.zip) identified as belonging to a single specific workunit - WU 12211307 is the source for task 22311544. And the CRC error was detected during unzipping, not downloading. I'd also expect a single-file error like that to affect a single task, not a whole sequence of them (*). It's possible that the file storage system on that host has developed a more significant fault, affecting multiple files. I'll try pulling out a network cable (**) during a download to test the interrupted/lost count theory, but others may have other explanations worth exploring. (*) Edit: I'd also expect a zip CRC error to be detected when a task starts its first run, not on a midway restart. (**) Edit2: Argh - all the tasks have been snapped up. The network cable will have to wait for the next batch. |
|
Send message Joined: 1 Jan 07 Posts: 1061 Credit: 36,695,025 RAC: 10,194 |
Credit is based on trickles, so credit will be given for work done.Is that the last upload file received, or the tiny administrative trickle sent to the scheduler? I'm only seeing administrative trickles at timesteps 864000 and 1295100 (associated with upload zips 9 and 14) for these tasks. If a task fails after upload 8, does it get any credit? I suppose we'll find out on Sunday morning ... |
|
Send message Joined: 5 Aug 04 Posts: 1120 Credit: 17,202,915 RAC: 2,154 |
My first ever double free or corruption Task 22308784 Name oifs_43r3_bl_a1ei_2016092300_15_991_12208605_0 Workunit 12208605 Error tasks for computer 1511241 22308784 12208605 13 Feb 2023, 15:24:42 UTC 14 Feb 2023, 10:48:08 UTC Error while computing 19,858.70 19,708.54 --- OpenIFS 43r3 Baroclinic Lifecycle v1.11 x86_64-pc-linux-gnu End of stderr ... Uploading the intermediate file: upload_file_10.zip 04:46:49 STEP 1057 H= 264:15 +CPU= 19.396 04:47:06 STEP 1058 H= 264:30 +CPU= 17.089 double free or corruption (!prev) </stderr_txt> Would this help? https://linuxhint.com/double-free-corruption-error/ 
|
|
Send message Joined: 5 Aug 04 Posts: 1120 Credit: 17,202,915 RAC: 2,154 |
Here is one. Not recent. 22250807 12166550 19 Dec 2022, 2:23:58 UTC 19 Dec 2022, 9:23:21 UTC Completed 22,360.51 22,031.03 1,232.00 OpenIFS 43r3 Baroclinic Lifecycle v1.07 x86_64-pc-linux-gnu
|
|
Send message Joined: 29 Oct 17 Posts: 1048 Credit: 16,419,240 RAC: 16,637 |
My first ever double free or corruptionIt's a badge of honour, wear it with pride ;) Would this help? https://linuxhint.com/double-free-corruption-error/Not really. I know what's causing the problem but catching/reproducing the fails on my machines so I can debug them is nigh impossible. So it means examining the code very carefully (I didn't write it). There have been a number of fixes recently that have gone in. The wrapper process now running includes most of them and I'm glad to say memory issues now account for ~5% of fails. The development code version includes a couple more fixes. It's priority to get these sorted as it's holding up putting out the higher resolution configurations of OpenIFS (which I thought I'd be able to do last year!). |
|
Send message Joined: 5 Aug 04 Posts: 1120 Credit: 17,202,915 RAC: 2,154 |
My first ever double free or corruption Giggle. Would this help? https://linuxhint.com/double-free-corruption-error/ In the old days, when I worked for a large regulated monopoly we used a program called lint written by Steve Johnson, to examine C (and later, C++) code for all kinds of subtle coding errors (e.g., using uninitialized variables (especially pointers)). It cured lots of problems before they bit us. The main limitation is that it is a static analysis, but since so many errors were detectable this way it was very useful. I tried to find lint for use these days, but it is difficult to find one in which I have confidence. It may be as much trouble getting a version of it working as to ignore it and try using a debugger such as gdb that you may already be using. There have been a number of fixes recently that have gone in. The wrapper process now running includes most of them and I'm glad to say memory issues now account for ~5% of fails. The development code version includes a couple more fixes. It's priority to get these sorted as it's holding up putting out the higher resolution configurations of OpenIFS (which I thought I'd be able to do last year!).
|
|
Send message Joined: 12 Apr 21 Posts: 317 Credit: 14,797,862 RAC: 19,598 |
My first ever double free or corruption Pretty cool run, it's like your PC is custom built for CPDN, sprinkled with a bit of luck and good resource usage. You're also the only person that I've seen who uses RHEL. I wonder if Glenn has seen any correlations between failure rates and Linux distros? |
|
Send message Joined: 12 Apr 21 Posts: 317 Credit: 14,797,862 RAC: 19,598 |
Model crashes. There will also be tasks that fail in the model because this experiment is designed to push the limits of what the model can do. If you have a failed task with a very long code traceback in the output, that'll be why. Credit is based on trickles, so credit will be given for work done. Is this one of them or is it something else? https://www.cpdn.org/result.php?resultid=22311226. I'm wondering as it was easy to tell with Hadley models when failures were due to this reason, not sure what it looks like with OIFS yet. |
|
Send message Joined: 5 Aug 04 Posts: 1120 Credit: 17,202,915 RAC: 2,154 |
Pretty cool run, it's like your PC is custom built for CPDN, sprinkled with a bit of luck and good resource usage. My machine was not custom-built for anything. It is a Dell Precision 5820 Tower that came with an Intel Xeon Processor W-2245 (8 C 3.9GHz 4.7GHz Turbo HT 16.5M B, 155W DDR4-2933), 32GB 2x16GB DDR4 2933MHz RDIM M ECC Memory that I doubled to 64 GBytes anticipating large RAM requirements of Oifs tasks coming up. It has a M.2 512GB PCIe NVMe Class 40 Solid State Drive mainly for system stuff, and two 4 TByte spinning hard drives. One of those hard drives has a partition just for Boinc stuff: 524 GB — 508 GB free (3.1% full). I started being glad it was so big when it started filling up with trickles during the period where the upload server was not working properly. I suppose I am one of the few using RHEL because it costs money (around $1/day). CentOS has an almost identical system that used to be free and probably still is. Even Fedora is similar. The main differences are that RHEL releases are supported for 10 years. I suspect CentOS is supported for less time, and Fedora is more interested at being the latest and greatest, so it changes a lot (something I do not want to be bothered with).
|
|
Dave Jackson Send message Joined: 15 May 09 Posts: 4535 Credit: 18,984,965 RAC: 21,892 |
My last four of latest batch now finished. I just have to wait another five hours for the last three to finish uploading before they can report. |
|
Send message Joined: 29 Oct 17 Posts: 1048 Credit: 16,419,240 RAC: 16,637 |
Yes, that's one of them. There's a line that saysModel crashes. There will also be tasks that fail in the model because this experiment is designed to push the limits of what the model can do. If you have a failed task with a very long code traceback in the output, that'll be why. Credit is based on trickles, so credit will be given for work done.Is this one of them or is it something else? https://www.cpdn.org/result.php?resultid=22311226. I'm wondering as it was easy to tell with Hadley models when failures were due to this reason, not sure what it looks like with OIFS yet. forrtl: error (72): floating overflow'forrtl' is 'Fortran runtime library'. Floating overflow means a real number became too big to be represented in 64 bits. Further down there's a reference to a subroutine called 'turbulence layer'. In essence the model blew up because the vertical turbulence became too strong. As this is likely with this configuration the model timestep was reduced from its normal 60 mins to 20mins, however, extreme perturbations can still blow the model. |
|
Send message Joined: 18 Nov 18 Posts: 21 Credit: 6,590,193 RAC: 1,871 |
Is there much of a chance we will ever see more of the apps being made to run on Windows pc's? As you know currently only the WAH2 apps run on Windows pc's. Thank you VERY much, working on it means it will work at some point and that's the whole idea, to get even more pc's in the mix to help some some of these calculations. |
|
Send message Joined: 5 Aug 04 Posts: 178 Credit: 18,741,194 RAC: 54,085 |
Yes, that's one of them. There's a line that saysThis WU will go the same way: https://www.cpdn.org/workunit.php?wuid=12207576 Supporting BOINC, a great concept ! |
|
Dave Jackson Send message Joined: 15 May 09 Posts: 4535 Credit: 18,984,965 RAC: 21,892 |
Two resends one reported, one waiting to report today from the same user. I suspect they both ran into memory problems. Could the push notifications be used to ask users to limit the number of tasks they run concurrently? this is the computer in question. |
©2024 cpdn.org
