Difference between revisions of "CodeBreak 24/11/2021"

(Created page with " = <span style="font-size:13.999999999999998pt; font-family:Arial; color:#434343; background-color:transparent; font-weight:400; font-style:normal; font-variant:normal;...")
 
Line 6: Line 6:
 
= <span style="font-size:13.999999999999998pt;  font-family:Arial;  color:#434343;  background-color:transparent;  font-weight:400;  font-style:normal;  font-variant:normal;  text-decoration:none;  vertical-align:baseline;  white-space:pre;  white-space:pre-wrap">Antecedent precipitation index</span> =
 
= <span style="font-size:13.999999999999998pt;  font-family:Arial;  color:#434343;  background-color:transparent;  font-weight:400;  font-style:normal;  font-variant:normal;  text-decoration:none;  vertical-align:baseline;  white-space:pre;  white-space:pre-wrap">Antecedent precipitation index</span> =
  
<span id="docs-internal-guid-fc14831d-7fff-4114-765f-ff5bb7b6b81c" style="font-size:11pt; font-family:Arial; color:#000000; background-color:transparent; font-weight:400; font-style:normal; font-variant:normal; text-decoration:none; vertical-align:baseline; white-space:pre; white-space:pre-wrap">The question was to calculate antecedent precipitation index efficiently for the entire AGCD dataset. There are several ways to proceed, one can split the calculation on different time periods and write the output for each time period to different netcdf files. It is also possible to use dask with <code>dask.array.map_blocks</code> but it can be more tricky to get it right. The following blog post show how to proceed for those 2 approaches</span><span style="font-size:11pt;  font-family:Arial;  color:#1155cc;  background-color:transparent;  font-weight:400;  font-style:normal;  font-variant:normal;  text-decoration:underline;  -webkit-text-decoration-skip:none;  text-decoration-skip-ink:none;  vertical-align:baseline;  white-space:pre;  white-space:pre-wrap"></span>[https://climate-cms.org/posts/2021-11-24-api.html <span style="font-size:11pt;  font-family:Arial;  color:#1155cc;  background-color:transparent;  font-weight:400;  font-style:normal;  font-variant:normal;  text-decoration:underline;  -webkit-text-decoration-skip:none;  text-decoration-skip-ink:none;  vertical-align:baseline;  white-space:pre;  white-space:pre-wrap">https://climate-cms.org/posts/2021-11-24-api.html</span>]<span style="font-size:11pt;  font-family:Arial;  color:#000000;  background-color:transparent;  font-weight:400;  font-style:normal;  font-variant:normal;  text-decoration:none;  vertical-align:baseline;  white-space:pre;  white-space:pre-wrap">&nbsp;</span>
+
<span id="docs-internal-guid-fc14831d-7fff-4114-765f-ff5bb7b6b81c" style="font-size:11pt; font-family:Arial; color:#000000; background-color:transparent; font-weight:400; font-style:normal; font-variant:normal; text-decoration:none; vertical-align:baseline; white-space:pre; white-space:pre-wrap">The question was to calculate antecedent precipitation index efficiently for the entire AGCD dataset. There are several ways to proceed, one can split the calculation on different time periods and write the output for each time period to different netcdf files. It is also possible to use dask with <tt>dask.array.map_blocks</tt> but it can be more tricky to get it right. The following blog post show how to proceed for those 2 approaches</span>[https://climate-cms.org/posts/2021-11-24-api.html <span style="font-size:11pt;  font-family:Arial;  color:#1155cc;  background-color:transparent;  font-weight:400;  font-style:normal;  font-variant:normal;  text-decoration:underline;  -webkit-text-decoration-skip:none;  text-decoration-skip-ink:none;  vertical-align:baseline;  white-space:pre;  white-space:pre-wrap">https://climate-cms.org/posts/2021-11-24-api.html</span>]<span style="font-size:11pt;  font-family:Arial;  color:#000000;  background-color:transparent;  font-weight:400;  font-style:normal;  font-variant:normal;  text-decoration:none;  vertical-align:baseline;  white-space:pre;  white-space:pre-wrap">&nbsp;</span>

Revision as of 23:52, 29 November 2021

Eddy kinetic energy calculation

Calculation would not complete due to dask error. It was suggested to precalculate the mean of the velocity fields to reduce the size and complexity of the dask task graph. This did not solve the immediate issue, but checking the cosima-recipeon which it was based suggested this calculation requires more resources (200GB of RAM, 48 cpus) than the largest OOD instance (46GB, 16 cpus).

Antecedent precipitation index

The question was to calculate antecedent precipitation index efficiently for the entire AGCD dataset. There are several ways to proceed, one can split the calculation on different time periods and write the output for each time period to different netcdf files. It is also possible to use dask with dask.array.map_blocks but it can be more tricky to get it right. The following blog post show how to proceed for those 2 approacheshttps://climate-cms.org/posts/2021-11-24-api.html