WRF

Model description

Weather Research and Forecast model (WRF, Skamarock, 2008) is a limited area atmospheric model developed by a consortium of american institutions with contributions from the whole community. It is a non-hydrostatic primitive equations model used in a large variety of research areas.

Techincal aspects of the model (v 4.0) are provided in this pdf

WRF and CIMA

CIMA has an HPC called hydra to perform high intensive modelling efforts with WRF.

hydra

The HPC is part of the Servicio Nacional de Cómputo de Alta Capacidad (SNCAD).

It has been re-installed and re-sorted in Jan 2022 and it has been adapted to common configurations in other international research centers.

Selection of compilers & libraries

It does not have the module software to manage compilers and libraries, but it does have an script which allows users to select the compiler. This is done at the terminal with the script source /opt/load-libs.sh

Example selecting intel 2021.4.0, mpich and netcdf 4

$ source /opt/load-libs.sh

Available libraries:
1) INTEL 2021.4.0: MPICH 3.4.2, NetCDF 4, HDF5 1.10.5, JASPER 2.0.33
2) INTEL 2021.4.0: OpenMPI 4.1.2, NetCDF 4, HDF5 1.10.5, JASPER 2.0.33
3) GNU 10.2.1: MPICH 3.4.2, NetCDF 4, HDF5 1.10.5, JASPER 2.0.33
4) GNU 10.2.1: OpenMPI 4.1.2, NetCDF 4, HDF5 1.10.5, JASPER 2.0.33
0) Exit
Choose an option: 1

The following libraries, compiled with Intel 2021.4.0 compilers, were loaded:
* MPICH 3.4.2
* NetCDF 4
* HDF5 1.10.5
* JASPER 2.0.33

To change it please logout and login again.

To load this libraries from within a script add this line to script:
source /opt/load-libs.sh 1

Compilation

Multiple different versions are available and pre-compiled in hydra, with the folloing structure:

/opt/wrf/WRF-{version}/{compiler}/{compiler-version}/{architecture}/{WRF/WPS}

Forcings

Atmospheric forcings

This will change once papa-deimos is fully operational

Provide the atmospheric conditions to the model at a given date.

• There is a shared space called /share
• At hydra all forcings are at:

/share/DATA/

• ERA-Interim
  • Thus, part of ERA-Interim forcings are:

/share/DATA/re-analysis/ERA-Interim/

• • Global monthly files at 0.75° horizontal resolution and all time-steps: 00, 06, 12, 18 are labelled as:
    • ERAI_pl[YYYY][MM]_[var1]-[var2].grib: pressure levels variables (all levels). GRIB codes as:
      • 129: geopotential
      • 157: relative humidty
      • 130: Temperature
      • 131: u-wind
      • 132: v-wind
    • ERAI_sfc[YYYY][MM].grib: all surface levels variables (step 0)
  • To download data:
    • Generate files from ECMWF web-page ERA-Interim
    • Go to the folder in hydra

cd /share/DATA/re-analysis/ERA-Interim/

• • • get the file (as link from ECMWF web-page (right bottom on `Download grib'), as e.g.:

$ wget https://stream.ecmwf.int/data/atls05/data/data02/scratch/_mars-atls05-a82bacafb5c306db76464bc7e824bb75-zn7P44.grib

• • • Rename file according to its content

$ mv _mars-atls05-a82bacafb5c306db76464bc7e824bb75-zn7P44.grib ERAI_sfc201302.grib

• ERA5
  • Thus, part of ERA5 forcings are:

/share/DATA/re-analysis/ERA5/

• 'NCEP-NNRP1'
  • Thus, full NCEP-NNRP1 forcings are (NOTE: no land data!!!):

/share/DATA/re-analysis/NCEP_NNRP/

Morphological forcings

Provide the geomorphological information for the domain of simulation: topography, land-use, vegetation-types, etc...

• In WRF there is a huge amount of data ans sources at different resolutions. At hydra everything is already there at:

/share/GEOG/

• They are ready to be use

GENERIC Model use

WRF has two main parts:

• WPS: Generation of the domain, initial and boundary condition: Runs 4 programs:
  • geogrid: domain generation
  • ungrib: unpack atmospheric forcing \verb+grib+ files
  • metgrid: horizontal interpolation of the unpacked atmospheric forcing files at the domain of simulation
  • real: generation of the initial and boundary conditions using \verb+metgrid+ output
• WRF: model it self: wrf.exe

At hydra all the code is already compiled at /opt/wrf (with folders WPS and WRFV3) As example WRFv4.3.3 compiled with intel-compilers with distributed and shared memory

/opt/wrf/WRF-4.3.3/intel/2021.4.0/dm+sm/WRF

IMPORTANT NOTES
* No simulation has to be run in the $HOME directory, use salidas/ instead
* Always run all the components of the model using the queue

Recommendations:

• Do not run the WPS/WRF components at the same folder where the code resides. Always run it on another place and link the exectuables and the necessary files
• Split the execution of the model in 2 different places:
  • Where the model components run
  • Where the outputs of the model components are to be stored and kept
  • For example

WRFrunning:	salidas/estudios/[ExperimentName]/[Simulation]
WRFstorage:	salidas/EXPS/[ExperimentName]/[Simulation]
Inside WRFrunning, WRFstorage	geogrid/, ungrib/, metgrid/, run/, ...

• Dimension the amount of nodes/cores/walltime accordingly to the needs of your simulations
• Make use of restarts, try to avoid to run simulations for more than a week

WPS

Let's assume that we work in a folder called $WORKDIR (at users' ${HOME} at hydra) and we will work with a given WRF version located in $WRFversion. As example, let's create a two nested domain of 30 km for the entire South America and a second one of 4.285 km for Córdoba mountain ranges (see the WRF/namelist.wps)