WRFles

De Wikicima
(Diferencias entre revisiones)
Saltar a: navegación, buscar
(→‎Compilation)
 
(No se muestran 9 ediciones intermedias realizadas por un usuario)
Línea 1: Línea 1:
−
Here are derscribed the steps to perform a LES test simulation using WRF in CIMA’s HPC resource called
 +
Here are described the steps to perform a LES test simulation using WRF in CIMA’s HPC resource called <CODE>`hydra'</CODE>
−
`hydra'
 +
  +
Initial similar pdf version of the same information is available here [[Archivo:WRFles.pdf]]
   
 
= Compilation =
 
= Compilation =
−
Here are described the steps followed to compile LES test WRF code in hydra
 +
Here are described the steps followed to compile LES test WRF code in hydra
  +
  +
'''NOTE:''' There is no need to re-compile the model. It is only provided for completeness !
   
 
# First we need to specify a location for the compilation of the code:
  
# First we need to specify a location for the compilation of the code:
Línea 31: Línea 31:
 
</PRE>
  
</PRE>
   
−
+
= Use =
  +
Here are described the different steps to perform the WRF’s LES test simulation. This run test is done with standard original WRF’s LES ideal simulation (see details in table 1).
  +
# Simulation will be done in a different folder than the one where the compilation was made:
  +
<PRE>
  +
/home/lluis.fita/estudios/LES/test
  +
</PRE>
  +
# Link the compiled and necessary files
  +
<PRE>
  +
$ ln -s /share/WRF/WRFgit/WRFlesideal/ifort/dmpar/WRF/run/* ./
  +
</PRE>
  +
# Remove the namelist file in order to do not modify WRF's original namelist in case we want to modify something
  +
<PRE>
  +
$ rm namelist.input
  +
$ cp /share/WRF/WRFgit/WRFlesideal/ifort/dmpar/WRF/test/em_les/namelist.input ./
  +
</PRE>
  +
# Get the PBS jobs and scripts to run the simulation
  +
<PRE>
  +
$ cp /share/WRF/run_ideal.pbs ./
  +
$ cp /share/WRF/launch_pbs.bash ./
  +
</PRE>
  +
# Generate the initial conditions specific for the case (in this case using the default <CODE>input_sounding</CODE> (see <CODE>/share/WRF/WRFgit/WRFlesideal/ifort/dmpar/WRF/test/em_les/README.les</CODE> for more details and options)
  +
<PRE>
  +
$ qsub run_ideal.pbs
  +
</PRE>
  +
# Verify job is running propertly (it is very short, it last 18 seconds!)
  +
<PRE>
  +
$ qstat
  +
Job id Name User Time Use S Queue
  +
––––––––––––- –––––––– –––––––- –––– - ––-
  +
(...)
  +
8498.hydra WRF4L_ideal lluis.fita 00:00:12 R larga
  +
</PRE>
  +
# At the end one should have inside <CODE>realout/</CODE> folder:
  +
<PRE>
  +
$ ls -lrt realout/
  +
total 3744
  +
-rw-r–r– 1 lluis.fita cima 85706 Apr 12 13:27 namelist.output_00010101000000-00010101010000
  +
-rw-r–r– 1 lluis.fita cima 5020 Apr 12 13:27 namelist.input_00010101000000-00010101010000
  +
drwxr-xr-x 2 lluis.fita cima 4096 Apr 12 13:27 00010101000000-00010101010000
  +
-rw-r–r– 1 lluis.fita cima 3733932 Apr 12 13:27 wrfinput_d01
  +
</PRE>
  +
# And the <CODE>rsl</CODE> file should content:
  +
<PRE>
  +
$ cat realout/00010101000000-00010101010000/rsl.error.0000
  +
Ntasks in X 4, ntasks in Y 4
  +
IDEAL V4.0.3 PREPROCESSOR
  +
************************************
  +
Parent domain
  +
ids,ide,jds,jde 1 40 1 40
  +
ims,ime,jms,jme -4 17 -4 17
  +
ips,ipe,jps,jpe 1 10 1 10
  +
************************************
  +
DYNAMICS OPTION: Eulerian Mass Coordinate
  +
alloc_space_field: domain 1, 11151368 bytes allocated
  +
wrf: SUCCESS COMPLETE IDEAL INIT
  +
</PRE>
 
{| class="wikitable"
  
{| class="wikitable"
−
|+ Table 1: WRF’s LES test simulation specificications (from WRF namelist.input file from test/em_les folder specification
 +
|+ Table 1: WRF’s LES test simulation specifications (from WRF namelist.input file from test/em_les folder specification ([[http://www2.mmm.ucar.edu/wrf/users/docs/user_guide_v4/v4.0/users_guide_chap5.html#_Installing_WRF after WRF user’s web page instructions]])
  +
|-
 
! value
  
! value
 
! specification
  
! specification
Línea 71: Línea 71:
 
|}
  
|}
   
−
= Use =
 +
# Once initial conditions are generated (<CODE>realout/wrfinput_d01</CODE>), one can run the simulation. First the WRF
−
Here are describerd the different steps to perform the WRF’s LES test simulation. This run test is done with standard original WRF’s LES ideal simulation (see details in table 1).
  
−
# Simulation will be done in a different folder than the one where the compilation was made:
  
−
/home/lluis.fita/estudios/LES/test
  
−
# Link the compiled and necessary files
  
−
$ ln -s /share/WRF/WRFgit/WRFlesideal/ifort/dmpar/WRF/run/* ./
  
−
# Remove the namelist file in order to do not modify WRF’s original namelist in case we want to modify something
  
−
$ rm namelist.input
  
−
$ cp /share/WRF/WRFgit/WRFlesideal/ifort/dmpar/WRF/test/em_les/namelist.input ./
  
−
# Get the PBS jobs and scripts to run the simulation
  
−
$ cp /share/WRF/run_ideal.pbs ./
  
−
$ cp /share/WRF/launch_pbs.bash ./
  
−
# Generate the initial conditions specific for the case (in this case using the defailt input_sounding 2 )
  
−
$ qsub run_ideal.pbs
  
−
# Verify job is running propertly (it is very short, it last 18 seconds!)
  
−
$ qstat
  
−
Job id
  
−
Name
  
−
User
  
−
––––––––––––- –––––––– –––––––- –––– - ––-
  
−
(...)
  
−
8498.hydra
  
−
WRF4L_ideal
  
−
lluis.fita
  
−
Time Use S Queue
  
−
00:00:12 R larga
  
−
7. At the end one should have
  
−
$ ls -lrt realout/
  
−
total 3744
  
−
-rw-r–r– 1 lluis.fita cima
  
−
85706 Apr 12 13:27
  
−
namelist.output_00010101000000-00010101010000
  
−
-rw-r–r– 1 lluis.fita cima
  
−
5020 Apr 12 13:27
  
−
namelist.input_00010101000000-00010101010000
  
−
drwxr-xr-x 2 lluis.fita cima
  
−
4096 Apr 12 13:27 00010101000000-00010101010000
  
−
-rw-r–r– 1 lluis.fita cima 3733932 Apr 12 13:27 wrfinput_d01
  
−
1 after WRF user’s web page instructions http://www2.mmm.ucar.edu/wrf/users/docs/user_guide_v4/v4.0/users_guide_chap5.html#
  
−
_Installing_WRF
  
−
2 see /share/WRF/WRFgit/WRFlesideal/ifort/dmpar/WRF/test/em_les/README.les for more details and optionsWRF-LES test at CIMA
  
−
3
  
−
# And the rsl file shoult content:
  
−
$ cat realout/00010101000000-00010101010000/rsl.error.0000
  
−
Ntasks in X
  
−
4, ntasks in Y
  
−
4
  
−
IDEAL V4.0.3 PREPROCESSOR
  
−
************************************
  
−
Parent domain
  
−
ids,ide,jds,jde
  
−
1
  
−
40
  
−
1
  
−
40
  
−
ims,ime,jms,jme
  
−
-4
  
−
17
  
−
-4
  
−
17
  
−
ips,ipe,jps,jpe
  
−
1
  
−
10
  
−
1
  
−
10
  
−
************************************
  
−
DYNAMICS OPTION: Eulerian Mass Coordinate
  
−
alloc_space_field: domain
  
−
1,
  
−
11151368 bytes allocated
  
−
wrf: SUCCESS COMPLETE IDEAL INIT
  
−
# Once initial conditions are generated (realout/wrfinput_d01), one can run the simulation. First the WRF
  
 
PBS job script is required
  
PBS job script is required
  +
<PRE>
 
$ cp /share/WRF/run_wrf.pbs ./
  
$ cp /share/WRF/run_wrf.pbs ./
  +
</PRE>
 
# Simulation can be performed (by default 24 nodes)
  
# Simulation can be performed (by default 24 nodes)
  +
<PRE>
 
$ qsub ru_wrf.pbs
  
$ qsub ru_wrf.pbs
 
8499.hydra
  
8499.hydra
  +
</PRE>
 
# Verify simulation is propertly running
  
# Verify simulation is propertly running
  +
<PRE>
 
$ qstat
  
$ qstat
−
Job id
 +
Job id Name UserT ime Use S Queue
−
Name
  
−
User
  
 
––––––––––––- –––––––– –––––––- –––– - ––-
  
––––––––––––- –––––––– –––––––- –––– - ––-
 
(...)
  
(...)
−
8498.hydra
 +
8498.hydra WRF4L_ideal lluis.fita 00:00:18 C larga
−
WRF4L_ideal
 +
8499.hydra WRF4L_wrf lluis.fita 0 R larga
−
lluis.fita
 +
</PRE>
−
8499.hydra
 +
# At the end of job
−
WRF4L_wrf
 +
<PRE>
−
lluis.fita
  
−
Time Use S Queue
  
−
00:00:18 C larga
  
−
0 R larga
  
−
12. At the end of job
  
 
$ cat wrfout/00010101000000-00010101010000/rsl.error.0000
  
$ cat wrfout/00010101000000-00010101010000/rsl.error.0000
−
taskid: 0 hostname: node44
 +
taskid: 0 hostname: node44
−
module_io_quilt_old.F
 +
module_io_quilt_old.F 2931 F
−
2931 F
 +
Quilting with 1 groups of 0 I/O tasks.
−
Quilting with
 +
Ntasks in X 4, ntasks in Y 6
−
1 groups of
 +
For domain 1, the domain size is too small for this many processors,
−
0 I/O tasks.
  
−
Ntasks in X
  
−
4, ntasks in Y
  
−
6
  
−
For domain
  
−
1, the domain size is too small for this many processors,
  
 
or the decomposition aspect ratio is poor.
  
or the decomposition aspect ratio is poor.
−
Minimum decomposed computational patch size, either x-dir or y-dir, is 10 grid
 +
Minimum decomposed computational patch size, either x-dir or y-dir, is 10 grid
 
cells.
  
cells.
−
e_we =
 +
e_we =40, nproc_x =4, with cell width in x-direction =10
−
40, nproc_x =
 +
e_sn =40, nproc_y =6, with cell width in y-direction =6
−
4, with cell width in x-direction =
 +
–- ERROR: Reduce the MPI rank count, or redistribute the tasks.
−
10
  
−
e_sn =
  
−
40, nproc_y =
  
−
6, with cell width in y-direction =
  
−
6
  
−
–- ERROR: Reduce the MPI rank count, or redistribute the tasks.
  
 
––––––– FATAL CALLED –––––––-
  
––––––– FATAL CALLED –––––––-
−
FATAL CALLED FROM FILE: <stdin> LINE:
 +
FATAL CALLED FROM FILE: <stdin> LINE:1745
−
1745
 +
NOTE:1 namelist settings are wrong. Please check and reset these options
−
NOTE:
  
−
1 namelist settings are wrong. Please check and reset these options
  
 
–––––––––––––––––––––-
  
–––––––––––––––––––––-
−
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0WRF-LES test at CIMA
 +
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0
−
4
 +
</PRE>
−
# An error is found related to an exceess of cpu resources used. Thus there is a need to reduce it. Current
 +
# An error is found related to an excess of cpu resources used. Thus there is a need to reduce it. Current configuration
−
configuration
 +
<PRE>
 
$ cat run_wrf.pbs
  
$ cat run_wrf.pbs
 
(...)
  
(...)
 
## Queue
  
## Queue
−
##PBS -q large
 +
#PBS -q large
 
### Max run time
  
### Max run time
 
#PBS -l walltime=168:00:00
  
#PBS -l walltime=168:00:00
Línea 119: Línea 124:
 
#PBS -l nodes=1:ppn=24
  
#PBS -l nodes=1:ppn=24
 
(...)
  
(...)
−
mpiexec -n 24 ./launch_pbs.bash ./${execn}.exe >&
 +
mpiexec -n 24 ./launch_pbs.bash ./${execn}.exe >& run_${execn}0.log
−
run_${execn}0.log
 +
</PRE>
 
# There is a need to reduce the amount of resources. Let’s try with 16 processes
  
# There is a need to reduce the amount of resources. Let’s try with 16 processes
  +
<PRE>
 
$ vim run_wrf.pbs
  
$ vim run_wrf.pbs
 
(...)
  
(...)
Línea 127: Línea 133:
 
#PBS -l nodes=1:ppn=16
  
#PBS -l nodes=1:ppn=16
 
(...)
  
(...)
−
mpiexec -n 16 ./launch_pbs.bash ./${execn}.exe >&
 +
mpiexec -n 16 ./launch_pbs.bash ./${execn}.exe >& run_${execn}0.log
−
run_${execn}0.log
 +
</PRE>
 
# re-launching job
  
# re-launching job
  +
<PRE>
 
$ qsub ru_wrf.pbs
  
$ qsub ru_wrf.pbs
 
8501.hydra
  
8501.hydra
  +
</PRE>
 
# Checking job running
  
# Checking job running
  +
<PRE>
 
$ qstat
  
$ qstat
−
Job id
 +
Job id Name User Time Use S Queue
−
Name
  
−
User
  
 
––––––––––––- –––––––– –––––––- –––– - ––-
  
––––––––––––- –––––––– –––––––- –––– - ––-
 
(...)
  
(...)
−
8500.hydra
 +
8500.hydra WRF4L_wrf lluis.fita 00:01:03 R larga
−
WRF4L_wrf
 +
</PRE>
−
lluis.fita
 +
# At the end of simulation (note that with 16 process 1 time-step (1 s) is simulated with 0.0738 seconds of real time. Meaning that there is a simulation speed of around 13.6 times faster than simulation time)
−
Time Use S Queue
 +
<PRE>
−
00:01:03 R larga
  
−
# At the end of simulation ( 3 )
  
−
3 note that with 16 process 1 time-step (1 s) is simulated with 0.0738 seconds of real time. Meaning that there is a simulation speed of
  
−
around 13.6 times faster than simulation timeWRF-LES test at CIMA
  
−
5
  
 
$ tail wrfout/00010101000000-00010101010000/rsl.error.0000
  
$ tail wrfout/00010101000000-00010101010000/rsl.error.0000
−
Timing for main: time 0001-01-01_00:59:54 on domain
 +
Timing for main: time 0001-01-01_00:59:54 on domain 1: 0.07378 elapsed seconds
−
1:
 +
Timing for main: time 0001-01-01_00:59:55 on domain 1: 0.07362 elapsed seconds
−
0.07378 elapsed
 +
Timing for main: time 0001-01-01_00:59:56 on domain 1: 0.07385 elapsed seconds
−
seconds
 +
Timing for main: time 0001-01-01_00:59:57 on domain 1: 0.07380 elapsed seconds
−
Timing for main: time 0001-01-01_00:59:55 on domain
 +
Timing for main: time 0001-01-01_00:59:58 on domain 1: 0.07392 elapsed seconds
−
1:
 +
Timing for main: time 0001-01-01_00:59:59 on domain 1: 0.07399 elapsed seconds
−
0.07362 elapsed
 +
Timing for main: time 0001-01-01_01:00:00 on domain 1: 0.07406 elapsed seconds
−
seconds
 +
Timing for Writing wrfout_d01_0001-01-01_01:00:00 for domain 1: 0.31056 elapsed seconds
−
Timing for main: time 0001-01-01_00:59:56 on domain
 +
Timing for Writing restart for domain 1: 2.09555 elapsed seconds
−
1:
  
−
0.07385 elapsed
  
−
seconds
  
−
Timing for main: time 0001-01-01_00:59:57 on domain
  
−
1:
  
−
0.07380 elapsed
  
−
seconds
  
−
Timing for main: time 0001-01-01_00:59:58 on domain
  
−
1:
  
−
0.07392 elapsed
  
−
seconds
  
−
Timing for main: time 0001-01-01_00:59:59 on domain
  
−
1:
  
−
0.07399 elapsed
  
−
seconds
  
−
Timing for main: time 0001-01-01_01:00:00 on domain
  
−
1:
  
−
0.07406 elapsed
  
−
seconds
  
−
Timing for Writing wrfout_d01_0001-01-01_01:00:00 for domain
  
−
1:
  
−
0.31056
  
−
elapsed seconds
  
−
Timing for Writing restart for domain
  
−
1:
  
−
2.09555 elapsed seconds
  
 
d01 0001-01-01_01:00:00 wrf: SUCCESS COMPLETE WRF
  
d01 0001-01-01_01:00:00 wrf: SUCCESS COMPLETE WRF
−
# One should have inside wrfout folder
 +
</PRE>
  +
# One should have inside <CODE>wrfout/</CODE> folder
  +
<PRE>
 
$ ls wrfout
  
$ ls wrfout
 
00010101000000-00010101010000
  
00010101000000-00010101010000
Línea 161: Línea 170:
 
wrfout_d01_0001-01-01_00:00:00
  
wrfout_d01_0001-01-01_00:00:00
 
wrfrst_d01_0001-01-01_01:00:00
  
wrfrst_d01_0001-01-01_01:00:00
  +
</PRE>
 
# Standard output of 2 time-steps is about 6.7M
  
# Standard output of 2 time-steps is about 6.7M
  +
<PRE>
 
$ ls -lh wrfout/wrfout_d01_0001-01-01_00:00:00
  
$ ls -lh wrfout/wrfout_d01_0001-01-01_00:00:00
 
-rw-r–r– 1 lluis lluis 6.7M Apr 12 13:38 wrfout/wrfout_d01_0001-01-01_00:00:00
  
-rw-r–r– 1 lluis lluis 6.7M Apr 12 13:38 wrfout/wrfout_d01_0001-01-01_00:00:00
−
# See figures from 1 to 4 with some preiliminary results
 +
</PRE>
  +
# See figures from 1 to 4 with some preliminarily results
  +
  +
[[Archivo:WRFles_qv2.gif|frame|50px|Figure 3: Temporal evolution of of the 2-m mixing ratio (kgkg<SUP>-1</SUP>) every 15 minutes of simulation]]
  +
  +
[[Archivo:WRFles_qv.gif|frame|50px|Figure 4: Temporal evolution of vertical cross section at the center of the domain of mixing ratio (kgkg<SUP>-1</SUP>) every 15 minutes of simulation]]
  +
  +
[[Archivo:WRFles_wss.gif|frame|50px|Figure 3: Temporal evolution of of the 10-m wind-speed (ms<SUP>-1</SUP>) every 15 minutes of simulation]]
  +
  +
[[Archivo:WRFles_ws.gif|frame|50px|Figure 4: Temporal evolution of vertical cross section at the center of the domain of the wind-speed (ms<SUP>-1</SUP>) every 15 minutes of simulation]]

Última revisión de 15:09 15 abr 2019

Here are described the steps to perform a LES test simulation using WRF in CIMA’s HPC resource called `hydra'

Initial similar pdf version of the same information is available here Archivo:WRFles.pdf

[editar] Compilation

Here are described the steps followed to compile LES test WRF code in hydra

NOTE: There is no need to re-compile the model. It is only provided for completeness !

  1. First we need to specify a location for the compilation of the code: 
/share/WRF/WRFgit/WRFlesideal/ifort/dmpar
  1. From that location in hydra, download the source code of the WRF model (since V4.0 there is not provided anymore a compressed file with the code) 
$ git clone https://github.com/wrf-model/WRF
  1. In order to compile, first it is required to define some environmental variables into the system 
$ export NETCDF=’/usr/local’
$ export WRFIO_NCD_LARGE_FILE_SUPPORT=1
  1. Then prepare the file with the specific configuration for the compilation (in this case using intel compiler in `dmpar' mode [distributed memory only] and no nesting 1 ) 
configure.wrf (13. (serial) 14. (smpar) 15. (dmpar) 16. (dm+sm) INTEL (ifort/icc)) [no nesting]
  1. Compile te LES test case 
$ ./compile em_les >& compile.log
  1. Verify that there was not any error during compilation (should be empty) 
$ cat -n compile.log | grep Error

[editar] Use

Here are described the different steps to perform the WRF’s LES test simulation. This run test is done with standard original WRF’s LES ideal simulation (see details in table 1).

  1. Simulation will be done in a different folder than the one where the compilation was made: 
/home/lluis.fita/estudios/LES/test
  1. Link the compiled and necessary files 
$ ln -s /share/WRF/WRFgit/WRFlesideal/ifort/dmpar/WRF/run/* ./
  1. Remove the namelist file in order to do not modify WRF's original namelist in case we want to modify something 
$ rm namelist.input
$ cp /share/WRF/WRFgit/WRFlesideal/ifort/dmpar/WRF/test/em_les/namelist.input ./
  1. Get the PBS jobs and scripts to run the simulation 
$ cp /share/WRF/run_ideal.pbs ./
$ cp /share/WRF/launch_pbs.bash ./
  1. Generate the initial conditions specific for the case (in this case using the default input_sounding (see /share/WRF/WRFgit/WRFlesideal/ifort/dmpar/WRF/test/em_les/README.les for more details and options) 
$ qsub run_ideal.pbs
  1. Verify job is running propertly (it is very short, it last 18 seconds!) 
$ qstat
Job id Name User Time Use S Queue
––––––––––––- –––––––– –––––––- –––– - ––-
(...)
8498.hydra WRF4L_ideal lluis.fita 00:00:12 R larga
  1. At the end one should have inside realout/ folder: 
$ ls -lrt realout/
total 3744
-rw-r–r– 1 lluis.fita cima 85706 Apr 12 13:27 namelist.output_00010101000000-00010101010000
-rw-r–r– 1 lluis.fita cima  5020 Apr 12 13:27 namelist.input_00010101000000-00010101010000
drwxr-xr-x 2 lluis.fita cima 4096 Apr 12 13:27 00010101000000-00010101010000
-rw-r–r– 1 lluis.fita cima 3733932 Apr 12 13:27 wrfinput_d01
  1. And the rsl file should content: 
$ cat realout/00010101000000-00010101010000/rsl.error.0000
Ntasks in X 4, ntasks in Y 4
IDEAL V4.0.3 PREPROCESSOR
************************************
Parent domain
ids,ide,jds,jde 1 40 1 40
ims,ime,jms,jme -4 17 -4 17
ips,ipe,jps,jpe 1 10 1 10
************************************
DYNAMICS OPTION: Eulerian Mass Coordinate
   alloc_space_field: domain 1, 11151368 bytes allocated
  wrf: SUCCESS COMPLETE IDEAL INIT
Table 1: WRF’s LES test simulation specifications (from WRF namelist.input file from test/em_les folder specification ([after WRF user’s web page instructions])
value specification value specification value specification
dimx 40 dimy 40 dimz 40
dx (m) 100. dy (m) 100. time_step (s) 1
ztop (m) 2000 periodic_bx true periodic_by true
c_s 0.18 c_k 0.10 tke_heat_flux 0.24
  1. Once initial conditions are generated (realout/wrfinput_d01), one can run the simulation. First the WRF

PBS job script is required 

$ cp /share/WRF/run_wrf.pbs ./
  1. Simulation can be performed (by default 24 nodes) 
$ qsub ru_wrf.pbs
8499.hydra
  1. Verify simulation is propertly running 
$ qstat
Job id Name UserT ime Use S Queue
––––––––––––- –––––––– –––––––- –––– - ––-
(...)
8498.hydra WRF4L_ideal lluis.fita 00:00:18 C larga
8499.hydra WRF4L_wrf lluis.fita          0 R larga
  1. At the end of job 
$ cat wrfout/00010101000000-00010101010000/rsl.error.0000
taskid: 0 hostname: node44 
module_io_quilt_old.F 2931 F
Quilting with 1 groups of 0 I/O tasks.
Ntasks in X 4, ntasks in Y 6
  For domain 1, the domain size is too small for this many processors,
or the decomposition aspect ratio is poor.
  Minimum decomposed computational patch size, either x-dir or y-dir, is 10 grid
cells.
  e_we =40, nproc_x =4, with cell width in x-direction =10
  e_sn =40, nproc_y =6, with cell width in y-direction =6
  –- ERROR: Reduce the MPI rank count, or redistribute the tasks.
––––––– FATAL CALLED –––––––-
FATAL CALLED FROM FILE: <stdin> LINE:1745
NOTE:1 namelist settings are wrong. Please check and reset these options
–––––––––––––––––––––-
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0
  1. An error is found related to an excess of cpu resources used. Thus there is a need to reduce it. Current configuration 
$ cat run_wrf.pbs
(...)
## Queue
#PBS -q large
### Max run time
#PBS -l walltime=168:00:00
### Max memory
#PBS -l vmem=20gb
#PBS -l mem=20gb
## resources
#PBS -l nodes=1:ppn=24
(...)
mpiexec -n 24 ./launch_pbs.bash ./${execn}.exe >& run_${execn}0.log
  1. There is a need to reduce the amount of resources. Let’s try with 16 processes 
$ vim run_wrf.pbs
(...)
## resources
#PBS -l nodes=1:ppn=16
(...)
mpiexec -n 16 ./launch_pbs.bash ./${execn}.exe >& run_${execn}0.log
  1. re-launching job 
$ qsub ru_wrf.pbs
8501.hydra
  1. Checking job running 
$ qstat
Job id Name User Time Use S Queue
––––––––––––- –––––––– –––––––- –––– - ––-
(...)
8500.hydra WRF4L_wrf lluis.fita 00:01:03 R larga
  1. At the end of simulation (note that with 16 process 1 time-step (1 s) is simulated with 0.0738 seconds of real time. Meaning that there is a simulation speed of around 13.6 times faster than simulation time) 
$ tail wrfout/00010101000000-00010101010000/rsl.error.0000
Timing for main: time 0001-01-01_00:59:54 on domain 1: 0.07378 elapsed seconds
Timing for main: time 0001-01-01_00:59:55 on domain 1: 0.07362 elapsed seconds
Timing for main: time 0001-01-01_00:59:56 on domain 1: 0.07385 elapsed seconds
Timing for main: time 0001-01-01_00:59:57 on domain 1: 0.07380 elapsed seconds
Timing for main: time 0001-01-01_00:59:58 on domain 1: 0.07392 elapsed seconds
Timing for main: time 0001-01-01_00:59:59 on domain 1: 0.07399 elapsed seconds
Timing for main: time 0001-01-01_01:00:00 on domain 1: 0.07406 elapsed seconds
Timing for Writing wrfout_d01_0001-01-01_01:00:00 for domain 1: 0.31056 elapsed seconds
Timing for Writing restart for domain 1: 2.09555 elapsed seconds 
d01 0001-01-01_01:00:00 wrf: SUCCESS COMPLETE WRF
  1. One should have inside wrfout/ folder 
$ ls wrfout
00010101000000-00010101010000
namelist.input_00010101000000-00010101010000
namelist.output_00010101000000-00010101010000
run_wrf-00010101000000-00010101010000.log
stations
wrfout_d01_0001-01-01_00:00:00
wrfrst_d01_0001-01-01_01:00:00
  1. Standard output of 2 time-steps is about 6.7M 
$ ls -lh wrfout/wrfout_d01_0001-01-01_00:00:00
-rw-r–r– 1 lluis lluis 6.7M Apr 12 13:38 wrfout/wrfout_d01_0001-01-01_00:00:00
  1. See figures from 1 to 4 with some preliminarily results
(thumbnail)
Figure 3: Temporal evolution of of the 2-m mixing ratio (kgkg-1) every 15 minutes of simulation
(thumbnail)
Figure 4: Temporal evolution of vertical cross section at the center of the domain of mixing ratio (kgkg-1) every 15 minutes of simulation
(thumbnail)
Figure 3: Temporal evolution of of the 10-m wind-speed (ms-1) every 15 minutes of simulation
(thumbnail)
Figure 4: Temporal evolution of vertical cross section at the center of the domain of the wind-speed (ms-1) every 15 minutes of simulation
Obtenido de «https://wiki.cima.fcen.uba.ar/mediawiki/index.php?title=WRFles&oldid=1334»
Herramientas personales