CMAQ

De Wikicima
(Diferencias entre revisiones)
Saltar a: navegación, buscar
(→‎intel)
(→‎Hermes)
 
(No se muestran 204 ediciones intermedias realizadas por 2 usuarios)
Línea 1: Línea 1:
 
Instalando y usando CMAQ en hydra
  
Instalando y usando CMAQ en hydra
   
−
= Instalando librerias =
 +
Para inventarios y condiciones de borde e iniciales [[CMAQInvCbCi]]
  +
  +
= Instalando librerías =
   
 
Instalación en <code>/home/solange.luque/MODELOS/CMAQ/[comp]</code>.
  
Instalación en <code>/home/solange.luque/MODELOS/CMAQ/[comp]</code>.
Línea 12: Línea 12:
   
 
Instalándolas en <code>/home/solange.luque/libraries/</code>
  
Instalándolas en <code>/home/solange.luque/libraries/</code>
  +
  +
Instalación y uso de ioapi desde python con [https://github.com/barronh/pseudonetcdf PseudoNetCDF] [[/pyIOAPI]]
   
 
=== intel ===
  
=== intel ===
Línea 100: Línea 102:
 
#export LD_LIBRARY_PATH=/opt/jasper/jasper-version-2.0.33/intel/2021.4.0/lib:
  
#export LD_LIBRARY_PATH=/opt/jasper/jasper-version-2.0.33/intel/2021.4.0/lib:
 
/opt/netcdf/netcdf-4/intel/2021.4.0/lib:$LD_LIBRARY_PATH
  
/opt/netcdf/netcdf-4/intel/2021.4.0/lib:$LD_LIBRARY_PATH
  +
/opt/load-libs.sh 3
  +
source /opt/env_scripts/load_gnu-10.2.1_mpich-3.4.2.sh
 
</pre>
  
</pre>
   
Línea 143: Línea 147:
 
> # IOAPIDEFS =
 
> # IOAPIDEFS =
 
> # PVMINCL =
  
> # PVMINCL =
  +
</pre>
  +
  +
Instalamos librerías
  +
<pre style="shell">
  +
make configure >& run_configure.log
  +
make all >& run_make.log
  +
</pre>
  +
  +
Si no hay problemas
  +
<pre style="shell">
  +
$ ls -l Linux2_x86_64/ | grep rwx | grep -v lrwxrwxrwx | awk '{print $9}' | tr '\n' ' '
  +
airs2m3 bcwndw camxtom3 datshift dayagg factor findwndw greg2jul gregdate gridprobe insertgrid jul2greg juldate juldiff julshift kfxtract
  +
latlon m3agmask m3agmax m3combo m3cple m3diff m3edhdr m3fake m3hdr m3interp m3mask m3merge m3pair m3probe m3stat m3totxt m3tproc m3tshift
  +
m3wndw m3xtract mpasdiff mpasstat mpastom3 mtxblend mtxbuild mtxcalc mtxcple presterp presz projtool randomstat selmrg2d timediff timeshift
  +
vertimeproc vertintegral vertot wndwdesc wrfgriddesc wrftom3
  +
</pre>
  +
  +
Instalando:
  +
<pre style="shell">
  +
mkdir ~/MODELOS/CMAQ/gnu/LIBRARIES/ioapi
  +
make install >& run_install.log
  +
ls ~/MODELOS/CMAQ/gnu/LIBRARIES/ioapi/Linux2_x86_64/
  +
airs2m3 greg2jul julshift m3cple m3merge m3utilio.mod modpdata.mod mtxcalc timediff wrftom3
  +
bcwndw gregdate kfxtract m3diff m3pair m3wndw modwrfio.mod mtxcple timeshift
  +
camxtom3 gridprobe latlon m3edhdr m3probe m3xtract mpasdiff presterp vertimeproc
  +
datshift insertgrid libioapi.a m3fake m3stat modatts3.mod mpasstat presz vertintegral
  +
dayagg jul2greg m3agmask m3hdr m3totxt modgctp.mod mpastom3 projtool vertot
  +
factor juldate m3agmax m3interp m3tproc modmpasfio.mod mtxblend randomstat wndwdesc
  +
findwndw juldiff m3combo m3mask m3tshift modncfio.mod mtxbuild selmrg2d wrfgriddesc
  +
cp ioapi/fixed_src/* Linux2_x86_64/
  +
cp ioapi/fixed_src/* /home/solange.luque/MODELOS/CMAQ/gnu/LIBRARIES/ioapi/Linux2_x86_64/
  +
</pre>
  +
  +
=== gnu - debug ===
  +
'''NOTA:''' Antes de empezar, quitar del <code>.bashrc</code> la carga por defecto del entorno intel
  +
<pre style=shell>
  +
##Para compiladores INTEL
  +
#source /opt/intel/oneapi/setvars.sh > /dev/null
  +
#export PATH=/opt/mpich/mpich-3.4.2/intel/2021.4.0/bin:$PATH
  +
#export LD_LIBRARY_PATH=/opt/jasper/jasper-version-2.0.33/intel/2021.4.0/lib:
  +
/opt/netcdf/netcdf-4/intel/2021.4.0/lib:$LD_LIBRARY_PATH
  +
</pre>
  +
  +
<pre style=shell>
  +
mkdir -p ioapi/v3.2-20200828/gnu-dbg
  +
cd ioapi/v3.2-20200828/gnu-dbg
  +
tar xvfz /home/solange.luque/libraries/ioapi-3.2-20200828.tar.gz
  +
mv ioapi-3.2-20200828/* ./
  +
rmdir ioapi-3.2-20200828
  +
export BIN=Linux2_x86_64dbg
  +
mkdir Linux2_x86_64dbg
  +
ln -s /opt/netcdf/netcdf-4/gnu/10.2.1/lib/*.so Linux2_x86_64dbg/
  +
ln -s /opt/netcdf/netcdf-4/gnu/10.2.1/lib/libnetcdf.a Linux2_x86_64dbg/
  +
ln -s /opt/netcdf/netcdf-4/gnu/10.2.1/lib/libnetcdff.a Linux2_x86_64dbg/
  +
ln -s /opt/hdf5/hdf5-1.10.5/gnu/10.2.1/lib/libhdf5*a Linux2_x86_64dbg/
  +
ln -s /opt/hdf5/hdf5-1.10.5/gnu/10.2.1/lib/libhdf5*so Linux2_x86_64dbg/
  +
ln -s /usr/lib/x86_64-linux-gnu/*curl*a Linux2_x86_64dbg/
  +
ln -s /usr/lib/x86_64-linux-gnu/*curl*so Linux2_x86_64dbg/
  +
cp ioapi/Makefile.nocpl ioapi/Makefile
  +
cp m3tools/Makefile.nocpl m3tools/Makefile
  +
cp Makefile.template Makefile
  +
cp ioapi/Makeinclude.Linux2_x86_64dbg ioapi/Makeinclude.Linux2_x86_64dbg.old
  +
</pre>
  +
  +
Editamos los Makefiles (<code>Makefile</code> y <code>ioapi/Makeinclude.Linux2_x86_64</code>)
  +
<pre style="shell">
  +
diff Makefile Makefile.template
  +
138,145c138,145
  +
< BIN = Linux2_x86_64dbg
  +
< BASEDIR = ${PWD}
  +
< INSTALL = /home/solange.luque/libs/ioapi/v3.2-20200828/gnu-dbg
  +
< LIBINST = $(INSTALL)/$(BIN)
  +
< BININST = $(INSTALL)/$(BIN)
  +
< CPLMODE = nocpl
  +
< IOAPIDEFS =
  +
< PVMINCL =
  +
---
  +
> # BIN = Linux2_x86_64
  +
> # BASEDIR = ${PWD}
  +
> # INSTALL = ${HOME}
  +
> # LIBINST = $(INSTALL)/$(BIN)
  +
> # BININST = $(INSTALL)/$(BIN)
  +
> # CPLMODE = nocpl
  +
> # IOAPIDEFS =
  +
> # PVMINCL =
  +
  +
diff ioapi/Makeinclude.Linux2_x86_64dbg ioapi/Makeinclude.Linux2_x86_64dbg.old
  +
26,28c26
  +
< DBGFLAGS = -g -Wall -Wextra -Warray-temporaries -Wconversion -fimplicit-none -fbacktrace -ffree-line-length-none -fcheck=all -ffpe-trap=zero,overflow,underflow -finit-real=nan
  +
< DBGFLAGS =
  +
< MFLAGS = -Wsurprising -fbounds-check -finit-real=snan -m64 ${DBGFLAGS} -Wno-tabs
  +
---
  +
> MFLAGS = -Wsurprising -fbounds-check -finit-real=snan -m64
  +
 
</pre>
  
</pre>
   
Línea 218: Línea 316:
 
Configuration and Run Scripts have been Extracted and placed in: /home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project
  
Configuration and Run Scripts have been Extracted and placed in: /home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project
 
You may now edit these scripts to conform to your system and run options.
  
You may now edit these scripts to conform to your system and run options.
  +
</pre>
  +
  +
Vamos al directorio
  +
<pre style="shell">
  +
cd CMAQ_Project/
  +
cp config_cmaq.csh config_cmaq.csh.old
  +
diff config_cmaq.csh config_cmaq.csh.old
  +
88,95c88,95
  +
< setenv IOAPI_INCL_DIR /home/solange.luque/libraries/ioapi/v3.2-20200828/intel/Linux2_x86_64ifort #> I/O API include header files
  +
< setenv IOAPI_LIB_DIR /home/solange.luque/libraries/ioapi/v3.2-20200828/intel/Linux2_x86_64ifort #> I/O API libraries
  +
< setenv NETCDF_LIB_DIR /opt/netcdf/netcdf-4/intel/2021.4.0/lib #> netCDF C directory path
  +
< setenv NETCDF_INCL_DIR /opt/netcdf/netcdf-4/intel/2021.4.0/include #> netCDF C directory path
  +
< setenv NETCDFF_LIB_DIR /opt/netcdf/netcdf-4/intel/2021.4.0/lib #> netCDF Fortran directory path
  +
< setenv NETCDFF_INCL_DIR /opt/netcdf/netcdf-4/intel/2021.4.0/include #> netCDF Fortran directory path
  +
< setenv MPI_INCL_DIR /opt/intel/oneapi/mpi/2021.4.0/include #> MPI Include directory path
  +
< setenv MPI_LIB_DIR /opt/intel/oneapi/mpi/2021.4.0/lib #> MPI Lib directory path
  +
---
  +
> setenv IOAPI_INCL_DIR ioapi_inc_intel #> I/O API include header files
  +
> setenv IOAPI_LIB_DIR ioapi_lib_intel #> I/O API libraries
  +
> setenv NETCDF_LIB_DIR netcdf_lib_intel #> netCDF C directory path
  +
> setenv NETCDF_INCL_DIR netcdf_inc_intel #> netCDF C directory path
  +
> setenv NETCDFF_LIB_DIR netcdff_lib_intel #> netCDF Fortran directory path
  +
> setenv NETCDFF_INCL_DIR netcdff_inc_intel #> netCDF Fortran directory path
  +
> setenv MPI_INCL_DIR mpi_incl_intel #> MPI Include directory path
  +
> setenv MPI_LIB_DIR mpi_lib_intel #> MPI Lib directory path
  +
99c99
  +
< setenv myFC /opt/intel/oneapi/mpi/2021.4.0/bin/mpiifort
  +
---
  +
> setenv myFC mpiifort
  +
103c103
  +
< setenv myLINK_FLAG "-qopenmp"
  +
---
  +
> setenv myLINK_FLAG #"-qopenmp-simd" openMP not supported w/ CMAQ
  +
./config_cmaq.csh intel >& run_config-cmaq.log
  +
cat run_config-cmaq.log
  +
Compiler is set to intel
  +
</pre>
  +
  +
Compilamos!
  +
<pre style="shell">
  +
cd CCTM/scripts/
  +
mkdir BLD_CCTM_v54_intel
  +
cp -R /opt/intel/oneapi/mpi/2021.4.0/include/* BLD_CCTM_v54_intel/
  +
./bldit_cctm.csh intel >& run_bldit_cctm.log
  +
ls BLD_CCTM_v54_intel/*exe
  +
BLD_CCTM_v54_intel/CCTM_v54.exe
  +
</pre>
  +
  +
'''NOTA:''' Si se está intentando una recompilación, antes vaciar el directorio
  +
  +
<pre style="shell">
  +
rm -rf BLD_CCTM_v54_intel/*
 
</pre>
  
</pre>
   
Línea 326: Línea 476:
   
 
= Usando =
  
= Usando =
  +
Ejemplos de uso de CMAQ (útiles ya compilados e instalados en hydra)
  +
  +
* Editor emisiones: [https://www.cmascenter.org/smoke SMOKE]
  +
* Editor condiciones atmosféricas iniciales / entorno: [https://www.epa.gov/cmaq/meteorology-chemistry-interface-processor MCIP]
  +
* Editor condiciones iniciales químicas: [https://github.com/USEPA/CMAQ/blob/main/PREP/icon/README.md icon]
  +
* Editor condiciones borde químicas: [https://github.com/USEPA/CMAQ/blob/main/PREP/bcon/README.md bcon]
  +
  +
=== Compilación MCIP ===
  +
  +
Preparado de las condiciones con MCIP usando la script: <code>/home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/PREP/mcip/scripts/run_mcip.csh</code>
  +
  +
<pre style="shell">
  +
export CMAQ_HOME=/home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/
  +
cd /home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/PREP/mcip/scripts
  +
cp run_mcip.csh run_mcip.csh.old
  +
diff run_mcip.csh run_mcip.csh.old
  +
125,129c125
  +
< # config_cmaq.csh modifies $CMAQ_HOME
  +
< set val = $CMAQ_HOME
  +
< source $CMAQ_HOME/config_cmaq.csh intel
  +
< set CMAQ_HOME = $val
  +
< echo $0": Solange 2 CMAQ_HOME: "$CMAQ_HOME
  +
---
  +
> source $CMAQ_HOME/config_cmaq.csh
  +
133c129
  +
< set GridName = AMBA # 16-character maximum
  +
---
  +
> set GridName = 2016_12SE1 # 16-character maximum
  +
135,136c131,132
  +
< set DataPath = /home/solange.luque/estudios/CMAQ-ideal/DATA
  +
< set InMetDir = $DataPath/wrf
  +
---
  +
> set DataPath = $CMAQ_DATA
  +
> set InMetDir = $DataPath/wrf
  +
142,143d137
  +
<
  +
< echo $0": Solange: "$ProgDir
  +
163,164c157,159
  +
< set InMetFiles = ( $InMetDir/wrfout_d03_2012-11-15_00:00:00 \
  +
< $InMetDir/wrfout_d03_2012-11-15_12:00:00 )
  +
---
  +
> set InMetFiles = ( $InMetDir/subset_wrfout_d01_2016-07-01_00:00:00 \
  +
> $InMetDir/subset_wrfout_d01_2016-07-02_00:00:00 \
  +
> $InMetDir/subset_wrfout_d01_2016-07-03_00:00:00 )
  +
166,167c161,162
  +
< set IfGeo = "T"
  +
< set InGeoFile = $InGeoDir/geo_em.d03.nc
  +
---
  +
> set IfGeo = "F"
  +
> set InGeoFile = $InGeoDir/geo_em_d01.nc
  +
194,195c189,190
  +
< set MCIP_START = 2012-11-15-00:10:00.0000 # [UTC]
  +
< set MCIP_END = 2012-11-15-50:00:00.0000 # [UTC]
  +
---
  +
> set MCIP_START = 2016-07-02-00:00:00.0000 # [UTC]
  +
> set MCIP_END = 2016-07-03-00:00:00.0000 # [UTC]
  +
197c192
  +
< set INTVL = 10 # [min]
  +
---
  +
> set INTVL = 60 # [min]
  +
221c216
  +
< set BTRIM = 5
  +
---
  +
> set BTRIM = 0
  +
</pre>
  +
  +
Compilando mcip
  +
<pre style="shell">
  +
cd /home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/PREP/mcip/src
  +
vim /opt/load-libs.sh
  +
source /opt/env_scripts/load_intel-2021.4.0_mpich-3.4.2.sh
  +
diff Makefile Makefile.old
  +
50,52c50,52
  +
< FC = /opt/intel/oneapi/compiler/2021.4.0/linux/bin/intel64/ifort
  +
< NETCDF = /opt/netcdf/netcdf-4/intel/2021.4.0
  +
< IOAPI_ROOT = /home/solange.luque/MODELOS/CMAQ/intel/LIBRARIES/ioapi
  +
---
  +
> FC = ifort
  +
> NETCDF = /usr/local/apps/netcdf-4.7.3/intel-19.0
  +
> IOAPI_ROOT = /usr/local/apps/ioapi-3.2_20181011/intel-19.0
  +
56,59c56,57
  +
< #LIBS = -L$(IOAPI_ROOT)/lib -lioapi \
  +
< # -L$(NETCDF)/lib -lnetcdff -lnetcdf
  +
< LIBS = -L$(IOAPI_ROOT)/Linux2_x86_64ifort -lioapi \
  +
< -L$(NETCDF)/lib -lnetcdff -lnetcdf -qopenmp
  +
---
  +
> LIBS = -L$(IOAPI_ROOT)/lib -lioapi \
  +
> -L$(NETCDF)/lib -lnetcdff -lnetcdf
  +
make all >& run_make.log
  +
ls *exe
  +
mcip.exe
  +
</pre>
  +
  +
Se copia la script en donde se generaron los datos para asegurar su reproducibilidad
  +
<pre>
  +
cp run_mcip.csh /home/solange.luque/estudios/CMAQ-ideal/DATA
  +
</pre>
  +
  +
=== Compilación icon ===
  +
  +
Empezando en
  +
<pre style="shell">
  +
/home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/PREP/icon/scripts
  +
./bldit_icon.csh intel >& run_bldit_icon.log
  +
ls BLD_ICON_v54_intel/
  +
avprofile_cb6r3m_ae7_kmtbr_hemi2016_v53beta2_m3dry_col051_row068.csv
  +
avprofile_racm_ae6_aq_derived_from_cb6r3m_ae7_kmtbr_hemi2016_v53beta2_m3dry_col051_row068.csv
  +
avprofile_saprc07tc_ae6_aq_derived_from_cb6r3m_ae7_kmtbr_hemi2016_v53beta2_m3dry_col051_row068.csv
  +
avprofile_saprc07tic_ae7i_aq_derived_from_cb6r3m_ae7_kmtbr_hemi2016_v53beta2_m3dry_col051_row068.csv
  +
findex.F
  +
HGRD_DEFN.F
  +
icon.F
  +
ICON_v54.cfg
  +
IC_PARMS.F
  +
lat_lon.F
  +
legacy_PM_toxics_profile.csv
  +
lr_interp.F
  +
m3_ck_ctmmet.F
  +
m3_ck_ctms.F
  +
m3_ck_icmet.F
  +
m3_driver.F
  +
m3_icout.F
  +
m3_vinterp.F
  +
Makefile
  +
Makefile.intel
  +
opn_ic_file.F
  +
prof_driver.F
  +
prof_icout.F
  +
prof_vinterp.F
  +
VGRD_DEFN.F
  +
</pre>
  +
  +
Compilando:
  +
<pre style="shell">
  +
cd BLD_ICON_v54_intel
  +
/opt/load-libs.sh 1
  +
source /opt/env_scripts/load_intel-2021.4.0_mpich-3.4.2.sh
  +
make >& run_make.log
  +
ls *exe
  +
ICON_v54.exe
  +
</pre>
  +
  +
=== Compilación bcon ===
  +
  +
Empezando desde <code>/home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/PREP/bcon/scripts</code>
  +
  +
<pre style="shell">
  +
./bldit_bcon.csh intel >& run_bldit_bcon.log
  +
cd BLD_BCON_v54_intel/
  +
ls BLD_BCON_v54_intel/*exe
  +
BLD_BCON_v54_intel/BCON_v54.exe
  +
</pre>
  +
  +
== Benchmark ==
  +
Descargando los datos del test-case de [https://drive.google.com/drive/folders/1AFUB-4kzIXXoZr4hOHNBqRvy9JQ9_MDp?usp=sharing acá]
  +
  +
Datos en hydra <code>$CMAQ_DATA=/home/solange.luque/MODELOS/CMAQ/DATA/benchmark</code>
  +
<pre style="shell">
  +
tar xvfz CMAQv5.4_2018_12NE3_Benchmark_2Day_Input.tar.gz
  +
du -hsc ./CMAQv5.4_2018_12NE3_Benchmark_2Day_Input
  +
56G ./CMAQv5.4_2018_12NE3_Benchmark_2Day_Input
  +
</pre>
  +
  +
Home CMAQ en hydra <code>$CMAQ_HOME=/home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project</code>
  +
  +
Enlazando los datos
  +
<pre style="shell">
  +
cd /home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project
  +
mkdir -p data
  +
cd data
  +
ln -s /home/solange.luque/MODELOS/CMAQ/DATA/benchmark/CMAQv5.4_2018_12NE3_Benchmark_2Day_Input/2018_12NE3 ./
  +
</pre>
  +
  +
Corriendo benchmark
  +
  +
<pre style="shell">
  +
source /opt/env_scripts/load_intel-2021.4.0_mpich-3.4.2.sh
  +
cd $CMAQ_HOME/CCTM/scripts
  +
cp run_cctm_Bench_2018_12NE3.csh run_cctm_Bench_2018_12NE3.csh.old
  +
diff run_cctm_Bench_2018_12NE3.csh run_cctm_Bench_2018_12NE3.csh.old
  +
716,717c716
  +
< #( time -p mpirun -np $NPROCS $BLD/$EXEC ) |& tee buff_${EXECUTION_ID}.txt
  +
< (mpirun -np $NPROCS $BLD/$EXEC ) |& tee buff_${EXECUTION_ID}.txt
  +
---
  +
> ( /usr/bin/time -p mpirun -np $NPROCS $BLD/$EXEC ) |& tee buff_${EXECUTION_ID}.txt
  +
721c720
  +
< #rm -rf buff_${EXECUTION_ID}.txt
  +
---
  +
> rm -rf buff_${EXECUTION_ID}.txt
  +
732d730
  +
< echo "Have a look into buff_${EXECUTION_ID}.txt"
  +
735d732
  +
< rm -rf buff_${EXECUTION_ID}.txt
  +
qsub launch_cctm_intel.pb
  +
#./run_cctm_Bench_2018_12NE3.csh >& run_Bench_2018-12NE3.log
  +
tail -n 30 run_Bench_2018-12NE3.log
  +
  +
==============================================
  +
|>--- PROGRAM COMPLETED SUCCESSFULLY ---<|
  +
==============================================
  +
Date and time 0:00:00 July 2, 2018 (2018183:000000)
  +
  +
The elapsed time for this simulation was 2206.0 seconds.
  +
  +
  +
CMAQ Processing of Day 20180701 Finished at Wed 16 Nov 2022 04:11:26 PM -03
  +
  +
\\\\\=====\\\\\=====\\\\\=====\\\\\=====/////=====/////=====/////=====/////
  +
  +
  +
==================================
  +
***** CMAQ TIMING REPORT *****
  +
==================================
  +
Start Day: 2018-07-01
  +
End Day: 2018-07-01
  +
Number of Simulation Days: 1
  +
Domain Name: 2018_12NE3
  +
Number of Grid Cells: 367500 (ROW x COL x LAY)
  +
Number of Layers: 35
  +
Number of Processes: 32
  +
All times are in seconds.
  +
  +
Num Day Wall Time
  +
01 2018-07-01 2206.0
  +
Total Time = 2206.00
  +
Avg. Time = 2206.00
  +
</pre>
  +
  +
== caso ideal ==
  +
Partimos de una simulación de WRF y creamos los forzantes atmosféricos
  +
  +
Directorio de trabajo: <code>/home/solange.luque/estudios/CMAQ-ideal</code>
  +
  +
===Meteorología===
  +
  +
Para generar las condiciones meteorológicas ideales vamos a usar de base condiciones reales de una corridade WRF (wrfout_d03: dominio sobre AMBA a 1 km resolución, wrfour_d01: dominio centro SA a 50 km):
  +
  +
<pre style="shell">
  +
ls /home/solange.luque/estudios/CMAQ-ideal/DATA/wrf
  +
geo_em.d03.nc wrfout_d01_2012-08-03_00:00:00 wrfout_d03_2012-11-15_00:00:00
  +
wrfout_d03_2012-11-15_12:00:00
  +
</pre>
  +
  +
  +
A partir de estos ficheros creamos los ficheros meteorológicos ideales con velocidad constante usando el script <code>getpoint_everywhere</code>. Por ejemplo, para una simulación con solo velocidad de viento en la dirección horizontal y velocidad de 3m/s
  +
  +
<pre>
  +
python3 getpoint_everywhere.py -i 60 -j 60 -x west_east -y south_north -f 'wrfout_d03_2012-11-15_00:00:00' -v 'U@alloverct@3.:V@alloverct@0.' -s 'Time@0@-9@6'
  +
python3 getpoint_everywhere.py -i 60 -j 60 -x west_east -y south_north -f 'wrfout_d03_2012-11-15_12:00:00' -v 'U@alloverct@3.:V@alloverct@0.' -s 'Time@0@-9@6'
  +
</pre>
  +
  +
Esto genera dos ficheros que renombramos:
  +
  +
<pre>
  +
wrfout_d03_2012-11-15_00:00:00_3
  +
wrfout_d03_2012-11-15_12:00:00_3
  +
</pre>
  +
  +
Ahora vamos a correr mcip con estas condiciones. Partiendo del <code>run_mcip.csh</pre> que creamos para el AMBA (caso prueba):
  +
  +
<pre style="shell">
  +
export CMAQ_HOME=/home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/
  +
cd /home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/PREP/mcip/scripts/
  +
cp run_mcip.csh run_mcip_ideal.csh
  +
  +
diff run_mcip.csh run_mcip_ideal.csh
  +
131c131
  +
< set APPL = AMBA
  +
---
  +
> set APPL = AMBAidealArea
  +
135c135
  +
< set DataPath = /home/solange.luque/estudios/CMAQ-AMBA/prueba/DATA
  +
---
  +
> set DataPath = /home/solange.luque/estudios/CMAQ-ideal/DATA
  +
163,165c163,164
  +
< set InMetFiles = ( $InMetDir/wrfout_d03_2012-11-14_12:00:00 \
  +
< $InMetDir/wrfout_d03_2012-11-15_00:00:00 \
  +
< $InMetDir/wrfout_d03_2012-11-15_12:00:00 )
  +
---
  +
> set InMetFiles = ( $InMetDir/wrfout_d03_2012-11-15_00:00:00_3 \
  +
> $InMetDir/wrfout_d03_2012-11-15_12:00:00_3)
  +
  +
</pre>
  +
  +
Ahora sí, corremos mcip:
  +
  +
<pre>
  +
./run_mcip_ideal.csh >& run_mcip_ideal.log
  +
</pre>
  +
  +
  +
== caso prueba AMBA ==
  +
WORKDIR = /home/solange.luque/estudios/CMAQ-AMBA/prueba
  +
  +
Qué se necesita:
  +
# Tener una salida de WRF y perararla para CMAQ
  +
# Inventarios de emisiones y prepararlos para CMAQ
  +
# Condiciones de contorno e iniciales para la química atmosférica
  +
# Configurar CMAQ
  +
# Simular CMAQ
  +
  +
Todos los pasos se explican con más detalle [[CMAQ/pruebaAMBA]]

Última revisión de 13:30 18 oct 2023

Instalando y usando CMAQ en hydra

Para inventarios y condiciones de borde e iniciales CMAQInvCbCi

Contenido

[editar] Instalando librerías

Instalación en /home/solange.luque/MODELOS/CMAQ/[comp].

Existe un foro: https://forum.cmascenter.org/t/ioapi-installtion-unable/805

[editar] ioapi

Agarrar librerías ioapi

Instalándolas en /home/solange.luque/libraries/

Instalación y uso de ioapi desde python con PseudoNetCDF /pyIOAPI

[editar] intel 

mkdir -p ioapi/v3.2-20200828/intel
cd ioapi/v3.2-20200828/intel
tar xvfz ~/ioapi-3.2-20200828.tar.gz
mv ioapi-3.2-20200828/* ./
rmdir ioapi-3.2-20200828
export BIN=Linux2_x86_64ifort
mkdir Linux2_x86_64ifort
ln -s /opt/netcdf/netcdf-4/intel/2021.4.0/lib/*.so Linux2_x86_64ifort/
ln -s /opt/netcdf/netcdf-4/intel/2021.4.0/lib/libnetcdf.a Linux2_x86_64ifort/
ln -s /opt/netcdf/netcdf-4/intel/2021.4.0/lib/libnetcdff.a Linux2_x86_64ifort/
ln -s /opt/hdf5/hdf5-1.10.5/intel/2021.4.0/lib/libhdf5*a Linux2_x86_64ifort/
ln -s /opt/hdf5/hdf5-1.10.5/intel/2021.4.0/lib/libhdf5*so Linux2_x86_64ifort/
ln -s /opt/curl/curl-4.1.2/intel/2021.4.0/lib/libcurl.* ./
cp ioapi/Makefile.nocpl ioapi/Makefile
cp m3tools/Makefile.nocpl m3tools/Makefile
cp Makefile.template Makefile

Seguimos (https://forum.cmascenter.org/t/ioapi-error-in-m3too/2341)[CMAS forum `Ioapi error in m3too;`]

Editamos los Makefiles (Makefile y ioapi/Makeinclude.Linux2_x86_64ifort) 

 diff Makefile Makefile.template 
138,145c138,145
<  BIN        = Linux2_x86_64ifort
<  BASEDIR    = ${PWD}
<  INSTALL    = /home/solange.luque/MODELOS/CMAQ/intel/LIBRARIES/ioapi
<  LIBINST    = $(INSTALL)/$(BIN)
<  BININST    = $(INSTALL)/$(BIN)
<  CPLMODE    = nocpl
<  IOAPIDEFS  = 
<  PVMINCL    =
---
> # BIN        = Linux2_x86_64
> # BASEDIR    = ${PWD}
> # INSTALL    = ${HOME}
> # LIBINST    = $(INSTALL)/$(BIN)
> # BININST    = $(INSTALL)/$(BIN)
> # CPLMODE    = nocpl
> # IOAPIDEFS  = 
> # PVMINCL    =

Instalamos librerías 

/opt/load-libs.sh 1 
make configure >& run_configure.log
make all >& run_make.log

Si todo fue bien: 

ls -l Linux2_x86_64ifort/ | grep rwx | grep -v lrwxrwxrwx | awk '{print $9}' | tr '\n' ' '
airs2m3 bcwndw camxtom3 datshift dayagg factor findwndw greg2jul gregdate gridprobe insertgrid 
jul2greg juldate juldiff julshift kfxtract latlon m3agmask m3agmax m3combo m3cple m3diff m3edhdr 
m3fake m3hdr m3interp m3mask m3merge m3pair m3probe m3stat m3totxt m3tproc m3tshift m3wndw 
m3xtract mpasdiff mpasstat mpastom3 mtxblend mtxbuild mtxcalc mtxcple presterp presz projtool 
randomstat selmrg2d timediff timeshift vertimeproc vertintegral vertot wndwdesc wrfgriddesc wrftom3

Instalando: 

mkdir -p ~/MODELOS/CMAQ/intel/LIBRARIES/ioapi
make install >& run_install.log
ls ~/MODELOS/CMAQ/intel/LIBRARIES/ioapi/Linux2_x86_64ifort/
airs2m3   greg2jul    julshift    m3cple    m3merge   m3utilio.mod    modpdata.mod  mtxcalc     timediff      wrftom3
bcwndw    gregdate    kfxtract    m3diff    m3pair    m3wndw          modwrfio.mod  mtxcple     timeshift
camxtom3  gridprobe   latlon      m3edhdr   m3probe   m3xtract        mpasdiff      presterp    vertimeproc
datshift  insertgrid  libioapi.a  m3fake    m3stat    modatts3.mod    mpasstat      presz       vertintegral
dayagg    jul2greg    m3agmask    m3hdr     m3totxt   modgctp.mod     mpastom3      projtool    vertot
factor    juldate     m3agmax     m3interp  m3tproc   modmpasfio.mod  mtxblend      randomstat  wndwdesc
findwndw  juldiff     m3combo     m3mask    m3tshift  modncfio.mod    mtxbuild      selmrg2d    wrfgriddesc
cp ioapi/fixed_src/* Linux2_x86_64ifort/
cp ioapi/fixed_src/* ~/MODELOS/CMAQ/intel/LIBRARIES/ioapi/Linux2_x86_64ifort/

[editar] gnu

NOTA: Antes de empezar, quitar del .bashrc la carga por defecto del entorno intel 

##Para compiladores INTEL
#source /opt/intel/oneapi/setvars.sh > /dev/null
#export PATH=/opt/mpich/mpich-3.4.2/intel/2021.4.0/bin:$PATH
#export LD_LIBRARY_PATH=/opt/jasper/jasper-version-2.0.33/intel/2021.4.0/lib:
  /opt/netcdf/netcdf-4/intel/2021.4.0/lib:$LD_LIBRARY_PATH
/opt/load-libs.sh 3
source /opt/env_scripts/load_gnu-10.2.1_mpich-3.4.2.sh

mkdir -p ioapi/v3.2-20200828/gnu
cd ioapi/v3.2-20200828/gnu
tar xvfz ~/ioapi-3.2-20200828.tar.gz
mv ioapi-3.2-20200828/* ./
rmdir ioapi-3.2-20200828
export BIN=Linux2_x86_64
mkdir Linux2_x86_64
ln -s /opt/netcdf/netcdf-4/gnu/10.2.1/lib/*.so Linux2_x86_64/
ln -s /opt/netcdf/netcdf-4/gnu/10.2.1/lib/libnetcdf.a Linux2_x86_64/
ln -s /opt/netcdf/netcdf-4/gnu/10.2.1/lib/libnetcdff.a Linux2_x86_64/
ln -s /opt/hdf5/hdf5-1.10.5/gnu/10.2.1/lib/libhdf5*a Linux2_x86_64/
ln -s /opt/hdf5/hdf5-1.10.5/gnu/10.2.1/lib/libhdf5*so Linux2_x86_64/
ln -s /usr/lib/x86_64-linux-gnu/*curl*a Linux2_x86_64/
ln -s /usr/lib/x86_64-linux-gnu/*curl*so Linux2_x86_64/
cp ioapi/Makefile.nocpl ioapi/Makefile
cp m3tools/Makefile.nocpl m3tools/Makefile
cp Makefile.template Makefile

Editamos los Makefiles (Makefile y ioapi/Makeinclude.Linux2_x86_64) 

 diff Makefile Makefile.template 
138,145c138,145
<  BIN        = Linux2_x86_64
<  BASEDIR    = ${PWD}
<  INSTALL    = /home/solange.luque/MODELOS/CMAQ/gnu/LIBRARIES/ioapi
<  LIBINST    = $(INSTALL)/$(BIN)
<  BININST    = $(INSTALL)/$(BIN)
<  CPLMODE    = nocpl
<  IOAPIDEFS  = 
<  PVMINCL    =
---
> # BIN        = Linux2_x86_64
> # BASEDIR    = ${PWD}
> # INSTALL    = ${HOME}
> # LIBINST    = $(INSTALL)/$(BIN)
> # BININST    = $(INSTALL)/$(BIN)
> # CPLMODE    = nocpl
> # IOAPIDEFS  = 
> # PVMINCL    =

Instalamos librerías 

make configure >& run_configure.log
make all >& run_make.log

Si no hay problemas 

$ ls -l Linux2_x86_64/ | grep rwx | grep -v lrwxrwxrwx | awk '{print $9}' | tr '\n' ' ' 
airs2m3 bcwndw camxtom3 datshift dayagg factor findwndw greg2jul gregdate gridprobe insertgrid jul2greg juldate juldiff julshift kfxtract 
  latlon m3agmask m3agmax m3combo m3cple m3diff m3edhdr m3fake m3hdr m3interp m3mask m3merge m3pair m3probe m3stat m3totxt m3tproc m3tshift 
  m3wndw m3xtract mpasdiff mpasstat mpastom3 mtxblend mtxbuild mtxcalc mtxcple presterp presz projtool randomstat selmrg2d timediff timeshift 
  vertimeproc vertintegral vertot wndwdesc wrfgriddesc wrftom3

Instalando: 

mkdir ~/MODELOS/CMAQ/gnu/LIBRARIES/ioapi
make install >& run_install.log
ls ~/MODELOS/CMAQ/gnu/LIBRARIES/ioapi/Linux2_x86_64/
airs2m3   greg2jul    julshift    m3cple    m3merge   m3utilio.mod    modpdata.mod  mtxcalc     timediff      wrftom3
bcwndw    gregdate    kfxtract    m3diff    m3pair    m3wndw          modwrfio.mod  mtxcple     timeshift
camxtom3  gridprobe   latlon      m3edhdr   m3probe   m3xtract        mpasdiff      presterp    vertimeproc
datshift  insertgrid  libioapi.a  m3fake    m3stat    modatts3.mod    mpasstat      presz       vertintegral
dayagg    jul2greg    m3agmask    m3hdr     m3totxt   modgctp.mod     mpastom3      projtool    vertot
factor    juldate     m3agmax     m3interp  m3tproc   modmpasfio.mod  mtxblend      randomstat  wndwdesc
findwndw  juldiff     m3combo     m3mask    m3tshift  modncfio.mod    mtxbuild      selmrg2d    wrfgriddesc
cp ioapi/fixed_src/* Linux2_x86_64/
cp ioapi/fixed_src/* /home/solange.luque/MODELOS/CMAQ/gnu/LIBRARIES/ioapi/Linux2_x86_64/

[editar] gnu - debug

NOTA: Antes de empezar, quitar del .bashrc la carga por defecto del entorno intel 

##Para compiladores INTEL
#source /opt/intel/oneapi/setvars.sh > /dev/null
#export PATH=/opt/mpich/mpich-3.4.2/intel/2021.4.0/bin:$PATH
#export LD_LIBRARY_PATH=/opt/jasper/jasper-version-2.0.33/intel/2021.4.0/lib:
  /opt/netcdf/netcdf-4/intel/2021.4.0/lib:$LD_LIBRARY_PATH

mkdir -p ioapi/v3.2-20200828/gnu-dbg
cd ioapi/v3.2-20200828/gnu-dbg
tar xvfz /home/solange.luque/libraries/ioapi-3.2-20200828.tar.gz
mv ioapi-3.2-20200828/* ./
rmdir ioapi-3.2-20200828
export BIN=Linux2_x86_64dbg
mkdir Linux2_x86_64dbg
ln -s /opt/netcdf/netcdf-4/gnu/10.2.1/lib/*.so Linux2_x86_64dbg/
ln -s /opt/netcdf/netcdf-4/gnu/10.2.1/lib/libnetcdf.a Linux2_x86_64dbg/
ln -s /opt/netcdf/netcdf-4/gnu/10.2.1/lib/libnetcdff.a Linux2_x86_64dbg/
ln -s /opt/hdf5/hdf5-1.10.5/gnu/10.2.1/lib/libhdf5*a Linux2_x86_64dbg/
ln -s /opt/hdf5/hdf5-1.10.5/gnu/10.2.1/lib/libhdf5*so Linux2_x86_64dbg/
ln -s /usr/lib/x86_64-linux-gnu/*curl*a Linux2_x86_64dbg/
ln -s /usr/lib/x86_64-linux-gnu/*curl*so Linux2_x86_64dbg/
cp ioapi/Makefile.nocpl ioapi/Makefile
cp m3tools/Makefile.nocpl m3tools/Makefile
cp Makefile.template Makefile
cp ioapi/Makeinclude.Linux2_x86_64dbg ioapi/Makeinclude.Linux2_x86_64dbg.old

Editamos los Makefiles (Makefile y ioapi/Makeinclude.Linux2_x86_64) 

 diff Makefile Makefile.template 
138,145c138,145
<  BIN        = Linux2_x86_64dbg
<  BASEDIR    = ${PWD}
<  INSTALL    = /home/solange.luque/libs/ioapi/v3.2-20200828/gnu-dbg
<  LIBINST    = $(INSTALL)/$(BIN)
<  BININST    = $(INSTALL)/$(BIN)
<  CPLMODE    = nocpl
<  IOAPIDEFS  = 
<  PVMINCL    =
---
> # BIN        = Linux2_x86_64
> # BASEDIR    = ${PWD}
> # INSTALL    = ${HOME}
> # LIBINST    = $(INSTALL)/$(BIN)
> # BININST    = $(INSTALL)/$(BIN)
> # CPLMODE    = nocpl
> # IOAPIDEFS  = 
> # PVMINCL    =

diff ioapi/Makeinclude.Linux2_x86_64dbg ioapi/Makeinclude.Linux2_x86_64dbg.old 
26,28c26
< DBGFLAGS  = -g -Wall -Wextra -Warray-temporaries -Wconversion -fimplicit-none -fbacktrace -ffree-line-length-none -fcheck=all -ffpe-trap=zero,overflow,underflow -finit-real=nan
< DBGFLAGS  = 
< MFLAGS    = -Wsurprising -fbounds-check -finit-real=snan -m64 ${DBGFLAGS} -Wno-tabs
---
> MFLAGS    = -Wsurprising -fbounds-check -finit-real=snan -m64

Instalamos librerías 

make configure >& run_configure.log
make all >& run_make.log

Si no hay problemas 

$ ls -l Linux2_x86_64/ | grep rwx | grep -v lrwxrwxrwx | awk '{print $9}' | tr '\n' ' ' 
airs2m3 bcwndw camxtom3 datshift dayagg factor findwndw greg2jul gregdate gridprobe insertgrid jul2greg juldate juldiff julshift kfxtract 
  latlon m3agmask m3agmax m3combo m3cple m3diff m3edhdr m3fake m3hdr m3interp m3mask m3merge m3pair m3probe m3stat m3totxt m3tproc m3tshift 
  m3wndw m3xtract mpasdiff mpasstat mpastom3 mtxblend mtxbuild mtxcalc mtxcple presterp presz projtool randomstat selmrg2d timediff timeshift 
  vertimeproc vertintegral vertot wndwdesc wrfgriddesc wrftom3

Instalando: 

mkdir ~/MODELOS/CMAQ/gnu/LIBRARIES/ioapi
make install >& run_install.log
ls ~/MODELOS/CMAQ/gnu/LIBRARIES/ioapi/Linux2_x86_64/
airs2m3   greg2jul    julshift    m3cple    m3merge   m3utilio.mod    modpdata.mod  mtxcalc     timediff      wrftom3
bcwndw    gregdate    kfxtract    m3diff    m3pair    m3wndw          modwrfio.mod  mtxcple     timeshift
camxtom3  gridprobe   latlon      m3edhdr   m3probe   m3xtract        mpasdiff      presterp    vertimeproc
datshift  insertgrid  libioapi.a  m3fake    m3stat    modatts3.mod    mpasstat      presz       vertintegral
dayagg    jul2greg    m3agmask    m3hdr     m3totxt   modgctp.mod     mpastom3      projtool    vertot
factor    juldate     m3agmax     m3interp  m3tproc   modmpasfio.mod  mtxblend      randomstat  wndwdesc
findwndw  juldiff     m3combo     m3mask    m3tshift  modncfio.mod    mtxbuild      selmrg2d    wrfgriddesc
cp ioapi/fixed_src/* Linux2_x86_64/
cp ioapi/fixed_src/* /home/solange.luque/MODELOS/CMAQ/gnu/LIBRARIES/ioapi/Linux2_x86_64/

[editar] CMAQ

NOTA: Asegurarse que no hay ningún entorno de compiladores activado automáticamente en el .bashrc

[editar] intel

Yendo al directorio de trabajo 

cd /home/solange.luque/MODELOS/CMAQ/intel
mkdir src
cd src
git clone https://github.com/USEPA/CMAQ.git CMAQ-master
head -n 1 CMAQ-master/README.md
CMAQv5.4
cd CMAQ-master/
ls 
bldit_project.csh  CCTM  config_cmaq.csh  DOCS  license.md  POST  PREP  PYTOOLS  README.md  UTIL

Preparando configuración de la compilación 

cp bldit_project.csh bldit_project.csh.old
diff bldit_project.csh bldit_project.csh.old 
19,20c19
<  #set CMAQ_HOME = ${PWD}/CMAQ_Project
<  set CMAQ_HOME = /home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project
---
>  set CMAQ_HOME = /home/username/path

Compilamos! 

./bldit_project.csh >& run_bldit.log
tail run_bldit.log 
mkdir: created directory '/home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/POST/hr2day'
mkdir: created directory '/home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/POST/hr2day/scripts'
mkdir: created directory '/home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/POST/sitecmp'
mkdir: created directory '/home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/POST/sitecmp/scripts'
mkdir: created directory '/home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/POST/sitecmp_dailyo3'
mkdir: created directory '/home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/POST/sitecmp_dailyo3/scripts'
mkdir: created directory '/home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/POST/writesite'
mkdir: created directory '/home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/POST/writesite/scripts'
Configuration and Run Scripts have been Extracted and placed in: /home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project
You may now edit these scripts to conform to your system and run options.

Vamos al directorio 

cd CMAQ_Project/
cp config_cmaq.csh config_cmaq.csh.old
diff config_cmaq.csh config_cmaq.csh.old 
88,95c88,95
<         setenv IOAPI_INCL_DIR   /home/solange.luque/libraries/ioapi/v3.2-20200828/intel/Linux2_x86_64ifort           #> I/O API include header files
<         setenv IOAPI_LIB_DIR    /home/solange.luque/libraries/ioapi/v3.2-20200828/intel/Linux2_x86_64ifort             #> I/O API libraries
<         setenv NETCDF_LIB_DIR   /opt/netcdf/netcdf-4/intel/2021.4.0/lib            #> netCDF C directory path
<         setenv NETCDF_INCL_DIR  /opt/netcdf/netcdf-4/intel/2021.4.0/include        #> netCDF C directory path
<         setenv NETCDFF_LIB_DIR  /opt/netcdf/netcdf-4/intel/2021.4.0/lib            #> netCDF Fortran directory path
<         setenv NETCDFF_INCL_DIR /opt/netcdf/netcdf-4/intel/2021.4.0/include        #> netCDF Fortran directory path
<         setenv MPI_INCL_DIR     /opt/intel/oneapi/mpi/2021.4.0/include             #> MPI Include directory path
<         setenv MPI_LIB_DIR      /opt/intel/oneapi/mpi/2021.4.0/lib                 #> MPI Lib directory path
---
>         setenv IOAPI_INCL_DIR   ioapi_inc_intel             #> I/O API include header files
>         setenv IOAPI_LIB_DIR    ioapi_lib_intel             #> I/O API libraries
>         setenv NETCDF_LIB_DIR   netcdf_lib_intel            #> netCDF C directory path
>         setenv NETCDF_INCL_DIR  netcdf_inc_intel            #> netCDF C directory path
>         setenv NETCDFF_LIB_DIR  netcdff_lib_intel           #> netCDF Fortran directory path
>         setenv NETCDFF_INCL_DIR netcdff_inc_intel           #> netCDF Fortran directory path
>         setenv MPI_INCL_DIR     mpi_incl_intel              #> MPI Include directory path
>         setenv MPI_LIB_DIR      mpi_lib_intel               #> MPI Lib directory path
99c99
<         setenv myFC /opt/intel/oneapi/mpi/2021.4.0/bin/mpiifort
---
>         setenv myFC mpiifort
103c103
<         setenv myLINK_FLAG "-qopenmp"
---
>         setenv myLINK_FLAG #"-qopenmp-simd" openMP not supported w/ CMAQ
./config_cmaq.csh intel >& run_config-cmaq.log
cat run_config-cmaq.log
Compiler is set to intel

Compilamos! 

cd CCTM/scripts/
mkdir BLD_CCTM_v54_intel
cp -R /opt/intel/oneapi/mpi/2021.4.0/include/* BLD_CCTM_v54_intel/
./bldit_cctm.csh intel >& run_bldit_cctm.log
ls BLD_CCTM_v54_intel/*exe
BLD_CCTM_v54_intel/CCTM_v54.exe

NOTA: Si se está intentando una recompilación, antes vaciar el directorio 

rm -rf BLD_CCTM_v54_intel/*

[editar] gnu

Yendo al directorio de trabajo 

/home/solange.luque/MODELOS/CMAQ/gnu/
mkdir src
cd src
git clone https://github.com/USEPA/CMAQ.git CMAQ-master
head -n 1 ../../../README.md 
CMAQv5.4
cd CMAQ-master/
ls 
bldit_project.csh  CCTM  config_cmaq.csh  DOCS  license.md  POST  PREP  PYTOOLS  README.md  UTIL

Preparando configuración de la compilación 

cp bldit_project.csh bldit_project.csh.old

diff bldit_project.csh bldit_project.csh.old 
19,20c19
<  #set CMAQ_HOME = ${PWD}/CMAQ_Project
<  set CMAQ_HOME = /home/solange.luque/MODELOS/CMAQ/gnu/src/CMAQ-master/CMAQ_Project
---
>  set CMAQ_HOME = /home/username/path

Compilamos! 

./bldit_project.csh >& run_bldit.log
tail run_bldit.log 
mkdir: created directory '/home/solange.luque/MODELOS/CMAQ/gnu/src/CMAQ-master/CMAQ_Project/POST/hr2day'
mkdir: created directory '/home/solange.luque/MODELOS/CMAQ/gnu/src/CMAQ-master/CMAQ_Project/POST/hr2day/scripts'
mkdir: created directory '/home/solange.luque/MODELOS/CMAQ/gnu/src/CMAQ-master/CMAQ_Project/POST/sitecmp'
mkdir: created directory '/home/solange.luque/MODELOS/CMAQ/gnu/src/CMAQ-master/CMAQ_Project/POST/sitecmp/scripts'
mkdir: created directory '/home/solange.luque/MODELOS/CMAQ/gnu/src/CMAQ-master/CMAQ_Project/POST/sitecmp_dailyo3'
mkdir: created directory '/home/solange.luque/MODELOS/CMAQ/gnu/src/CMAQ-master/CMAQ_Project/POST/sitecmp_dailyo3/scripts'
mkdir: created directory '/home/solange.luque/MODELOS/CMAQ/gnu/src/CMAQ-master/CMAQ_Project/POST/writesite'
mkdir: created directory '/home/solange.luque/MODELOS/CMAQ/gnu/src/CMAQ-master/CMAQ_Project/POST/writesite/scripts'
Configuration and Run Scripts have been Extracted and placed in: /home/solange.luque/MODELOS/CMAQ/gnu/src/CMAQ-master/CMAQ_Project
You may now edit these scripts to conform to your system and run options.

Vamos al directorio 

cd CMAQ_Project/
cp config_cmaq.csh config_cmaq.csh.old
diff config_cmaq.csh config_cmaq.csh.old 
151,158c151,158
<         setenv IOAPI_INCL_DIR   /home/solange.luque/libraries/ioapi/v3.2-20200828/gnu/Linux2_x86_64              #> I/O API include header files
<         setenv IOAPI_LIB_DIR    /home/solange.luque/libraries/ioapi/v3.2-20200828/gnu/Linux2_x86_64              #> I/O API libraries
<         setenv NETCDF_LIB_DIR   /opt/netcdf/netcdf-4/gnu/10.2.1/lib             #> netCDF C directory path
<         setenv NETCDF_INCL_DIR  /opt/netcdf/netcdf-4/gnu/10.2.1/include              #> netCDF C directory path
<         setenv NETCDFF_LIB_DIR  /opt/netcdf/netcdf_c-4.8.1_fortran-4.5.3/gnu/10.2.1/lib           #> netCDF Fortran directory path
<         setenv NETCDFF_INCL_DIR /opt/netcdf/netcdf_c-4.8.1_fortran-4.5.3/gnu/10.2.1/include           #> netCDF Fortran directory path
<         setenv MPI_INCL_DIR     /opt/openmpi/openmpi-4.1.2/gnu/10.2.1/include               #> MPI Include directory path
<         setenv MPI_LIB_DIR      /opt/openmpi/openmpi-4.1.2/gnu/10.2.1/lib               #> MPI Lib directory path
---
>         setenv IOAPI_INCL_DIR   iopai_inc_gcc             #> I/O API include header files
>         setenv IOAPI_LIB_DIR    ioapi_lib_gcc             #> I/O API libraries
>         setenv NETCDF_LIB_DIR   netcdf_lib_gcc            #> netCDF C directory path
>         setenv NETCDF_INCL_DIR  netcdf_inc_gcc            #> netCDF C directory path
>         setenv NETCDFF_LIB_DIR  netcdff_lib_gcc           #> netCDF Fortran directory path
>         setenv NETCDFF_INCL_DIR netcdff_inc_gcc           #> netCDF Fortran directory path
>         setenv MPI_INCL_DIR     mpi_incl_gcc              #> MPI Include directory path
>         setenv MPI_LIB_DIR      mpi_lib_gcc               #> MPI Lib directory path
162c162
<         setenv myFC /opt/openmpi/openmpi-4.1.2/gnu/10.2.1/bin/mpifort
---
>         setenv myFC mpifort
166,167c166,167
<         setenv myFFLAGS "-ffixed-form -ffixed-line-length-132 -funroll-loops -finit-character=32 -fallow-argument-mismatch"
<         setenv myFRFLAGS "-ffree-form -ffree-line-length-none -funroll-loops -finit-character=32 -frecursive -fallow-argument-mismatch"
---
>         setenv myFFLAGS "-ffixed-form -ffixed-line-length-132 -funroll-loops -finit-character=32"
>         setenv myFRFLAGS "-ffree-form -ffree-line-length-none -funroll-loops -finit-character=32"
169c169
<         setenv myLINK_FLAG  "-fopenmp" # openMP not supported w/ CMAQ
---
>         setenv myLINK_FLAG # "-fopenmp" openMP not supported w/ CMAQ
./config_cmaq.csh gcc >& run_config-cmaq.log
cat run_config-cmaq.log
Compiler is set to gcc

NOTA: en caso que aparezca en el run_config-cmaq.log 0: Event not found., editar el config_cmaq.csh e imprimir por pantalla la variable $CMAQ_LIB. Ahí se creean unos directorios que si ha habido algún problema, no se llenan bien. Es recomendable borrarlos y volver a lanzar. 

rm -rf /home/solange.luque/MODELOS/CMAQ/gnu/src/CMAQ-master/lib/x86_64/gcc/ioapi

Compilar el Chemistry Transport Model (CCTM) 

cd CCTM/scripts/
mkdir BLD_CCTM_v54_gcc
cp -R /opt/openmpi/openmpi-4.1.2/gnu/10.2.1/include/* BLD_CCTM_v54_gcc/
./bldit_cctm.csh gcc >& run_bldit_cctm.log
ls BLD_CCTM_v54_gcc/*exe
BLD_CCTM_v54_gcc/CCTM_v54.exe

NOTA: Si se está intentando una recompilación, antes vaciar el directorio 

rm  BLD_CCTM_v54_gcc/*

[editar] Usando

Ejemplos de uso de CMAQ (útiles ya compilados e instalados en hydra) 

* Editor emisiones: SMOKE
* Editor condiciones atmosféricas iniciales / entorno: MCIP
* Editor condiciones iniciales químicas: icon
* Editor condiciones borde químicas: bcon

[editar] Compilación MCIP

Preparado de las condiciones con MCIP usando la script: /home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/PREP/mcip/scripts/run_mcip.csh 

export CMAQ_HOME=/home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/
cd /home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/PREP/mcip/scripts
cp run_mcip.csh run_mcip.csh.old
diff run_mcip.csh run_mcip.csh.old 
125,129c125
< # config_cmaq.csh modifies $CMAQ_HOME
< set val = $CMAQ_HOME
< source $CMAQ_HOME/config_cmaq.csh intel
< set CMAQ_HOME = $val
< echo $0": Solange 2 CMAQ_HOME: "$CMAQ_HOME
---
> source $CMAQ_HOME/config_cmaq.csh
133c129
< set GridName   = AMBA              # 16-character maximum
---
> set GridName   = 2016_12SE1        # 16-character maximum
135,136c131,132
< set DataPath   = /home/solange.luque/estudios/CMAQ-ideal/DATA
< set InMetDir   = $DataPath/wrf 
---
> set DataPath   = $CMAQ_DATA
> set InMetDir   = $DataPath/wrf
142,143d137
< 
< echo $0": Solange: "$ProgDir
163,164c157,159
< set InMetFiles = ( $InMetDir/wrfout_d03_2012-11-15_00:00:00 \
<                    $InMetDir/wrfout_d03_2012-11-15_12:00:00 )
---
> set InMetFiles = ( $InMetDir/subset_wrfout_d01_2016-07-01_00:00:00 \
>                    $InMetDir/subset_wrfout_d01_2016-07-02_00:00:00 \
>                    $InMetDir/subset_wrfout_d01_2016-07-03_00:00:00 )
166,167c161,162
< set IfGeo      = "T"
< set InGeoFile  = $InGeoDir/geo_em.d03.nc
---
> set IfGeo      = "F"
> set InGeoFile  = $InGeoDir/geo_em_d01.nc
194,195c189,190
< set MCIP_START = 2012-11-15-00:10:00.0000  # [UTC]
< set MCIP_END   = 2012-11-15-50:00:00.0000  # [UTC]
---
> set MCIP_START = 2016-07-02-00:00:00.0000  # [UTC]
> set MCIP_END   = 2016-07-03-00:00:00.0000  # [UTC]
197c192
< set INTVL      = 10 # [min]
---
> set INTVL      = 60 # [min]
221c216
< set BTRIM = 5
---
> set BTRIM = 0

Compilando mcip 

cd /home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/PREP/mcip/src
vim /opt/load-libs.sh
source /opt/env_scripts/load_intel-2021.4.0_mpich-3.4.2.sh
diff Makefile Makefile.old 
50,52c50,52
< FC      = /opt/intel/oneapi/compiler/2021.4.0/linux/bin/intel64/ifort
< NETCDF = /opt/netcdf/netcdf-4/intel/2021.4.0
< IOAPI_ROOT = /home/solange.luque/MODELOS/CMAQ/intel/LIBRARIES/ioapi
---
> FC      = ifort
> NETCDF = /usr/local/apps/netcdf-4.7.3/intel-19.0
> IOAPI_ROOT = /usr/local/apps/ioapi-3.2_20181011/intel-19.0
56,59c56,57
< #LIBS    = -L$(IOAPI_ROOT)/lib -lioapi \
< #          -L$(NETCDF)/lib -lnetcdff -lnetcdf
< LIBS    = -L$(IOAPI_ROOT)/Linux2_x86_64ifort -lioapi \
<           -L$(NETCDF)/lib -lnetcdff -lnetcdf -qopenmp
---
> LIBS    = -L$(IOAPI_ROOT)/lib -lioapi \
>           -L$(NETCDF)/lib -lnetcdff -lnetcdf
make all >& run_make.log
ls *exe
mcip.exe

Se copia la script en donde se generaron los datos para asegurar su reproducibilidad 

cp run_mcip.csh /home/solange.luque/estudios/CMAQ-ideal/DATA

[editar] Compilación icon

Empezando en 

/home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/PREP/icon/scripts
./bldit_icon.csh intel >& run_bldit_icon.log
ls BLD_ICON_v54_intel/
avprofile_cb6r3m_ae7_kmtbr_hemi2016_v53beta2_m3dry_col051_row068.csv
avprofile_racm_ae6_aq_derived_from_cb6r3m_ae7_kmtbr_hemi2016_v53beta2_m3dry_col051_row068.csv
avprofile_saprc07tc_ae6_aq_derived_from_cb6r3m_ae7_kmtbr_hemi2016_v53beta2_m3dry_col051_row068.csv
avprofile_saprc07tic_ae7i_aq_derived_from_cb6r3m_ae7_kmtbr_hemi2016_v53beta2_m3dry_col051_row068.csv
findex.F
HGRD_DEFN.F
icon.F
ICON_v54.cfg
IC_PARMS.F
lat_lon.F
legacy_PM_toxics_profile.csv
lr_interp.F
m3_ck_ctmmet.F
m3_ck_ctms.F
m3_ck_icmet.F
m3_driver.F
m3_icout.F
m3_vinterp.F
Makefile
Makefile.intel
opn_ic_file.F
prof_driver.F
prof_icout.F
prof_vinterp.F
VGRD_DEFN.F

Compilando: 

cd BLD_ICON_v54_intel
/opt/load-libs.sh 1
source /opt/env_scripts/load_intel-2021.4.0_mpich-3.4.2.sh
make >& run_make.log
ls *exe
ICON_v54.exe

[editar] Compilación bcon

Empezando desde /home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/PREP/bcon/scripts 

./bldit_bcon.csh intel >& run_bldit_bcon.log
cd BLD_BCON_v54_intel/
ls BLD_BCON_v54_intel/*exe
BLD_BCON_v54_intel/BCON_v54.exe

[editar] Benchmark

Descargando los datos del test-case de acá

Datos en hydra $CMAQ_DATA=/home/solange.luque/MODELOS/CMAQ/DATA/benchmark 

tar xvfz CMAQv5.4_2018_12NE3_Benchmark_2Day_Input.tar.gz
du -hsc ./CMAQv5.4_2018_12NE3_Benchmark_2Day_Input
56G     ./CMAQv5.4_2018_12NE3_Benchmark_2Day_Input

Home CMAQ en hydra $CMAQ_HOME=/home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project

Enlazando los datos 

cd /home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project
mkdir -p data
cd data
ln -s /home/solange.luque/MODELOS/CMAQ/DATA/benchmark/CMAQv5.4_2018_12NE3_Benchmark_2Day_Input/2018_12NE3 ./

Corriendo benchmark 

source /opt/env_scripts/load_intel-2021.4.0_mpich-3.4.2.sh
cd $CMAQ_HOME/CCTM/scripts
cp run_cctm_Bench_2018_12NE3.csh run_cctm_Bench_2018_12NE3.csh.old
diff run_cctm_Bench_2018_12NE3.csh run_cctm_Bench_2018_12NE3.csh.old
716,717c716
<   #( time -p mpirun -np $NPROCS $BLD/$EXEC ) |& tee buff_${EXECUTION_ID}.txt
<   (mpirun -np $NPROCS $BLD/$EXEC ) |& tee buff_${EXECUTION_ID}.txt
---
>   ( /usr/bin/time -p mpirun -np $NPROCS $BLD/$EXEC ) |& tee buff_${EXECUTION_ID}.txt
721c720
<   #rm -rf buff_${EXECUTION_ID}.txt
---
>   rm -rf buff_${EXECUTION_ID}.txt
732d730
<     echo "Have a look into buff_${EXECUTION_ID}.txt"
735d732
<   rm -rf buff_${EXECUTION_ID}.txt
qsub launch_cctm_intel.pb
#./run_cctm_Bench_2018_12NE3.csh >& run_Bench_2018-12NE3.log
tail -n 30 run_Bench_2018-12NE3.log 
 
     ==============================================
     |>---   PROGRAM COMPLETED SUCCESSFULLY   ---<|
     ==============================================
     Date and time 0:00:00   July 2, 2018   (2018183:000000)
 
     The elapsed time for this simulation was    2206.0 seconds.
 

CMAQ Processing of Day 20180701 Finished at Wed 16 Nov 2022 04:11:26 PM -03

\\\\\=====\\\\\=====\\\\\=====\\\\\=====/////=====/////=====/////=====/////


==================================
  ***** CMAQ TIMING REPORT *****
==================================
Start Day: 2018-07-01
End Day:   2018-07-01
Number of Simulation Days: 1
Domain Name:               2018_12NE3
Number of Grid Cells:      367500  (ROW x COL x LAY)
Number of Layers:          35
Number of Processes:       32
   All times are in seconds.

Num  Day        Wall Time
01   2018-07-01   2206.0
     Total Time = 2206.00
      Avg. Time = 2206.00

[editar] caso ideal

Partimos de una simulación de WRF y creamos los forzantes atmosféricos

Directorio de trabajo: /home/solange.luque/estudios/CMAQ-ideal

[editar] Meteorología

Para generar las condiciones meteorológicas ideales vamos a usar de base condiciones reales de una corridade WRF (wrfout_d03: dominio sobre AMBA a 1 km resolución, wrfour_d01: dominio centro SA a 50 km): 

ls /home/solange.luque/estudios/CMAQ-ideal/DATA/wrf
geo_em.d03.nc wrfout_d01_2012-08-03_00:00:00  wrfout_d03_2012-11-15_00:00:00  
wrfout_d03_2012-11-15_12:00:00


A partir de estos ficheros creamos los ficheros meteorológicos ideales con velocidad constante usando el script getpoint_everywhere. Por ejemplo, para una simulación con solo velocidad de viento en la dirección horizontal y velocidad de 3m/s 

python3 getpoint_everywhere.py -i 60 -j 60 -x west_east -y south_north -f 'wrfout_d03_2012-11-15_00:00:00' -v 'U@alloverct@3.:V@alloverct@0.' -s 'Time@0@-9@6'
python3 getpoint_everywhere.py -i 60 -j 60 -x west_east -y south_north -f 'wrfout_d03_2012-11-15_12:00:00' -v 'U@alloverct@3.:V@alloverct@0.' -s 'Time@0@-9@6'

Esto genera dos ficheros que renombramos: 

wrfout_d03_2012-11-15_00:00:00_3
wrfout_d03_2012-11-15_12:00:00_3

Ahora vamos a correr mcip con estas condiciones. Partiendo del run_mcip.csh</pre> que creamos para el AMBA (caso prueba): 

export CMAQ_HOME=/home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/
cd /home/solange.luque/MODELOS/CMAQ/intel/src/CMAQ-master/CMAQ_Project/PREP/mcip/scripts/
cp run_mcip.csh run_mcip_ideal.csh

diff run_mcip.csh run_mcip_ideal.csh
131c131
< set APPL       = AMBA
---
> set APPL       = AMBAidealArea
135c135
< set DataPath   = /home/solange.luque/estudios/CMAQ-AMBA/prueba/DATA
---
> set DataPath   = /home/solange.luque/estudios/CMAQ-ideal/DATA
163,165c163,164
< set InMetFiles = ( $InMetDir/wrfout_d03_2012-11-14_12:00:00 \
< 		   $InMetDir/wrfout_d03_2012-11-15_00:00:00 \
<                    $InMetDir/wrfout_d03_2012-11-15_12:00:00 )
---
> set InMetFiles = ( $InMetDir/wrfout_d03_2012-11-15_00:00:00_3 \
> 		   $InMetDir/wrfout_d03_2012-11-15_12:00:00_3)

Ahora sí, corremos mcip: 

./run_mcip_ideal.csh >& run_mcip_ideal.log


[editar] caso prueba AMBA

WORKDIR = /home/solange.luque/estudios/CMAQ-AMBA/prueba

Qué se necesita:

  1. Tener una salida de WRF y perararla para CMAQ
  2. Inventarios de emisiones y prepararlos para CMAQ
  3. Condiciones de contorno e iniciales para la química atmosférica
  4. Configurar CMAQ
  5. Simular CMAQ

Todos los pasos se explican con más detalle CMAQ/pruebaAMBA

Obtenido de «https://wiki.cima.fcen.uba.ar/mediawiki/index.php?title=CMAQ&oldid=2921»
Herramientas personales