Revisión de 10:25 16 jul 2019

Compilation at DCAO's computational classroom

Installation steps using modipsl

• Required packages:

# apt-get install subversion git libnetcdf-dev libnetcdff-dev make ncview ksh csh mpi mpich vim 

• Additional for python and netcdf (not required by ORCHIDEE)

# apt-get install python-matplotlib python-matplotlib-data python-matplotlib-doc python-mpltoolkits.basemap 
  python-mpltoolkits.basemap-data dvipng python-netcdf4

• And fake gmake (Ubunutu)

ln -s /usr/bin/make /usr/bin/gmake

• Go to the installation directory [INSTALLDIR] (for hydra [INSTALLDIR]=/home/administrador/installOR)

$ cd [INSTALLDIR]

• Getting the IPSL-models suite

$ svn co https://forge.ipsl.jussieu.fr/igcmg/svn/modipsl/trunk modipsl

• Going to the util folder:

$ cd modipsl/util

• Getting the off-line version of ORCHIDEE

$ ./model ORCHIDEE_trunk

• We got some issues with the ports, thus, it needs to be manually done:

$ cd ../modeles

• Getting IOIPSL:

$ mkdir -p IOIPSL/src 
$ svn co https://forge.ipsl.jussieu.fr/igcmg/svn/IOIPSL/tags/v2_2_2/src IOIPSL/src

• Getting XIOS library

$ svn co -r 1043 https://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/trunk XIOS

• By the end one should have:

$ ls
IOIPSL  ORCHIDEE  XIOS

• Going back to util to proceed

$ cd ../util

• These are the final versions of the codes
  • ORCHIDEE: 6094
  • ORCHIDEE_OL: 4622
  • IOIPSL: 3379
  • XIOS: 1043
• Preparing compilation environment for hydra. One need to add into AA_make.gdef specifications for DCAO_gfortran (at the end of file):

#-Q- DCAO_gfortran  #- Global definitions for DCAO, mpi gfortran
#-Q- DCAO_gfortran  LIB_MPI = MPI1
#-Q- DCAO_gfortran  LIB_MPI_BIS = MPI1
#-Q- DCAO_gfortran  FCM_ARCH = DCAO_gfortran 
#-Q- DCAO_gfortran  M_K = make
#-Q- DCAO_gfortran  P_C = cpp
#-Q- DCAO_gfortran  P_O = -fpreprocessed -P -C -traditional $(P_P)
#-Q- DCAO_gfortran  F_C = gfortran -c -cpp
#-Q- DCAO_gfortran  #-D- MD    F_D = -g -Wall -fbounds-check -pedantic -finit-real=nan
#-Q- DCAO_gfortran  #-D- MN    F_D =
#-Q- DCAO_gfortran  #-P- I4R4  F_P =
#-Q- DCAO_gfortran  #-P- I4R8  F_P = -fdefault-real-8
#-Q- DCAO_gfortran  #-P- I8R8  F_P = -fdefault-integer-8 -fdefault-real-8
#-Q- DCAO_gfortran  w_w = -O3 -funroll-all-loops $(F_D) $(F_P) -I$(MODDIR)
#-Q- DCAO_gfortran  F_O = $(w_w) -J$(MODDIR)
#-Q- DCAO_gfortran  F_L = gfortran
#-Q- DCAO_gfortran  M_M = 0
#-Q- DCAO_gfortran  L_X = 0
#-Q- DCAO_gfortran  L_O =
#-Q- DCAO_gfortran  A_C = ar -rs
#-Q- DCAO_gfortran  A_G = ar -x
#-Q- DCAO_gfortran  C_C = cc -c
#-Q- DCAO_gfortran  C_O =
#-Q- DCAO_gfortran  C_L = cc
#-Q- DCAO_gfortran  #-
#-Q- DCAO_gfortran  NCDF_INC = /usr/include
#-Q- DCAO_gfortran  NCDF_LIB = -L/usr/lib/x86_64-linux-gnu -lnetcdf
#-Q- DCAO_gfortran  #-

• Creation of the configuration arch files for the compilation of each component

XIOS

• • Creation of 'fcm' compilation environtment file for DCAO_gfortran

$ vim ../modeles/XIOS/arch/arch-DCAO_gfortran.fcm

• • values:

################################################################################
###################                Projet XIOS               ###################
################################################################################

%CCOMPILER      /usr/bin/mpicc
%FCOMPILER      /usr/bin/mpif90
%LINKER         /usr/bin/mpif90 -nofor-main

%BASE_CFLAGS    -ansi -w -D_GLIBCXX_USE_CXX11_ABI=0
%PROD_CFLAGS    -O3 -DBOOST_DISABLE_ASSERTS
%DEV_CFLAGS     -g -O2
%DEBUG_CFLAGS   -g

%BASE_FFLAGS    -D__NONE__ -ffree-line-length-none
%PROD_FFLAGS    -O3
%DEV_FFLAGS     -g -O2
%DEBUG_FFLAGS   -g

%BASE_INC       -D__NONE__
%BASE_LD        -lstdc++

%CPP            cpp
%FPP            cpp -P
%MAKE           gmake

• • Creation of 'path' compilation environtment file for DCAO_gfortran

$ vim ../modeles/XIOS/arch/arch-DCAO_gfortran.path

• • values:

NETCDF_INCDIR="-I $NETCDF_INC_DIR"
NETCDF_LIBDIR="-L $NETCDF_LIB_DIR"
NETCDF_LIB="-lnetcdff -lnetcdf"

MPI_INCDIR=""
MPI_LIBDIR=""
MPI_LIB="-lcurl"

HDF5_INCDIR="-I $HDF5_INC_DIR"
HDF5_LIBDIR="-L $HDF5_LIB_DIR"
HDF5_LIB="-lhdf5_hl -lhdf5 -lhdf5 -lz"

#OASIS_INCDIR="-I$PWD/../../oasis3-mct/BLD/build/lib/psmile.MPI1"
#OASIS_LIBDIR="-L$PWD/../../oasis3-mct/BLD/lib"
#OASIS_LIB="-lpsmile.MPI1 -lscrip -lmct -lmpeu"

• • Creation of 'env' compilation environtment file for DCAO_gfortran

$ vim ../modeles/XIOS/arch/arch-DCAO_gfortran.env

• • values:

export HDF5_INC_DIR=/usr/include
export HDF5_LIB_DIR=/usr/lib/x86_64-linux-gnu

export NETCDF_INC_DIR=/usr/include
export NETCDF_LIB_DIR=/usr/lib/x86_64-linux-gnu

ORCHIDEE

• • Creation of 'fcm' compilation environtment file for DCAO_gfortran

$ vim ../modeles/ORCHIDEE/arch/arch-DCAO_gfortran.fcm

• • values:

%COMPILER            /usr/bin/mpicc
%LINK                /usr/bin/mpif90
%AR                  ar
%MAKE                make
%FPP_FLAGS           -P -C -traditional -nostdinc
%FPP_DEF
%BASE_FFLAGS         -fdefault-real-8
%PROD_FFLAGS         -O3 -funroll-all-loops
%DEV_FFLAGS          -O
%DEBUG_FFLAGS        -ffpe-trap=invalid,zero,overflow -Wall -fbounds-check -g3 -O0 -fstack-protector-all
%MPI_FFLAGS
%OMP_FFLAGS          -openmp -openmp-threadprivate compat
%BASE_LD
%MPI_LD
%OMP_LD              -openmp

• • Creation of 'path' compilation environtment file for DCAO_gfortran

$ vim ../modeles/ORCHIDEE/arch/arch-DCAO_gfortran.path

• • values:

NETCDF_INCDIR="-I/usr/include"
NETCDF_LIBDIR="-L/usr/lib/x86_64-linux-gnu"
NETCDF_LIB="-lnetcdf -lnetcdff"

IOIPSL_INCDIR="-I${ORCHDIR}/../../lib -I${ORCHDIR}/../IOIPSL/inc"
IOIPSL_LIBDIR="-L${ORCHDIR}/../../lib -L${ORCHDIR}/../IOIPSL/lib"
IOIPSL_LIB="-lioipsl"

XIOS_INCDIR="-I${ORCHDIR}/../XIOS/inc"
XIOS_LIBDIR="-L${ORCHDIR}/../XIOS/lib"
XIOS_LIB="-lxios"

Building & compiling

• • Installing the code:

$ ./ins_make -t DCAO_gfortran >& run_ins_make.log

• • Going there:

$ cd ../config/ORCHIDEE_OL

• • No need to loading the right environment since in hydra its by default
  • Compiling XIOS in sequential (in Makefile, adding no-parallel compilation):

./make_xios --netcdf_lib netcdf4_seq (...)

• • Compiling (we must compile without XIOS, otherwise it is not working):

$ gmake without_xios >& run_gmake.log

• • Checking if everything went fine:

$ cat -n run_gmake.log | grep Error
$ cat -n run_gmake.log | grep undefined
$ ls ../../bin/ 
orchideedriver orchidee_ol 

Compiling rebuild

Due to the fact that hydra does not have the netcdf libraries compiled in parallel, ORCHDEE output is written with multiple files being a file for each process. In order to overcome this output. IPSL does prepare a tool to join the files and provide the expected output. This tool is called rebuild (a Kornshell script which encapsulates de exacutable flio_rbld) and it comes with the IOIPSL libraries. The IOIPSL libraries, are a set of old libraries developed by the IPSL (M. A. Foujuls, J. Polcher among others) to deal with the netCDF data access for all the models developed at the IOIPSL. Compiling IOIPSL (following install rebuild. It has to be compiled independently. Here it is explained how it has to be done.

• Go to the instalation directory [INSTALLDIR] (at hydra share)

$ cd [INSTALLDIR]

• getting the last version of the code

$ svn co https://forge.ipsl.jussieu.fr/igcmg/svn/modipsl/trunk modipsl_IOIPSLtools

• going to prepare compilation/installation

$ cd modipsl_IOIPSLtools/util

• preparing environment

./model IOIPSL_PLUS

• We got some issues with the ports, thus, it needs to be manually done:

$ cd ../modeles

• Getting IOIPSL (NOTE: notice the difference in library's version from the previous one):

$ svn co https://forge.ipsl.jussieu.fr/igcmg/svn/IOIPSL/tags/v2_2_3 IOIPSL

• Comming back to the util folder

$ cd ../util

• Creation of the compilation environment for `DCAO' using 'gfortran' inside AA_make.gdef as in the full compilation

$ cp ../../modipsl/util/AA_make.gdef ./

• Preparing makefiles

$ ./ins_make -t "DCAO_gfortran" -p I4R8

• going to src folder to compile

$ cd ../modeles/IOIPSL/src

• cleaning (you never know...)

$ make clean

• compiling

$ make >& run_make.log

• Checking no errors:

$ cat -n run_make.log | grep Error

• compilation of tools (file concatenation)

$ cd ../tools 

• cleaning (you never know...)

$ make clean

• compiling

$ make >& run_make.log

• Checking no errors:

$ cat -n run_make.log | grep Error

• One should have:

$ ls ../../../bin/ 
flio_rbld  rebuild

debug Instalation

Getting a debugging copy for test purposes

Instalation steps using modipsl

• Go to the instalation directory [INSTALLDIR]

$ cd [INSTALLDIR]

• Getting the IPSL-models suite (r 3379)

$ svn co https://forge.ipsl.jussieu.fr/igcmg/svn/modipsl/trunk modipsl_dbg

• Once we got the codes, we need to copy the specificities of the installation for hydra

$ cp ../../modipsl/util/AA_make.gdef ./
$ cp ../../modipsl/modeles/XIOS/arch/*hydra* ../modeles/XIOS/arch/
$ cp ../../modipsl/modeles/ORCHIDEE/arch/*hydra* ../modeles/ORCHIDEE/arch/

• Everything is the same as for the installation, except that we need to indicate debugging compilation:

./ins_make -t DCAO_gfortran -d >& run_ins_make.log

• And on Makefile from config/ORCHIDEE_OL, change all the -prod by -debug

(...)
xios : 
        (cd  ../../modeles/XIOS ; ./make_xios --netcdf_lib netcdf4_seq \
--debug --arch ${FCM_ARCH} --job 8 ; cp bin/xios_server.exe ../../bin/. ; ) 
(...)
driver : 
        (cd ../../modeles/ORCHIDEE ; ./makeorchidee_fcm -j 8 -parallel mpi -debug -arch ${FCM_ARCH} -driver ; )
(...)
driver_xios : 
        (cd ../../modeles/ORCHIDEE ; ./makeorchidee_fcm -xios2 -j 8 -parallel mpi -debug -arch ${FCM_ARCH} -driver ; ) 
(...)

• Compiling:

$ gmake >& run_gmake.log

• Checking if everything went fine:

$ cat -n run_gmake.log | grep Error 
$ cat -n run_gmake.log | grep undefined
$ ls ../../bin/ 
orchidee_ol  xios_server.exe