Revisión actual - 12:21 2 oct 2019

Compilation at DCAO's computational classroom

Installation steps using modipsl

Required packages:

# apt-get install subversion git libnetcdf-dev libnetcdff-dev netcdf-bin make 
  ncview ksh csh mpi mpich vim

Additional for python and netcdf (not required by ORCHIDEE)

# apt-get install python-matplotlib python-matplotlib-data python-matplotlib-doc 
python-mpltoolkits.basemap python-mpltoolkits.basemap-data dvipng python-netcdf4

And fake gmake (Ubunutu)

ln -s /usr/bin/make /usr/bin/gmake

Go to the installation directory [INSTALLDIR] (for hydra [INSTALLDIR]=/home/administrador/installOR)

$ cd [INSTALLDIR]

Getting the IPSL-models suite

$ svn co https://forge.ipsl.jussieu.fr/igcmg/svn/modipsl/trunk modipsl

Going to the util folder:

$ cd modipsl/util

Getting the off-line version of ORCHIDEE

$ ./model ORCHIDEE_trunk

We got some issues with the ports, thus, it needs to be manually done:

$ cd ../modeles

Getting IOIPSL:

$ mkdir -p IOIPSL/src 
$ svn co https://forge.ipsl.jussieu.fr/igcmg/svn/IOIPSL/tags/v2_2_4/src IOIPSL/src

Getting XIOS library

$ svn co -r 1043 https://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/trunk XIOS

By the end one should have:

$ ls
IOIPSL  ORCHIDEE  XIOS

Going back to util to proceed

$ cd ../util

These are the final versions of the codes
- ORCHIDEE: 6094
- ORCHIDEE_OL: 4622
- IOIPSL: 3379
- XIOS: 1043
Preparing compilation environment for hydra. One need to add into AA_make.gdef specifications for DCAO_gfortran (at the end of file):

#-Q- DCAO_gfortran  #- Global definitions for DCAO, mpi gfortran
#-Q- DCAO_gfortran  LIB_MPI = MPI1
#-Q- DCAO_gfortran  LIB_MPI_BIS = MPI1
#-Q- DCAO_gfortran  FCM_ARCH = DCAO_gfortran 
#-Q- DCAO_gfortran  M_K = make
#-Q- DCAO_gfortran  P_C = cpp
#-Q- DCAO_gfortran  P_O = -fpreprocessed -P -C -traditional $(P_P)
#-Q- DCAO_gfortran  F_C = gfortran -c -cpp
#-Q- DCAO_gfortran  #-D- MD    F_D = -g -Wall -fbounds-check -pedantic -finit-real=nan
#-Q- DCAO_gfortran  #-D- MN    F_D =
#-Q- DCAO_gfortran  #-P- I4R4  F_P =
#-Q- DCAO_gfortran  #-P- I4R8  F_P = -fdefault-real-8
#-Q- DCAO_gfortran  #-P- I8R8  F_P = -fdefault-integer-8 -fdefault-real-8
#-Q- DCAO_gfortran  w_w = -O3 -funroll-all-loops $(F_D) $(F_P) -I$(MODDIR)
#-Q- DCAO_gfortran  F_O = $(w_w) -J$(MODDIR)
#-Q- DCAO_gfortran  F_L = gfortran
#-Q- DCAO_gfortran  M_M = 0
#-Q- DCAO_gfortran  L_X = 0
#-Q- DCAO_gfortran  L_O =
#-Q- DCAO_gfortran  A_C = ar -rs
#-Q- DCAO_gfortran  A_G = ar -x
#-Q- DCAO_gfortran  C_C = cc -c
#-Q- DCAO_gfortran  C_O =
#-Q- DCAO_gfortran  C_L = cc
#-Q- DCAO_gfortran  #-
#-Q- DCAO_gfortran  NCDF_INC = /usr/include
#-Q- DCAO_gfortran  NCDF_LIB = -L/usr/lib/x86_64-linux-gnu -lnetcdf
#-Q- DCAO_gfortran  #-

Creation of the configuration arch files for the compilation of each component

XIOS

- Creation of 'fcm' compilation environtment file for DCAO_gfortran

$ vim ../modeles/XIOS/arch/arch-DCAO_gfortran.fcm

- values:

################################################################################
###################                Projet XIOS               ###################
################################################################################

%CCOMPILER      /usr/bin/mpicc
%FCOMPILER      /usr/bin/mpif90
%LINKER         /usr/bin/mpif90 -nofor-main

%BASE_CFLAGS    -ansi -w -D_GLIBCXX_USE_CXX11_ABI=0
%PROD_CFLAGS    -O3 -DBOOST_DISABLE_ASSERTS
%DEV_CFLAGS     -g -O2
%DEBUG_CFLAGS   -g

%BASE_FFLAGS    -D__NONE__ -ffree-line-length-none
%PROD_FFLAGS    -O3
%DEV_FFLAGS     -g -O2
%DEBUG_FFLAGS   -g

%BASE_INC       -D__NONE__
%BASE_LD        -lstdc++

%CPP            cpp
%FPP            cpp -P
%MAKE           gmake

- Creation of 'path' compilation environtment file for DCAO_gfortran

$ vim ../modeles/XIOS/arch/arch-DCAO_gfortran.path

- values:

NETCDF_INCDIR="-I $NETCDF_INC_DIR"
NETCDF_LIBDIR="-L $NETCDF_LIB_DIR"
NETCDF_LIB="-lnetcdff -lnetcdf"

MPI_INCDIR=""
MPI_LIBDIR=""
MPI_LIB="-lcurl"

HDF5_INCDIR="-I $HDF5_INC_DIR"
HDF5_LIBDIR="-L $HDF5_LIB_DIR"
HDF5_LIB="-lhdf5_hl -lhdf5 -lhdf5 -lz"

#OASIS_INCDIR="-I$PWD/../../oasis3-mct/BLD/build/lib/psmile.MPI1"
#OASIS_LIBDIR="-L$PWD/../../oasis3-mct/BLD/lib"
#OASIS_LIB="-lpsmile.MPI1 -lscrip -lmct -lmpeu"

- Creation of 'env' compilation environtment file for DCAO_gfortran

$ vim ../modeles/XIOS/arch/arch-DCAO_gfortran.env

- values:

export HDF5_INC_DIR=/usr/include
export HDF5_LIB_DIR=/usr/lib/x86_64-linux-gnu

export NETCDF_INC_DIR=/usr/include
export NETCDF_LIB_DIR=/usr/lib/x86_64-linux-gnu

ORCHIDEE

- Creation of 'fcm' compilation environtment file for DCAO_gfortran

$ vim ../modeles/ORCHIDEE/arch/arch-DCAO_gfortran.fcm

- values:

%COMPILER            /usr/bin/mpicc
%LINK                /usr/bin/mpif90
%AR                  ar
%MAKE                make
%FPP_FLAGS           -P -C -traditional -nostdinc
%FPP_DEF
%BASE_FFLAGS         -fdefault-real-8
%PROD_FFLAGS         -O3 -funroll-all-loops
%DEV_FFLAGS          -O
%DEBUG_FFLAGS        -ffpe-trap=invalid,zero,overflow -Wall -fbounds-check -g3 -O0 -fstack-protector-all
%MPI_FFLAGS
%OMP_FFLAGS          -openmp -openmp-threadprivate compat
%BASE_LD
%MPI_LD
%OMP_LD              -openmp

- Creation of 'path' compilation environtment file for DCAO_gfortran

$ vim ../modeles/ORCHIDEE/arch/arch-DCAO_gfortran.path

- values:

NETCDF_INCDIR="-I/usr/include"
NETCDF_LIBDIR="-L/usr/lib/x86_64-linux-gnu"
NETCDF_LIB="-lnetcdf -lnetcdff"

IOIPSL_INCDIR="-I${ORCHDIR}/../../lib -I${ORCHDIR}/../IOIPSL/inc"
IOIPSL_LIBDIR="-L${ORCHDIR}/../../lib -L${ORCHDIR}/../IOIPSL/lib"
IOIPSL_LIB="-lioipsl"

XIOS_INCDIR="-I${ORCHDIR}/../XIOS/inc"
XIOS_LIBDIR="-L${ORCHDIR}/../XIOS/lib"
XIOS_LIB="-lxios"

Building & compiling

- Installing the code:

$ ./ins_make -t DCAO_gfortran >& run_ins_make.log

- Going there:

$ cd ../config/ORCHIDEE_OL

- No need to loading the right environment since in hydra its by default
- Compiling XIOS in sequential (in Makefile, adding no-parallel compilation):

./make_xios --netcdf_lib netcdf4_seq (...)

- Compiling (we must compile without XIOS, otherwise it is not working):

$ gmake without_xios >& run_gmake.log

- Checking if everything went fine:

$ cat -n run_gmake.log | grep Error
$ cat -n run_gmake.log | grep undefined
$ ls ../../bin/ 
orchideedriver orchidee_ol

Compiling rebuild

Due to the fact that hydra does not have the netcdf libraries compiled in parallel, ORCHDEE output is written with multiple files being a file for each process. In order to overcome this output. IPSL does prepare a tool to join the files and provide the expected output. This tool is called rebuild (a Kornshell script which encapsulates de exacutable flio_rbld) and it comes with the IOIPSL libraries. The IOIPSL libraries, are a set of old libraries developed by the IPSL (M. A. Foujuls, J. Polcher among others) to deal with the netCDF data access for all the models developed at the IOIPSL. Compiling IOIPSL (following install rebuild. It has to be compiled independently. Here it is explained how it has to be done.

Go to the instalation directory [INSTALLDIR]

$ cd [INSTALLDIR]

getting the last version of the code

$ svn co https://forge.ipsl.jussieu.fr/igcmg/svn/modipsl/trunk modipsl_IOIPSLtools

going to prepare compilation/installation

$ cd modipsl_IOIPSLtools/util

preparing environment

./model IOIPSL_PLUS

We got some issues with the ports, thus, it needs to be manually done:

$ cd ../modeles

Getting IOIPSL (NOTE: notice the difference in library's version from the previous one):

$ svn co https://forge.ipsl.jussieu.fr/igcmg/svn/IOIPSL/tags/v2_2_3 IOIPSL

Comming back to the util folder

$ cd ../util

Creation of the compilation environment for `DCAO' using 'gfortran' inside AA_make.gdef as in the full compilation

$ cp ../../modipsl/util/AA_make.gdef ./

Preparing makefiles

$ ./ins_make -t "DCAO_gfortran" -p I4R8

going to src folder to compile

$ cd ../modeles/IOIPSL/src

cleaning (you never know...)

$ make clean

compiling

$ make >& run_make.log

Checking no errors:

$ cat -n run_make.log | grep Error

compilation of tools (file concatenation)

$ cd ../tools

cleaning (you never know...)

$ make clean

compiling

$ make >& run_make.log

Checking no errors:

$ cat -n run_make.log | grep Error

One should have:

$ ls ../../../bin/ 
flio_rbld  rebuild

debug Instalation

Getting a debugging copy for test purposes

Instalation steps using modipsl

Go to the instalation directory [INSTALLDIR]

$ cd [INSTALLDIR]

Getting the IPSL-models suite (r 3379)

$ svn co https://forge.ipsl.jussieu.fr/igcmg/svn/modipsl/trunk modipsl_dbg

Once we got the codes, we need to copy the specificities of the installation for hydra

$ cp ../../modipsl/util/AA_make.gdef ./
$ cp ../../modipsl/modeles/XIOS/arch/*hydra* ../modeles/XIOS/arch/
$ cp ../../modipsl/modeles/ORCHIDEE/arch/*hydra* ../modeles/ORCHIDEE/arch/

Everything is the same as for the installation, except that we need to indicate debugging compilation:

./ins_make -t DCAO_gfortran -d >& run_ins_make.log

And on Makefile from config/ORCHIDEE_OL, change all the -prod by -debug

(...)
xios : 
        (cd  ../../modeles/XIOS ; ./make_xios --netcdf_lib netcdf4_seq \
--debug --arch ${FCM_ARCH} --job 8 ; cp bin/xios_server.exe ../../bin/. ; ) 
(...)
driver : 
        (cd ../../modeles/ORCHIDEE ; ./makeorchidee_fcm -j 8 -parallel mpi -debug -arch ${FCM_ARCH} -driver ; )
(...)
driver_xios : 
        (cd ../../modeles/ORCHIDEE ; ./makeorchidee_fcm -xios2 -j 8 -parallel mpi -debug -arch ${FCM_ARCH} -driver ; ) 
(...)

Compiling:

$ gmake >& run_gmake.log

Checking if everything went fine:

$ cat -n run_gmake.log | grep Error 
$ cat -n run_gmake.log | grep undefined
$ ls ../../bin/ 
orchidee_ol  xios_server.exe