CROCO

De Wikicima
(Diferencias entre revisiones)
Saltar a: navegación, buscar
(→‎dmpar: Upwelling)
(→‎v1.1 - Nicolás Aubone modified)
Línea 418: Línea 418:
   
 
== v1.1 - Nicolás Aubone modified ==
  
== v1.1 - Nicolás Aubone modified ==
  +
  +
Creating the folder of the compilation (using intel compiler and shared memory ''dmpar'') and where to deploy the code
  +
<PRE style="shell">
  +
$ mkdir $INSTALLDIR/v11
  +
$ cd $INSTALLDIR/v11
  +
$ mkdir -p intel/dmpar
  +
$ cd intel/dmpar
  +
$ tar xvfz [CROCOv1.1_NicolasAubone].tar.gz
  +
$ ls croco
  +
GRIF create_config.bash OBSTRUCTION README.md TEST_CASES
  +
BENCH MPI_NOLAND OCEAN requirements.txt XIOS
  +
CHANGELOG.md MUSTANG PISCES SCRIPTS
  +
</PRE>
  +
 
Forcing data to be copied in
  
Forcing data to be copied in
 
<PRE>
  
<PRE>

Revisión de 09:15 16 jul 2025

Web page of the instalation of the ocean model CROCO in CIMA's HPC 'hydra'

Contenido

Installation

CIMA's HPC has a common folder where all the people has access. This is going to be the place where CROCO model will be installed.

The installation is going to be in the following INSTALLDIR=/share/CROCO folder.

Downloading the code (there is a GItlab repository, but by now, we are going to install closed releases) 

$ wget https://gitlab.inria.fr/croco-ocean/croco/-/archive/v2.1.0/croco-v2.1.0.tar.gz

Creating the folder of the compilation (using intel compiler and shared memory dmpar) and where to deploy the code 

$ mkdir $INSTALLDIR/v210
$ cd $INSTALLDIR/v210
$ mkdir -p intel/dmpar
$ git clone --branch v2.1.0 https://gitlab.inria.fr/croco-ocean/croco.git croco-v2.1.0
Cloning into 'croco'...
remote: Enumerating objects: 52646, done.
remote: Counting objects: 100% (448/448), done.
remote: Compressing objects: 100% (263/263), done.
remote: Total 52646 (delta 268), reused 317 (delta 185), pack-reused 52198 (from 1)
Receiving objects: 100% (52646/52646), 300.67 MiB | 4.59 MiB/s, done.
Resolving deltas: 100% (40426/40426), done.
Note: switching to '291e8cd11b2329ee171c7a02c52c9c481dda981e'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false
$ ls croco
GRIF         create_config.bash  OBSTRUCTION  README.md         TEST_CASES
BENCH         MPI_NOLAND          OCEAN        requirements.txt  XIOS
CHANGELOG.md  MUSTANG             PISCES       SCRIPTS

Go to compile, first we set-up the environment for intel compilation 

$ source /opt/load-libs.sh 1
The following libraries, compiled with Intel 2021.4.0 compilers, were loaded:
* MPICH 3.4.2
* NetCDF 4
* HDF5 1.10.5
* JASPER 2.0.33

To change it please logout and login again.

To load this libraries from within a script add this line to script:
source /opt/load-libs.sh 1
$ cd croco

We create an ASCII file with the specificities of the compilation with intel in hydra, name CROCO_hydra_intel.env (following the instructios from here tutorial 00) 

#set environment variables for croco in HYDRA to be compiled with intel
source /opt/load-libs.sh 1
export CC=icc
export FC=ifort 
export F90=ifort 
export F77=ifort 
export NETCDF=/opt/netcdf/netcdf-4/intel/2021.4.0 
export PATH=$NETCDF/bin::${PATH} 
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}::${NETCDF}/lib

Loading the compilation environment 

$ source CROCO_hydra_intel.env

BASIN

The simplest installation would be the 'basin' configuration. We follow these instructions basin. Assuming that the source code resides in CROCOsrc=/share/CROCO/v210/intel/serie/croco 

$ CROCOsrc=/share/CROCO/v210/intel/serie/croco
$ cd ${CROCOsrc}
$ mkdir -p CONFIGS/BASIN
cp OCEAN/cppdefs.h CONFIGS/BASIN
cp OCEAN/param.h CONFIGS/BASIN
cp OCEAN/jobcomp CONFIGS/BASIN
cd CONFIGS/BASIN

We need to edit the configuration files.

Editing the file with the configuration 

$ cp cppdefs.h cppdefs.h_orig
$ diff cppdefs.h cppdefs.h_orig 
18c18
< #define  BASIN           /* Basin Example */
---
> #undef  BASIN           /* Basin Example */
56c56
< #undef REGIONAL        /* REGIONAL Applications */
---
> #define REGIONAL        /* REGIONAL Applications */

Editing the file with the size of the domain and configuration of the parallel environment 

$ cp param.h param.h_orig
$ diff param.h param.h_orig

Editing the file with the compilation 

$ cp jobcomp jobcomp_orig
$ diff jobcomp jobcomp_orig 
34,35c34
< CROCOsrc=/share/CROCO/v210/intel/serie/croco
< SOURCE1=${CROCOsrc}/OCEAN
---
> SOURCE1=../croco/OCEAN
45c45
< FC=ifort
---
> FC=gfortran

And compile 

$ ./jobcomp > jobcomp.log

If everything when fine, one should have: 

$ tail -n 15 jobcomp.log 
mv a.out croco
                                   
                                   
    .,:looddddddd:            ,;'  
  .cdddo;;oo'....            .ldd; 
 ,ddddd,                    ;odoc. 
.oddddo:               .'';odoc    
'dddddd;':,  .;;  .;;. :ccdo;      
.oddddddddooooddoooddooodl.        
 .oddddddddddddddddddddddl;;       
  .,ldddddddddddddddddddoc:lolo'.. 
    ;ddc;::::::::::codo,     .';:, 
   .ccc             ,cc,           
                                   
CROCO is OK
$ ls -l croco
-rwxrwxr-x 1 lluis.fita cima 1012160 Jul 15 15:11 croco
IMPORTANT NOTES
1. do not run the model in the same place where you compiled the source
2. Always use a queue-system job to run simulations, never run without it
3. Always run in your salidas/ folder never in $HOME

Upwelling

Compilation of the upwelling test case in distributed memory (dmpar) with intel. 


$ $ source /opt/load-libs.sh 1

The following libraries, compiled with Intel 2021.4.0 compilers, were loaded:
* MPICH 3.4.2
* NetCDF 4
* HDF5 1.10.5
* JASPER 2.0.33
$ mkdir /share/CROCO/v210/intel/dmpar/
$ git clone --branch v2.1.0 https://gitlab.inria.fr/croco-ocean/croco.git croco
$ cd croco
$ CROCOsrc=/share/CROCO/v210/intel/dmpar/croco

Copying configuration files 

$ mkdir -p CONFIGS/Upwelling
cp OCEAN/cppdefs.h CONFIGS/Upwelling
cp OCEAN/param.h CONFIGS/Upwelling
cp OCEAN/jobcomp CONFIGS/Upwelling
cd CONFIGS/Upwelling

Configuring the case 

$ cp cppdefs.h cppdefs.h_orig
$ diff cppdefs.h cppdefs.h_orig 
29c29
< #define  UPWELLING       /* Upwelling Example */
---
> #undef  UPWELLING       /* Upwelling Example */
56c56
< #undef REGIONAL        /* REGIONAL Applications */
---
> #define REGIONAL        /* REGIONAL Applications */
1037c1037
< # define MPI
---
> # undef  MPI

$ cp param.h param.h_orig
$ diff param.h param.h_orig 

$ cp jobcomp jobcomp_orig
$ diff jobcomp jobcomp_orig 
34,35c34
< CROCOsrc=/share/CROCO/v210/intel/dmpar/croco
< SOURCE1=${CROCOsrc}/OCEAN
---
> SOURCE1=../croco/OCEAN
46c45
< FC=ifort
---
> FC=gfortran
52,53c51,52
< MPILIB="-L/opt/mpich/mpich-3.4.2/intel/2021.4.0/lib"
< MPIINC="-I/opt/mpich/mpich-3.4.2/intel/2021.4.0/include"
---
> MPILIB=""
> MPIINC=""

Compiling in parallel 

$ ./jobcomp >& jobcomp.log
$ tail -n 15 jobcomp.log
 mv a.out croco
                                   
                                   
    .,:looddddddd:            ,;'  
  .cdddo;;oo'....            .ldd; 
 ,ddddd,                    ;odoc. 
.oddddo:               .'';odoc    
'dddddd;':,  .;;  .;;. :ccdo;      
.oddddddddooooddoooddooodl.        
 .oddddddddddddddddddddddl;;       
  .,ldddddddddddddddddddoc:lolo'.. 
    ;ddc;::::::::::codo,     .';:, 
   .ccc             ,cc,           
                                   
CROCO is OK

$ ls -l croco
-rwxrwxr-x 1 lluis.fita cima 1184600 Jul 15 16:34 croco


IMPORTANT NOTES
1. do not run the model in the same place where you compiled the source
2. Always use a queue-system job to run simulations, never run without it
3. Always run in your salidas/ folder never in $HOME

v1.1 - Nicolás Aubone modified

Creating the folder of the compilation (using intel compiler and shared memory dmpar) and where to deploy the code 

$ mkdir $INSTALLDIR/v11
$ cd $INSTALLDIR/v11
$ mkdir -p intel/dmpar
$ cd intel/dmpar
$ tar xvfz [CROCOv1.1_NicolasAubone].tar.gz 
$ ls croco
GRIF         create_config.bash  OBSTRUCTION  README.md         TEST_CASES
BENCH         MPI_NOLAND          OCEAN        requirements.txt  XIOS
CHANGELOG.md  MUSTANG             PISCES       SCRIPTS

Example of use

serie: BASIN

How to run a basic example 

$ cd $HOME/salidas
$ mkdir CROCO
$ cd CROCO
$ mkdir tests
$ cd tests/
$ cp /share/tools/workflows/direct/CROCO/hydra/launch_ideal_test_serie.pbs ./

Link the necessary files and editng them if needed: 

$ ln -s $CROCOsrc/CONFIGS/BASIN/croco ./
$ cp $CROCOsrc/TEST_CASES/croco.in.Basin ./

And execute 

$ qsub launch_ideal_test_serie.pbs
70619.hydra
$ qstat
Job ID                    Name             User            Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
(...)
70619.hydra                CROCO4L_ideal    lluis.fita             0 R larga

Looking for the current step: 

$ tail croco.out 
    2698   299.77778 8.942712777E-05-3.3148734E+00-3.3147840E+00 5.0400000E+16  0
    2699   299.88889 8.942810068E-05-3.3148728E+00-3.3147833E+00 5.0400000E+16  0
    2700   300.00000 8.942902900E-05-3.3148721E+00-3.3147827E+00 5.0400000E+16  0
      WRT_HIS -- wrote history fields into time record =   31 /   31
    2701   300.11111 8.942991352E-05-3.3148715E+00-3.3147820E+00 5.0400000E+16  0
    2702   300.22222 8.943075502E-05-3.3148708E+00-3.3147814E+00 5.0400000E+16  0
    2703   300.33333 8.943155431E-05-3.3148701E+00-3.3147807E+00 5.0400000E+16  0
    2704   300.44444 8.943231222E-05-3.3148695E+00-3.3147800E+00 5.0400000E+16  0
    2705   300.55556 8.943302959E-05-3.3148688E+00-3.3147794E+00 5.0400000E+16  0
    2706   300.66667 8.943370728E-05-3.3148682E+00-3.3147787E+00 5.0400000E+16  0

Once finished: 

$ qstat
Job ID                    Name             User            Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
(...)
70619.hydra                CROCO4L_ideal    lluis.fita      00:01:13 C larga          

$ tail croco.out 
    3239   359.88889 9.041893005E-05-3.3144938E+00-3.3144033E+00 5.0400000E+16  0
    3240   360.00000 9.041879739E-05-3.3144930E+00-3.3144026E+00 5.0400000E+16  0
      WRT_HIS -- wrote history fields into time record =   37 /   37

 MAIN - number of records written into history  file(s):   37
        number of records written into restart  file(s):    0


 MAIN: DONE

$ ls -lrt
total 15576
-rw-rw-r-- 1 lluis.fita cima     1412 Jul 15 15:40 launch_ideal_test_serie.pbs
lrwxrwxrwx 1 lluis.fita cima       55 Jul 15 15:44 croco -> /share/CROCO/v210/intel/serie/croco/CONFIGS/BASIN/croco
-rw-rw-r-- 1 lluis.fita cima     3361 Jul 15 15:46 croco.in.Basin
-rw------- 1 lluis.fita cima     3361 Jul 15 15:47 croco.in
-rw------- 1 lluis.fita cima      824 Jul 15 15:48 CROCO4L_ideal.o70619
-rw------- 1 lluis.fita cima 15653792 Jul 15 15:48 basin_his.nc
-rw------- 1 lluis.fita cima   274775 Jul 15 15:48 croco.out

dmpar: Upwelling

How to run a basic example 

$ cd $HOME/salidas
$ mkdir CROCO
$ cd CROCO
$ mkdir mpi
$ cd mpi/
$ cp /share/tools/workflows/direct/CROCO/hydra/launch_ideal_test_mpi.pbs ./

Link the necessary files and editng them if needed: 

$ ln -s $CROCOsrc/CONFIGS/Upwelling/croco ./
$ cp $CROCOsrc/TEST_CASES/croco.in.Upwelling ./

And execute 

$ qsub launch_ideal_test_mpi.pbs
70649.hydra
$ qstat
Job ID                    Name             User            Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
(...)
70645.hydra               CROCO4L_ideal_   lluis.fita             0 R larga

Once finished 

$ tail -3 croco.out 

 MAIN: DONE

$ ls -lrt
total 6956
lrwxrwxrwx 1 lluis.fita cima      59 Jul 15 16:42 croco -> /share/CROCO/v210/intel/dmpar/croco/CONFIGS/Upwelling/croco
-rw-rw-r-- 1 lluis.fita cima    1500 Jul 15 16:42 croco.in.Upwelling
-rw------- 1 lluis.fita cima    1500 Jul 15 16:43 croco.in
-rwxrwxr-x 1 lluis.fita cima    9328 Jul 15 16:43 a.out
-rw-rw-r-- 1 lluis.fita cima    1448 Jul 15 16:50 launch_ideal_test_mpi.pbs
-rw------- 1 lluis.fita cima  238630 Jul 15 16:50 CROCO4L_ideal_mpi.o70634
-rw------- 1 lluis.fita cima  717220 Jul 15 16:50 upwelling_rst.nc
-rw------- 1 lluis.fita cima 3316688 Jul 15 16:50 upwelling_his.nc
-rw------- 1 lluis.fita cima 2771248 Jul 15 16:50 upwelling_avg.nc
-rw------- 1 lluis.fita cima   42450 Jul 15 16:50 croco.out

v1.1 - Nicolás Aubone modified

Creating the folder of the compilation (using intel compiler and shared memory dmpar) and where to deploy the code 

$ mkdir $INSTALLDIR/v11
$ cd $INSTALLDIR/v11
$ mkdir -p intel/dmpar
$ cd intel/dmpar
$ tar xvfz [CROCOv1.1_NicolasAubone].tar.gz 
$ ls croco
GRIF         create_config.bash  OBSTRUCTION  README.md         TEST_CASES
BENCH         MPI_NOLAND          OCEAN        requirements.txt  XIOS
CHANGELOG.md  MUSTANG             PISCES       SCRIPTS

Forcing data to be copied in 

/share/DATA/CROCO
Obtenido de «https://wiki.cima.fcen.uba.ar/mediawiki/index.php?title=CROCO&oldid=3634»
Herramientas personales