Revisión de 11:19 18 jul 2019

ORCHIDEE's work-flow management for 1 processor.

Climate runs require simulations for long periods of time. Geophysical models require different sources of forcing. Usually one should not and ofently can not prepare boundary conditions for all the period of the run (usually more than 10 years). Instead, models perform long simulations by `chunks' or fractions of the period making use of restart files (in ORCHIDEE 1 year). For each of these chunks, forcing data has to be prepared.

One could do this manually waiting until the end of the run. It is much more efficient if a work-flow can be generated in order to concatenated the chunks in an automatic way.

Here is presented a very simple work-flow manager for off-line ORCHIDEE simulations on single processor machines.

Files & structure

It is assumed that ORCHIDEE has already been installed into the machine, and user already has downloaded all the required data for the run. On a main folder called [foldOR1proc] (as example) it is expected to contain:

• components: with two additional folders
  • ORtemplate: Folder with an example of ORCHIDEE's run.def; bin/, folder with the compiled binaries: orchideedriver, orchidee_ol, rebuild
  • python: Folder with required work-flow python scripts: change_namelist.py
  • shell: Folder with the work-flow scripts: run_multi-or_1proc.bash, prepare_forcings.bash; configuration file: or_1-proc.config; rebuild.bash shell to rebuild ORCHIDEE's output files
• forcings: Folder with all the ancialliary data and meteorological forcings: work-flow expects to find a folder labelled ATMOS with the atmospheric forcings, and any organization of folders for the rest of the data (it is sorted with IGCM/SRF/ original IPSL-ORCHIDEE sorting of forcings)

or_1-proc.config

This is the main file for the work-flow and the unique one to be modified by the user. It contains the definition of a series of variables. As an example:

# scratch
scratch=true

# Period of simulation
period=1990-2010

# config
pyBIN=/usr/bin/python
OUTfolder=/home/administrador/installOR/testORnewdriver
foldOR1proc=/home/administrador/installOR
forcingfolder=/home/administrador/installOR/forcings

# Anciliary data 1 file without year (original name of the files with the ancilliary data [folderName, since 
  ${forcings}]@[origFilen]@[ORfilen])
filestolink=IGCM/SRF/albedo@alb_bg_modisopt_2D_ESA_v3.nc@alb_bg.nc:IGCM/SRF@cartepente2d_15min.nc@cartepente2d_15min.nc:
  IGCM/SRF@reftemp.nc@reftemp.nc:IGCM/SRF@routing.nc@routing.nc:IGCM/SRF/SOIL/soil_bulk_and_ph.nc@soil_bulk_and_ph.nc:
  IGCM/SRF@soils_param.nc@soils_param.nc

# Anciliary data 1 file with year (original name of the files with the ancilliary data [folderName, since ${forcings}]@[origFilen]@[ORfilen], [YYYY] to be changed by the correspondant year)
filestolinkyr=IGCM/SRF/PFTMAPS/CMIP6/ESA-LUH2v2/historical/15PFT.v2@PFTmap_YYYY.nc@PFTmap.nc:
  IGCM/SRF/WOODHARVEST/LUH2v2/historical4@woodharvest_YYYY.nc@woodharvest.nc

# Anciliary data 3 files with year-1,year,year+1 (original name of the files with the ancilliary data 
  [folderName, since ${forcings}]@[origFilen]@[ORfilen], [YYYY] to be changed by the correspondant year-1, year, yerar+1)
filestolink3yr=

# Atmospheric forcing as [ForcingName]@[HeaderForcingFiles] [assuming ${forcingfolder}/ATMOS/{ForcingName}/
  {HeaderForcingFiles}_YYYY.nc]
atmos=CRUNCEP@cruncep_halfdeg

# Additional parameters for run.def as [paramName1]@[values1];...;[paramNameN]@[valuesN] (! for spaces)
addparams=

• period: period of simulation providing the initial and the ending year in format [iyear]-[eyear]
• pyBIN: path to the binary of python (2.x version is required)
• OUTfolder: path to the place where the simulation will be performed
• foldOR1proc: path to the main folder to the work-flow manager
• forcingfolderc: path to the folder with the forcing data (atmospheric under ATMOS, and the rest)
• filestolink: ':' separated list of values for the anciliary data that comes from files without year dependency (in format [origFilen]@[ORfilen]; [origFilen], original name of the file; [ORfilen]: expected name by ORCCHIDEE)
• filestolinkyr: ':' separated list of values for the ancillary data that comes from files with year dependency (in format [origFilen]@[ORfilen]; [origFilen], original name of the file; [ORfilen]: expected name by ORCCHIDEE. Name of the files must contain the string chain 'YYYY' that will be automatically changed)
• filestolink3yr: ':' separated list of values for the ancillary data that comes from files with year dependency (in format [origFilen]@[ORfilen]; [origFilen], original name of the file; [ORfilen]: expected name by ORCCHIDEE. Name of the files must contain the string chain 'YYYY' that will be automatically changed as year-1, year, yerar+1)
• atmos Atmospheric forcing as [ForcingName]@[HeaderForcingFiles] [assuming ${forcingfolder}/ATMOS/[ForcingName]/{HeaderForcingFiles}_YYYY.nc]
• addparams: ';' separated list of additional parameters for run.def as [paramName1]@[values1] (using ! for spaces)

How to run

One located at the desired output folder [OUTfolder]:

1. copy the file or_1-proc.config
2. link the main work-flow manager script: run_multi-or_1proc.bash
3. adapt the content of or_1-proc.config to the experiment
4. run the work-flow (recommending to run it in background as the example)

$ cd [OUTfolder]
$ cp [foldOR1proc]/components/shell/or_1-proc.config ./
$ ln -s [foldOR1proc]/components/shell/run_multi-or_1proc.bash ./
$ vim or_1-proc.config
$ nohup ./run_multi-or_1proc.bash >& run_multi_or_1proc.log &

Whilst experiment is running a new file is created chunk_attempts.inf. In each is listed the number of attempts carried out to simulate a given chunk. If the number is greater than 2, in order to avoid an infinite loop of the work-flow, experiment is stopped.

orout

Along the experiment the work-flow will create in [OUTfolder] a new directory called orout within which will be antother folder with the output of each year of simulation. Work-flow manager looks for the presence of sechiba_rest_out.nc in orout/[year] in order to determine if such chunk should be run.

Inside the folder user will also find:

• run.def: ORCHIDEE's configuration file used to run the chunk
• used_run.def: ORCHIDEE's full configuration (also with the parameters not set in run.def) file used to run the chunk
• out_orchidee_0000: ORCHIDEE's standard output for the simulation
• run_or.log: standard output of the execution of the model